Spatial detection device having a scanning device, spatial detection system, and method for detecting the surroundings

Abstract

The invention relates to a spatial detection device (1) having: at least one scanning device (3) for detecting the surroundings in an emission space, wherein the emission space is formed between two beams (S1, S2) rotating about a central axis (4) and enclosing an opening angle; and a frame (2), which has a lower bearing element (5) and has an upper bracket to which the scanning device (3) is fastened. A scanning pose is defined for the spatial detection device (1), in which scanning pose the scanning device (3) is oriented such that, when the beams (S1, S2) are directed toward the lower bearing element (5) during a rotation with respect to the horizontal direction, the lower beam (S1) of the beams encloses an angle with a horizontal plane (H) in a range from -20° to +20°, and, in the scanning pose of the spatial detection device (1), the distance from the scanning device (3) to the lower bearing element (5) is selected such that the vertical distance (V1) from the lower bearing element (5) to the vertex of the opening angle lies in a range from 18 cm to 68 cm. A support (8) corresponds to the lower bearing element (5) of the spatial detection device (1) such that the lower bearing element (5) can rest on the support (8) in the scanning pose. Cameras (6) are arranged on a circular ring such that the axis of the circular ring coincides with the central axis (4) of the scanning device (3). A trigger button is integrated into a handle (7) directly below the display surface (13). The invention also relates to a spatial detection system having such a spatial detection device, and to a method for detecting the surroundings in an emission space by means of such a spatial detection device.

Description

[0001] NavVis GmbH

[0002] Room sensing device with a scanning device, room sensing system and method for sensing the environment

[0003] The present invention relates to a spatial detection device with at least one scanning device for detecting the environment in an emission chamber, wherein the emission chamber is formed between two beams rotating about a central axis and enclosing an opening angle, and a frame comprising a lower mounting element and an upper support to which the scanning device is attached. The invention further relates to a spatial detection system with such a spatial detection device and a method for detecting the environment in an emission chamber with such a spatial detection device.

[0004] Various acquisition systems for capturing object spaces inside buildings and outdoors are known. The present invention relates in particular to capturing an object space inside a building. Such a system is described, for example, in EP 2 913 796 A1. In this case, a laser scanner is used in conjunction with several cameras. A point cloud is generated from the signals of the laser scanner and the images from the cameras, from which a three-dimensional building model is created.

[0005] For outdoor use, comparable data acquisition systems exist that can be mounted on vehicles and aircraft. In these systems, the reference of the acquired data to a coordinate system is usually achieved by determining the position using satellite navigation systems.

[0006] Inside buildings, this method of positioning is not possible because there is no signal connection to navigation satellites. Furthermore, satellite navigation is too imprecise for determining the position of an object within a given space. For this reason, even outdoors, radodometry, laser odometry, or inertial navigation systems (INS) are used as supplementary methods for positioning. Satellite navigation plays a role in georeferencing and reducing long-term drift.

[0007] For determining position within buildings during mobile surveying of object spaces, the fastest possible real-time positioning is particularly important in order to display a representation of the location to the system operator on a screen.

[0008] M / NAVVTR-030-PC NavVis GmbH

[0009] - 2 -

[0010] to be able to deliver real-time information about the recording process in the environment, so that it can control the recording process in such a way that the building interior is scanned as completely and in high quality as possible.

[0011] Furthermore, it is necessary that the most precise possible subsequent positioning over time, i.e., the determination of the trajectory during object space acquisition, is possible during post-processing. Only in this case can the continuously acquired measurements from the laser scanners and the panoramic images, typically captured at intervals of a few meters, be combined into a precise, consistent 3D model of the building, for example, by creating a point cloud or a polygon mesh.

[0012] The various methods for determining position and trajectory are discussed below. First, a description of the data acquisition methods and application scenarios follows:

[0013] When capturing point clouds using laser scanners, systems are typically employed in which a laser beam is emitted in a plane in space by a mirror rotating around an axis. Alternatively, phased-array lasers without moving parts can be used to generate a scanning laser beam.

[0014] The data provided typically contains, for each data element (point in the point cloud), the respective timestamp of the emitted laser pulse along with its corresponding angular position within the rotation axis. Furthermore, each of these data elements contains one or more values ​​derived from one or more successively received reflection signals. These values ​​indicate the distance of the reflecting surfaces in the direction of the emitted beam, calculated from the laser light travel time, as well as the corresponding reflection intensities. Semi-transparent or semi-reflective surfaces can result in several reflection signals being received in quick succession, which then correspond to surfaces at different distances.

[0015] Distances are calculated from the received reflection signals. These distances, along with the intensities of the reflection signals, are used to calculate three-dimensional point coordinates, which then form the point cloud. To create a consistent three-dimensional model from the data acquisition using the moving laser scanner, a timestamp and the exact positional orientation of the laser scanner in space are recorded for each measurement.

[0016] M / NAWTR-030-PC NavVis GmbH

[0017] - 3 -

[0018] The situation is similar with the image information from panoramic cameras, which generally consist only of image files time-stamped. Here, too, the exact position and orientation of the respective camera in space must be known or determined for each timestamp and each image file so that—using known or calibration-determined camera parameters, such as lens focal length and imaging characteristics, as well as sensor size and resolution—the image data and the point cloud data can be correlated. In this way, an object space can be captured three-dimensionally.

[0019] Panoramic images can also be used to create a highly realistic virtual tour of the captured object space. Here, the focus is on the image files, which, using a process called "stitching" and 3D information (the position and orientation of each camera in space), can be stitched together to create seamless 360-degree panoramas. These panoramas accurately represent the view from a specific point in the environment, as perceived by an observer on-site. The totality of the panoramic images represents a multitude of individual, discrete positions from which the underlying images were taken. Unlike the point cloud model mentioned above, which can be continuously "flown through," the viewer can only jump from one discrete position to another and switch between panoramic images.The point cloud model, available as background information, can be used to animate the transitions between individual panoramic images as blends of differently transformed individual sections (for example, table surfaces) in such a way that the viewer gets the impression of a relatively fluid movement in 3D space between the two discrete positions. The point cloud model also offers further possibilities, such as overlaying the point cloud onto the photo panorama view or assigning a precise 3D coordinate to each pixel of the panorama image (which, for example, enables length measurements of captured objects by clicking on the boundary points in the panorama image, as well as the overlaying of location-based information ("points of interest") into the panorama images).

[0020] For smaller buildings, capturing the interior environment by simultaneously acquiring point cloud data and panoramic images is also possible using stationary, tripod-mounted devices that are moved from position to position. The positions can be aligned, for example, with fixed reference points and markers in the room.

[0021] M / NAVVTR-030-PC NavVis GmbH

[0022] - 4 - which are also found in pre-existing plans, making the allocation easier.

[0023] For the rapid surveying of large buildings, especially their interiors, continuous data acquisition using a mobile system is advantageous. For this purpose, for example, wheeled devices in a trolley design are used, which are pushed by an operator. A mobile frame provides greater stability in this case. Therefore, shake-free images can be captured when the device is stationary. Furthermore, larger and heavier, higher-quality camera lenses, laser scanners, electronic components, and energy storage devices can be attached to the wheeled device and thus moved very easily. With all the aforementioned mobile data acquisition systems, the challenge lies—as explained above—in that the trajectory, and for systems that allow visual monitoring of the data acquisition process on a screen, the instantaneous position, must be determined efficiently and precisely in real time.

[0024] Various methods can be used for this purpose, and these can also be combined. One option is inertial measurement units (IMUs) for inertial navigation, which combine one or more inertial sensors, such as accelerometers and gyroscopes. However, a problem with this approach is that measurement errors accumulate, which can lead to significant drift. For this reason, IMUs are often only used as a supplementary tool. The same applies to odometers.

[0025] In practice, mobile systems also employ so-called SLAM methods ("Simultaneous Localization and Mapping"). These are based on the assumption that the captured environment is static and only the capture system itself moves. In the case of a laser scanner, for example, the data captured from one laser mirror rotation pass are compared with that from one or more previous passes.Assuming the environment is static and the scanning system moved linearly parallel to the laser scan plane, the two sets of points from the two measurement passes would be more or less congruent within measurement tolerances, but shifted translationally and / or rotationally. This would directly and simultaneously generate a profile of the environment as a 2D section through 3D space (corresponding to the laser scanner plane) and, at the same time, a profile of the movement / rotation of the scanning system within this 2D section (hence the term "Simultaneous Localization and Mapping"). In practice, however, the movement, and especially the rotation, must not be too rapid relative to the scan frequency.

[0026] M / NAWTR-030-PC NavVis GmbH

[0027] - 5 -

[0028] The algorithmic assignment of temporally separated measurement points to identical, repeatedly scanned environmental features, and from this the determination of the trajectory of the acquisition system and the creation of an overall model of the environment, is possible with a sufficient number and redundancy of measurement points, even if the laser scanner's acquisition direction changes over time and is arbitrarily arranged relative to the movement of the acquisition system. However, depending on the size and distribution of the point cloud and features in space, this can require very long computation times, so that these methods can generally only be used in post-processing at a high level of detail, but not for the real-time representation of the movement in space during the acquisition process.For example, with the stationary, tripod-mounted solutions mentioned above, it is common to upload the captured data from the individual scan positions to a cloud-based data center and have it combined there in a post-processing step to create a consistent model.

[0029] Photogrammetric methods are comparable, as they allow the creation of a textured 3D model from a large number of images taken of the same object or environment from different perspectives. This is achieved, for example, by using bundle adjustment techniques. In this process, the positions of the points in 3D space, the positions and orientations of the observing cameras, and their internal calibration parameters are simultaneously adjusted to the measured images. These methods produce good results for well-textured surfaces but fail with uniformly colored, featureless surfaces, as well as with more complex intersections and reflective objects.

[0030] In so-called virtual reality or augmented reality applications, which can also be run on mobile phones (smartphones), there are solutions that function similarly to the SLAM or photogrammetric methods. These methods analyze image sequences captured in real time by smartphone cameras to track environmental features over time, usually supported by measurement data from the IMUs also built into smartphones. This allows for a rough, real-time mapping of the environment and the smartphone's movement in space, which then enables, for example, the precise overlay of virtual objects onto the camera viewfinder image.

[0031] M / NAVVTR-030-PC NavVis GmbH

[0032] - 6 -

[0033] For smaller rooms and short distances, so-called "structured light" solutions are also suitable, in which (infrared) dot patterns are emitted from the detection system, the distortion of which in the camera image provides information about the 3D structure of the captured scene.

[0034] Furthermore, so-called time-of-flight cameras are known, which, similar to a parallel laser scanner, emit a flash of light and very precisely determine the individual time at which the reflection signal is captured for each pixel of the camera sensor, so that distance information for the respective pixel is derived from the light's travel time. However, due to their low resolution and limited range and precision, these systems are not suitable for the detailed mapping of large buildings.

[0035] The same applies to stereo depth cameras, which, similar to the human eye, derive depth information from the parallax information of two camera images. Here too, the precision and resolution are insufficient for surveying applications.

[0036] Laser scanners are therefore particularly suitable for high-precision scanning systems that are intended to scan larger buildings with an accuracy of a few millimeters (e.g. trolley-based mobile mapping systems).

[0037] With these mobile mapping systems, real-time visualization of the data acquisition process and movement in space on an operator screen can be particularly simple, robust, and fast if—as shown in the example above—a 2D laser scanner scans in a plane that remains constant during movement. This means that the data acquisition system also moves in a parallel 2D plane, as is the case in buildings with flat floors in the rooms and corridors. In this case, it is also referred to as 2D-SLAM or real-time 2D-SLAM with three degrees of freedom (3 DoF) (i.e., two spatial axes XY and one rotational axis—yaw).

[0038] Since the aforementioned laser scanner, designed for the 2D-SLAM process, is horizontally aligned during movement through the room and always scans the same constant plane, and does not cover the entire area of ​​the room itself, additional 2D laser scanners are used to capture the actual point cloud. These scanners are arranged in different planes so that, as the capture system moves, these scan planes cover the room evenly, ensuring that the environment is scanned and captured as evenly and completely as possible.

[0039] M / NAVVTR-030-PC NavVis GmbH

[0040] - 7 -

[0041] When capturing large buildings, it is desirable to capture as much area as possible in a single, uninterrupted scan to minimize the effort required for registration—that is, merging partial point cloud models from individual scans into a complete point cloud model by precisely aligning and matching the overlapping areas of the partial point clouds. While this registration process is theoretically possible algorithmically, it can be computationally intensive depending on the size of the partial models and may still require manual pre- or post-adjustment.

[0042] Trolley-based mobile mapping systems that use 2D-SLAM methods generally require the current scan process to be terminated and a new scan process to be started as soon as, for example, a larger step, steeper ramp or even stairs have to be overcome, even though individual systems are able to process ramps with low gradients or compensate for disturbances caused by bumps, run-over cables, etc. using correction algorithms, for example by evaluating IMU data.

[0043] Furthermore, detection systems with six degrees of freedom (6 DoF) (i.e., three spatial directions XYZ and three rotation directions ("roll-pitch-yaw" / "roll-pitch-yaw" / 6DoF-SLAM method) are known.

[0044] For example, the publication by George Vosselman, “DESIGN OF AN INDOOR MAPPING SYSTEM USING THREE 2D LASER SCANNERS AND 6 DOF SLAM”, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume 11-3, 2014. ISPRS Technical Commission III Symposium, September 5-7, 2014, Zurich, Switzerland. 10.5194 / isprsannals-ll-3-173-2014 (https: / / www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net / ll-3 / 173 / 2014 / isprsannals-ll-3-173-2014.pdf), describes a method for mapping the space of an object within a building. This method uses multiple single-plane scanners whose scan planes are not parallel to each other. However, the processing of the data captured by this system is algorithmically very complex, so this method is not suitable for real-time visualization of the scanning process, but only for calculating a point cloud model in post-processing.Furthermore, a six-degrees-of-freedom acquisition system using 3D-SLAM technology is known from EP 3 228 985 A1.

[0045] Various laser scanners are known from DE 10 2011 121 115 B4 or DE 10 2004 050 682 A1. Furthermore, a multi-scanner is known from EP 2 388 615 A1 and US 2017 / 0269215 A1, which emits signal beams in a fan shape and measures the reflections of these signal beams.

[0046] M / NAVVTR-030-PC NavVis GmbH

[0047] - 8 -

[0048] Trolley-based systems are particularly suitable when the building to be scanned has a level surface, allowing the system to be rolled across it. In buildings with uneven surfaces, or those with numerous stairs, steps, steep ramps, or complex layouts where a trolley-based system cannot be pushed over long distances and would frequently require readjustment due to changes in elevation, portable backpack-style systems or handheld systems in various designs are available. These systems are subject to weight restrictions. The weight must be low enough to be carried or held by a single person. Furthermore, the scanning devices must be designed to generate sharp, shake-free images despite the movements made by the user with the scanner.

[0049] US 6,141,034 discloses a room detection device comprising a scanning device and a frame that can be carried on a person's shoulders or by means of a wheelchair.

[0050] Finally, a room detection device with a scanning unit is known from WO 2021 / 018900 A1.

[0051] The invention is based on the objective of providing a room detection device and a room detection system of the type mentioned above, in which the room detection device can be carried by a person through a building, and the room can be detected during movement through the building. Particularly good handling by the person carrying the frame is to be ensured. Furthermore, a method for detecting the environment in an emission room with such a room detection device is to be provided.

[0052] According to the invention, this problem is solved by a space detection device with the features of claim 1, a space detection system with the features of claim 16, and a method with the features of claim 24. Advantageous embodiments and further developments are described in the dependent claims.

[0053] The spatial detection device according to the invention comprises at least one scanning device for detecting the environment in an emission space, wherein the emission space is formed between two beams rotating about a central axis and enclosing an opening angle, and

[0054] M / NAVVTR-030-PC NavVis GmbH

[0055] - 9 - a frame comprising a lower mounting element and an upper support to which the scanning device is attached. According to the invention, a scan pose is defined for the room sensing device in which the scanning device is oriented such that the lower beam of the beams, when the beams are directed towards the lower mounting element during a rotation with respect to the horizontal direction, forms an angle with a horizontal plane in a range of -20° to +20°. In the scan pose of the room sensing device, the distance of the scanning device from the lower mounting element is selected such that the vertical distance of the lower mounting element from the apex of the opening angle is in a range of 18 cm to 68 cm.

[0056] The emission chamber formed by the scanning device has the shape of a double fan when viewed in cross-section along the central axis. When the double fan is rotated around the central axis, the emission chamber is formed approximately in the shape of a sphere from which a double cone has been cut out, with the central axis as the cone's axis. During the rotation of the beams that enclose the opening angle and form the emission chamber, two angular positions result: one where the beams are directed horizontally towards the lower component of the system, and the other where they are directed horizontally away from the lower component.

[0057] The horizontal direction refers to the component of the vector defining the ray that is horizontally oriented in the scan pose of the spatial acquisition direction. The projection of the ray onto the horizontal plane in this angular position points in the direction of the system element if the ray is directed towards the lower system element with respect to the horizontal direction. Conversely, if the ray is directed away from the lower system element with respect to the horizontal direction, the projection of the ray onto the horizontal plane points away from the lower system element.

[0058] In this document, the term "scan pose" refers to the position and orientation of the spatial sensing device. In this scan pose, the spatial sensing device can capture the surroundings. The scan pose is defined by the orientation of the lower beam of the sensors when the sensors, during rotation with respect to the horizontal direction, are directed towards the lower mounting element. If the angle of the lower beam with the horizontal plane lies within a range of -20° to +20°, the scanning device is oriented such that it is in the scan pose.

[0059] M / NAVVTR-030-PC NavVis GmbH

[0060] - 10 -

[0061] In this text, negative angles between a ray and the horizontal plane are those angles that extend downwards from the horizontal plane. Positive angles, on the other hand, extend upwards from the horizontal plane. An angle of 0° with the horizontal plane means that the ray is horizontally oriented.

[0062] The spatial detection device according to the invention has a specific vertical extent in the scan position. Specifically, in the scan position, the distance of the scanning device from the lower support element is selected such that the vertical distance of the lower support element from the apex of the opening angle lies within a range of 18 cm to 68 cm. This limits the height of the scanning device in the scan position.

[0063] In this document, a vertical distance refers to a distance in the vertical direction. Two elements that have a vertical distance can also have a horizontal distance. The vertical distance is the distance when projected onto a vertically oriented plane, whereas the horizontal distance is the distance when projected onto a horizontally oriented plane.

[0064] The angle formed by the lower ray of rays with the horizontal plane when the rays are directed towards the lower mounting element during rotation with respect to the horizontal direction is particularly in a range of -20° to +20°, preferably in a range of -10° to +10° and most preferably this angle is 0°, i.e. the lower ray is horizontally oriented in this case.

[0065] The vertical distance of the lower support element from the apex of the opening angle is, for example, in a range of 28 cm to 58 cm, and particularly in a range of 37 cm to 46 cm. Preferably, this vertical distance is 43 cm.

[0066] According to a further development of the spatial detection device according to the invention, the scanning device in the scan position is aligned such that the upper beam of the beams, when the beams are directed towards the lower mounting element during a rotation with respect to the horizontal direction, forms an angle with the horizontal plane in a range of 0° to +60°. This angle is, for example, in a range of +10° to +50°, more particularly in a range of +20° to +45°, and preferably this angle is +40°.

[0067] M / NAVVTR-030-PC NavVis GmbH

[0068] - 11 -

[0069] The opening angle between the lower beam and the upper beam is, for example, in a range of 20° to 60°, particularly in a range of 30° to 50°, and preferably this opening angle is 40°.

[0070] According to a further development of the spatial detection device according to the invention, in the scan position the scanning device is aligned such that the bisector of the beams, when directed towards the lower mounting element during a rotation with respect to the horizontal direction, forms an angle with the horizontal plane in a range of 0° to 40°. This angle is, for example, in a range of +10° to +30°, particularly in a range of +15° to +25°, and preferably +20°.

[0071] According to a further development of the space detection device according to the invention, in the scan pose, in which an average-sized person wears the space detection device in such a way that the lower attachment element rests vertically in the area of ​​the hip, the scanning device is located below the person's head.

[0072] The height of an average person ranges from 165 cm to 185 cm, particularly from 170 cm to 180 cm. For example, the height of an average person is considered to be 175 cm. A position below the person's head is understood to mean that the position is below the person's height, i.e., the top of their head.

[0073] The scanning device, in particular the apex of the opening angle of the beams emitted by the scanning device, is thus located below the top, i.e., the crown, of the person's head. Furthermore, the scanning device, in particular the apex of the opening angle of the beams emitted by the scanning device, is located above the top of the person's shoulders.

[0074] According to a further development of the space detection device according to the invention, the frame has a handle which, in the scan position, is arranged horizontally between the lower support element and the scanning device at a horizontal distance from the lower support element that lies in a range of 10 cm to 30 cm. This horizontal distance is particularly in a range of 12 cm to 22 cm, preferably in a range of 15 cm to 18 cm. The horizontal distance is measured particularly at the center of the handle.

[0075] M / NAWTR-030-PC NavVis GmbH

[0076] - 12 -

[0077] According to a further development of the space detection device according to the invention, the handle in the scan position is arranged vertically between the lower support element and the scanning device at a vertical distance from the lower support element that lies in a range of 12 cm to 35 cm. This vertical distance is particularly in a range of 15 cm to 31 cm, preferably in a range of 21 cm to 25 cm. The vertical distance is measured particularly at the center of the handle.

[0078] The handle features an integrated trigger button located directly below the display. This allows for a natural grip position for both left- and right-handed users. The trigger button can be easily operated with the thumb, which is ergonomically advantageous.

[0079] According to a further development of the space detection device according to the invention, the space detection device has a display device with a display surface whose normal in the scan position forms an angle with the horizontal plane in a range of 24° to 44°. This angle is particularly in a range of 29° to 39°, preferably in a range of 33° to 36°.

[0080] According to a further development of the space detection device according to the invention, the display surface in the scan position is arranged at a horizontal distance from the lower mounting element that lies in a range of 12 cm to 35 cm. This horizontal distance is particularly in a range of 15 cm to 31 cm, preferably in a range of 21 cm to 25 cm. The horizontal distance is measured particularly at the center of the display surface.

[0081] The display surface is oriented in landscape format and positioned below the cameras. This ensures that the display surface is relatively high when the room scanning device is worn by a person, providing them with a clear view of the display. The display surface is positioned approximately 40 cm from the person's eyes, with their gaze directed downwards. Preferably, the downward angle allows the person, in the scan position, to see their own face reflected in the surface, thus enabling easy visual confirmation of the correct scan position. This arrangement of the display surface allows the person to maintain both a clear view of the display and a good view of the immediate surroundings in front of the room scanning device.This will happen.

[0082] M / NAVVTR-030-PC NavVis GmbH

[0083] - 13 - increases safety when wearing the room detection device and when moving with the room detection device.

[0084] According to a further development of the spatial detection device according to the invention, the scanning device is a multi-scanner, in particular a LiDAR sensor. The LiDAR is in particular a 32-layer LiDAR sensor.

[0085] The multi-scanner comprises, in particular, a multitude of emission units integrated into a single component for generating a multitude of signal beams in defined emission directions, a receiver for detecting reflected radiation generated by reflections of the signal beams from one or more objects in the object space, and a scanning device for changing the emission directions of the signal beams. The use of the multi-scanner enables uninterrupted acquisition of the object space. Specifically, the multi-scanner allows for the use of a real-time 3D SLAM method with six degrees of freedom. It is not necessary to divide the acquisition process into sub-processes and reassemble these sub-processes in post-processing.

[0086] The use of the multi-scanner in the spatial detection device according to the invention offers the advantage that, during movement, the device not only detects new surface sections of the object space by having the signal beams sweep over these sections, but that the signal beams also always encounter previously detected surface sections, i.e., surface sections that have already been detected by other previously emitted signal beams. This makes it possible to compare the reflected radiation detected by the receiver with previously detected reflected radiation. From this comparison, the movement of the spatial detection device can then be calculated, thus making it possible to determine the position of the spatial detection device in the object space.This, in turn, makes it possible to generate and output a graphical representation of those areas of object space through which the spatial detection device was moved. From this, a preliminary model of object space, based on data that can be obtained at least from the reflected radiation, allows us to determine through which areas of object space the spatial detection device can move. These possible movements of the device within object space can also be graphically represented and output.

[0087] M / NAVVTR-030-PC NavVis GmbH

[0088] - 14 -

[0089] The emission directions of the multi-scanner are, in particular, fan-shaped, forming an emission fan with a central axis. The opening angle of the emission fan can, in particular, be in a range of 20° to 60°. A preferred opening angle is 40°.

[0090] The emission units of the multi-scanner are, in particular, one or more lasers. The signal beams can be emitted simultaneously by several lasers in a fan-shaped pattern in the emission directions. Preferably, however, laser pulses (signal pulses) are emitted sequentially in the emission directions, so that the fan-shaped emission of the signal beams in the emission directions only occurs when considering a specific time interval. The laser pulses in the emission directions can be emitted by a laser whose emission direction is changed. Preferably, however, several lasers are used that emit pulses sequentially in different emission directions. The intervals between the pulses can be selected such that the reflection of the laser pulse is detected before the next laser pulse is emitted.Thus, the time interval between the laser pulses depends on the range that the signal beams are intended to achieve for detecting the object space.

[0091] The scanning device is specifically designed to rotate the emission directions of the signal beams around a rotational axis. The multi-scanner thus scans the volume of the rotating body of a fan, i.e., the emission space. According to a preferred embodiment, the multi-scanner is mounted on the frame such that the rotational axis is inclined forward at an angle to the vertical when the person is carrying the room-sensing device on the frame. This angle is preferably 20°. This ensures that, in the direction in which the rotational axis is tilted, closer surface sections of the floor on which the person carrying the room-sensing device is moving can be detected. In the opposite direction, the fan-shaped emission is tilted upwards, so that fewer areas below the multi-scanner are illuminated.

[0092] The rotation axis of the multi-scanner is thus tilted forward, particularly with respect to the direction of movement of the spatial scanning device. This tilting of the rotation axis is also advantageous for the real-time 3D SLAM process. In this case, not only are precisely horizontal cross-sections through the object space provided for real-time visualization, but also cross-sections perpendicular to the direction of movement.

[0093] M / NAVVTR-030-PC NavVis GmbH

[0094] - 15 -

[0095] This allows the information necessary for the SLAM process to be captured, meaning that recurring environmental features captured in successive rotations of the multi-scanner, which is primarily a laser scanner, can be identified. For example, an environmental feature captured in the first scan layer of the multi-scanner during a rotation could reappear in the data set of the next or subsequent layer of the scanner during the following rotation.

[0096] On the other hand, this also allows for the rapid capture of large areas for 3D visualization by the operator, including, in particular, nearby features of the floor in front of the room capture device and more distant features of the ceiling behind it. Specifically, it enables a 3D visualization of the captured environment, providing more detail than a multi-slice line section representation, which is preferred when rapid, real-time capture of large areas is required, especially with a long forward range in the direction of travel, as is relevant, for example, in autonomous driving.

[0097] According to a further development of the room detection device according to the invention, the room detection device further comprises at least one camera, in particular several cameras, which are attached to the frame between the scanning device and the handle.

[0098] The camera(s) mounted on the frame are specifically designed to capture images of at least a portion of the object space. The images captured by the camera(s) can be incorporated into the real-time generation of various graphical representations of the object space. This allows for the creation of a highly realistic, real-time representation of the area of ​​the object space that has been entered and / or is accessible, or of the area of ​​the object space already scanned by the spatial detection device.

[0099] According to a further development of the spatial detection device according to the invention, images can be recorded with the cameras in the scan position at a recording angle that lies in a range of 180° to 360°, the recording angle being particularly 270°. In this case, a person wearing the spatial detection device in the scan position is positioned in the area not covered by the recording angle.

[0100] M / NAVVTR-030-PC NavVis GmbH

[0101] - 16 -

[0102] In a vertical plane, the recording angle of the camera(s) lies, for example, in a range of 180° to 360°, particularly in a range of 230° to 300°, whereby the person is predominantly outside the recording angle, and in particular completely outside the recording angle, of the camera(s) when the person is wearing the room detection device. A recording angle of 270° is also preferred in the vertical plane. The camera(s) thus enable a substantially complete recording of the area in front of the person, and in particular around the person, wherein the camera(s) are arranged such that the person wearing the room detection device is predominantly outside the recording area, and in particular completely outside the recording angle, of the camera(s).

[0103] According to a further development of the spatial detection device according to the invention, the cameras are arranged spaced apart on a ring, in particular a circular ring, wherein the axis of the ring is aligned parallel to the central axis of the scanning device.

[0104] The distance between the cameras is chosen to be as small as possible, so that the area captured by the cameras in the immediate vicinity of the person wearing the room-capturing device is as large as possible, while the person is predominantly outside the recording angle, and in particular completely outside the recording angle, of the cameras when wearing the room-capturing device.

[0105] The room-capture device according to the invention can further be raised from the scan position into an overhead position, in which images can be captured by the camera(s) when the trigger button is pressed. Advantageously, in this case, a 360° all-round image is possible using the cameras, which, however, excludes the area of ​​the room in which the person wearing the room-capture device is located.

[0106] The multi-scanner is positioned above the camera(s) in the scan position. A minimal distance between the multi-scanner and the cameras advantageously results in the lowest possible parallax error between the point cloud data captured by the scanner and the image data simultaneously captured by the cameras. This is beneficial when colorizing the point cloud data using image data ("point cloud coloring").

[0107] The number, spacing, angular alignment, positioning and field of view, as well as the focal length of the cameras, and optionally also of the multi-scanner, are determined according to the invention.

[0108] M / NAVVTR-030-PC NavVis GmbH

[0109] - 17 -

[0110] The spatial recording device is optimized to allow for a natural recording height, essentially corresponding to the viewing height of an upright human observer of the surroundings, while simultaneously keeping the so-called baseline (i.e., the camera distance) low, thus reducing stitching artifacts. Positioning the camera below the top of the person's head also serves as protection against accidental collisions with objects located above the head.

[0111] Preferably, four cameras are used to capture panoramic images. Additionally, two global shutter cameras are used to stabilize the SLAM scanning process by employing visual odometry in areas that lack distinguishable geometric features for the SLAM scanning process.

[0112] The panoramic cameras are arranged to maximize their overlap with adjacent cameras. This arrangement also creates a blind zone where the room sensing device and the person carrying it may be obscured during the scanning process. Two of the cameras used for visual odometry have lenses with a very wide field of view, specifically exceeding 180°. They are oriented forward and to the side to capture as many visual features as possible that can be used for visual odometry. These two cameras record a video stream that can be used in subsequent data analysis to estimate the movement of the room sensing device. These cameras are preferably designed as global shutter cameras.

[0113] According to a further development, the space detection device according to the invention comprises an energy storage device, a data storage device, a control unit and / or a computing unit, which are arranged in or on the frame in the scan position below the handle.

[0114] Positioning the energy storage device, data storage device, control unit, and / or processing unit in the lower section of the room sensing device lowers its center of gravity and brings it closer to the user's body. This limits the weight and torque that must be held by hand.

[0115] The energy and data storage devices, as well as a power switch, are preferably accessible by opening a flap on the underside of the room detection device.

[0116] M / NAVVTR-030-PC NavVis GmbH

[0117] - 18 -

[0118] The control unit is specifically linked to the cameras via data transmission, and is configured to trigger image capture by the cameras simultaneously. This makes it possible to create a 360-degree panorama using the cameras at a specific point in time.

[0119] According to a further embodiment of the room detection device according to the invention, an evaluation unit is attached in the lower region of the room detection device. This evaluation unit is data-connected to the cameras and / or the multi-scanner and / or WLAN / Wi-Fi antennas and / or an inertial measurement unit, with data transmission occurring via a bundling and synchronization interface. For example, a USB cable can be provided between the cameras and the bundling and synchronization interface, and an Ethernet cable can be provided between the first multi-scanner and the bundling and synchronization interface. Likewise, a USB cable or Ethernet cable can be provided between the WLAN / Wi-Fi antennas and the inertial measurement unit and the bundling and synchronization interface.The bundling and synchronization interface links the received data with precise timestamps it generates itself and simultaneously converts the received data so that further data transmission can take place via a single USB or Ethernet cable. The bundling and synchronization interface is then connected to the evaluation unit via this single USB or Ethernet cable, enabling data transmission from the bundling and synchronization interface to the evaluation unit. Only a single data transmission cable is required from the bundling and synchronization interface to the evaluation unit, thus acting as a shared data transmission cable, which can be routed, in particular, within the conduits of the frame.The immediate linking of data received from the various units with precise timestamps, which takes place within the bundling and synchronization interface, minimizes any timing and latency problems associated with buffering and intermediate storage on the subsequent data transmission path to the evaluation unit and during the processing of the incoming data there. The SLAM algorithms can thus rely on the timestamps of the received data assigned with minimal latency by the bundling and synchronization interface close to the respective data sources. This ensures that even under heavy load on the evaluation unit, any delays or jumps in the processing order do not lead to consistency or precision problems.

[0120] M / NAVVTR-030-PC NavVis GmbH

[0121] - 19 -

[0122] Advantageously, the evaluation unit is coupled with the display unit for displaying the graphical representations generated by the evaluation unit.

[0123] The evaluation unit is specifically designed to calculate, at least from data generated by detected reflected radiation captured by the receiver(s), the position of the spatial detection device within the object space and / or its trajectory in real time, and to output this information graphically. In this way, the area of ​​the object space that has been entered and / or is accessible, the area already scanned, or the area not yet scanned can be displayed in a comprehensive representation of the object space based on the data already acquired. The user can thus identify, based on the graphical representation, which areas of the object space still need to be scanned.In addition, the quality of the recorded data for each recorded room area can be displayed in real time in a so-called "Live Quality Map", so that the person can assess which room areas may need to be recorded better.

[0124] Finally, the room detection device can have a data interface that is at least technically coupled to the evaluation unit and is designed to output data to a storage device for post-processing.

[0125] According to a further development, the spatial detection device according to the invention has at least one reference marking, for example in the form of an arrow or a notch, for detecting the position and / or orientation of the spatial detection device relative to a stationary coordinate system. The reference marking is formed, in particular, by a foldable hook.

[0126] The fixed coordinate system can be equipped with corresponding reference markers, which in turn are attached to specific points in the environment to be scanned, for example, as stickers on the floors or walls of a building to be scanned. These markers are marked with a crosshair and a unique designation or identification number (hereinafter referred to as "Ground Control Point"). During an ongoing scan, the room scanning device is briefly held, for example, with the arrowhead of its reference marker precisely against the crosshair of such a "Ground Control Point," and the time and the unique designation or identification number of the "Ground Control Point" are recorded. Recording the exact time can be achieved using a particularly ergonomically advantageous method.

[0127] M / NAVVTR-030-PC NavVis GmbH

[0128] - 20 - a separate actuation button is attached, which is pressed as soon as the arrowhead and crosshair marker touch. For example, it is ergonomically advantageous if the actuation button is easily accessible while the room detection device is standing upright on the floor with the reference marker, or being held while touching the "Ground Control Point", or if the actuation button is reachable with one finger while the room detection device is held against the wall with two hands using the handle with the reference marker.

[0129] The use of "ground control points" enables particularly precise trajectory determination using SLAM methods, as necessary correction parameters can be determined, for example, due to accumulated drift during longer scans. Furthermore, data sets from adjacent building sections, acquired in separate scans and overlapping in the area of ​​the ground control points, can later be aligned with exceptional precision and combined into a unified data model. Preferably, the ground control points are located at precisely defined points, determined, for example, from building plans or measured using conventional surveying methods. This makes it possible to align or compare the data acquired with the inventive spatial sensing device with exceptional precision against other known or acquired data.

[0130] Alternatively, the ground control points can also be detected by the cameras of the room detection device if these cameras have a unique identifier, for example, in the form of a QR code. Furthermore, it is also possible to provide a laser pointer on the room detection device, which can be used to detect a ground control point without contact by, for example, directing the laser dot of the laser pointer at a defined distance onto the crosshairs of the sticker.

[0131] According to a further development of the spatial detection device according to the invention, it further comprises at least one receiving antenna and / or an inertial measuring unit. The receiving antenna serves to receive signals from transmitting stations of a wireless local area network (WLAN / WiFi). Furthermore, another receiving antenna can be provided, which serves to receive signals from Bluetooth transmitters. The receiving antenna(s) are coupled to a signal processing unit. In this way, signals from wireless local area networks and / or Bluetooth signals, along with their respective received strength, can be acquired and stored while the spatial detection device is moving. From these acquired signals, a spatial profile of the ID identifiers of the transmitting stations can be generated.

[0132] M / NAVVTR-030-PC NavVis GmbH

[0133] - 21 -

[0134] (WLAN / WiFi SSIDs, Bluetooth beacons) and their spatially varying signal strengths are created. This is known as WLAN / WiFi or Bluetooth "fingerprinting." The recorded profile of ID identifiers and signal strengths can be used to later locate mobile phones moving through a room. For example, an indoor navigation app on a smartphone can capture the ID identifiers of the transmitting stations in real time and measure their signal strengths. This data is then compared, for example, via a server-based database with the profile previously recorded by the room detection device to derive a location.

[0135] The inertial measurement unit (IMU) can include accelerometers and gyroscopes. By evaluating the signals from the IMU, the movement of the spatial sensing device in space, and thus the temporal evolution of its position in space, can be determined.

[0136] According to a further development of the room detection device according to the invention, it further comprises a status sensor which is designed to detect how the room detection device is worn by a person.

[0137] The state sensor can, for example, interrupt the object space acquisition if it detects that the space-capturing device has been tilted or briefly set down in a specific way, such as to pass through a narrow passage or low doorway, or to open a locked door. This state data can be taken into account during the processing of the acquired scan data, for example, in post-processing, so that point cloud artifacts, which can occur when the laser scanners scan body parts of the person while the space-capturing device is removed or tilted, can be suppressed during data processing.

[0138] Additionally, the status sensor can also detect whether the room sensing device has been stably placed on the floor for a certain period of time. For example, to save energy, data acquisition could be automatically paused when the status sensors detect that the room sensing device is stably placed on the floor, without affecting the consistency of the ongoing SLAM location determination. Setting the room sensing device down is particularly useful when it is being handed over from one operator to another or when, for example, bulky furniture needs to be moved within the environment.

[0139] M / NAVVTR-030-PC NavVis GmbH

[0140] - 22 -

[0141] It is also advantageous if information from the condition sensors can be used to homogenize and optimize the density and quality of the generated point cloud, for example by selectively discarding data, because when the device is removed from the operator's body, the otherwise uniform recording process during normal walking changes towards stronger fluctuations in the quality and density of the point cloud data due to the operator stopping and tilting, turning or shaking the room recording device.

[0142] According to one embodiment of the room sensing device according to the invention, the upper area containing the scanning unit and the cameras is passively cooled. The lower area of ​​the room sensing device features active cooling. For this purpose, air inlets are located on the underside of the room sensing device. The air outlets are located on both sides above the processing unit, so that the airflow follows the natural flow and the warm air rises. The air inlets and outlets are designed in such a way that neither solid objects nor water can penetrate the room sensing device.

[0143] The room detection system according to the invention comprises a room detection device as described above, and a support which corresponds to the lower mounting element in such a way that the lower mounting element can rest on the support in the scan position.

[0144] In the spatial detection system according to the invention, the support is located, in particular, in a vertical area extending from the person's crotch to the person's ribcage. For example, the support is located at the level of the person's hips when the person is wearing the spatial detection device. In this document, the level of a person's hips is understood to be a vertical area extending from the person's crotch to the lower ribs.

[0145] According to a further development of the room detection system according to the invention, the room detection device is pivotable about a mounting axis which is horizontally aligned in the scan pose when the lower mounting element rests on the support.

[0146] According to a further development of the space detection system according to the invention, a hook-like connection can be established between the support and the lower mounting element, in which the space detection device can be pivoted about the mounting axis.

[0147] M / NAVVTR-030-PC NavVis GmbH

[0148] - 23 -

[0149] According to a further development of the space detection system according to the invention, the support interacts with the system element in such a way that the system element can be pivoted about the system axis when it rests on the support, but pivoting about an axis perpendicular to the system axis is hindered.

[0150] According to further training, the room sensing system also has a harness for adjustable attachment of the support to a person wearing the room sensing device, wherein the distance of the scanning device from the lower system element is chosen such that the scanning device is located lower than the top of the head of the person wearing the room sensing device.

[0151] The harness is specifically designed so that the scanning device is positioned approximately just above the upper shoulders, even for people of different sizes.

[0152] According to a further development, the spatial sensing system has a first coupling element, and the harness includes a second coupling element, with which a detachable connection between the harness and the spatial sensing device can be established. One of the coupling elements limits the pivoting movement of the spatial sensing device about the axis of attachment formed between the support and the resting support element, such that the spatial sensing device is held in the scan position. The second coupling element includes, in particular, a retaining strap, for example, a neck strap. The retaining strap can, for example, be connected to the harness. The detachable connection between the first coupling element and the second coupling element can, for example, be realized by a ball pin that can be hooked into place. Preferably, this ball pin is hooked into place by means of a keyhole-shaped recess provided in the handle.In particular, a magnet is provided in combination with an undercut that ensures a secure hold.

[0153] According to a further development of the space detection device according to the invention, the second coupling element comprises a spherical element arranged at the end of a pin-shaped element. The pin-shaped element can, in turn, be held by a retaining element. In this further development, the first coupling element comprises a keyhole-shaped recess, wherein the spherical element can be inserted into the keyhole-shaped recess and held in an undercut. In the undercut, the spherical element thus has a first position, in which it can be inserted and removed through the opening of the keyhole-shaped recess, and a second position, in which

[0154] M / NAVVTR-030-PC NavVis GmbH

[0155] - 24 - which holds it in the undercut, so that the first and second coupling elements are coupled together. The pin-shaped element can be held by a retaining element.

[0156] According to a further development of the space detection device according to the invention, the spherical element contains ferromagnetic material and the first coupling element comprises a magnet arranged such that the spherical element is held in an undercut position by the magnetic interaction between the magnet and the ferromagnetic material of the spherical element. The magnet thus holds the spherical element in the aforementioned second position.

[0157] In particular, the coupling elements are frictionally engaged in all directions of pull that can act on the second coupling element via the retaining strap. Pulling on the retaining strap therefore cannot release the coupling of the coupling elements. On the other hand, the coupling of the coupling elements can advantageously be released very easily with one hand, in particular by pushing the pin-shaped element in a direction that moves the spherical element towards its initial position. For example, a moment or a superposition of two forces can act on the pin-shaped element to release the coupling. The pin-shaped element can, for example, be held between the index finger and thumb of a user to exert the forces or moment.Furthermore, the user can, for example, slide their thumb down the handle and move the pin-shaped element, thus releasing the coupling of the coupling elements. Advantageously, this provides an optimized solution for the portable room sensing device, ensuring both a secure hold when carrying the device and quick and easy operation when disconnecting it from the second coupling element. This is achieved advantageously through the use of various mechanical principles: a bayonet-type connection with a keyhole-shaped recess, a ball joint with a rounded receptacle featuring an undercut in the insertion direction, and a magnet that exerts a magnetic force to hold the second coupling element.

[0158] According to a further development of the space detection device according to the invention, the keyhole-shaped recess is formed by a contact plate. This contact plate is preferably made of metal. The contact plate is encased in a shell, which is made, for example, of plastic. Preferably, the contact plate has a second feature on its back side.

[0159] M / NAVVTR-030-PC NavVis GmbH

[0160] - 25 -

[0161] The contact plate is designed in a position where the spherical element is held in the undercut by the contact plate, preventing its removal. It also features a corresponding receptacle where the surfaces of the spherical element and the contact plate are in full contact. In this second position, the spherical element can also engage with the contact plate in a snap-fit ​​connection. Furthermore, the spherical element is held by the magnet, particularly in this second position. The contact plate is designed such that, after insertion into the keyhole-shaped recess, the spherical element is guided into the second position by sliding action from the rear of the contact plate.

[0162] According to a further development of the space detection device according to the invention, the maximum swivel angle a1 of the second coupling element upwards is 8°, the maximum swivel angle a2 of the second coupling element downwards is 60°, the maximum swivel angle a3 of the second coupling element to one side is 7°, the maximum swivel angle a4 of the second coupling element to the other side is 7° and the rotation angle a5 about the axis of the second coupling element is 360°.

[0163] According to a further development of the space detection device according to the invention, the retaining strap has an elastic section. In particular, the retaining strap is a non-elastic webbing onto which an elastic band has been attached in a stretched state, for example, sewn on. When no load hangs from the retaining strap, the elastic section gathers. The retaining strap has, in particular, two sides extending from the second coupling element. The elastic section is located only on one side, so that the second coupling element does not hang centrally when the retaining strap is gathered. This advantageously ensures that the retaining strap is pulled out of the way laterally when uncoupled and does not obstruct the user's interaction with the centrally arranged operating elements, such as the display device, which would be the case if it were hanging freely.

[0164] When the room scanning device is supported by resting the lower component on the support, a balance can be achieved between high SLAM quality and point cloud coverage. When the user carries the room scanning device in the scan pose using the support and, if necessary, the harness, an optimal compromise is achieved between maximum scan quality (i.e., high accuracy or low drift) and room coverage, as the scanning device is in a forward-tilted position, allowing the beams to project very far into a long corridor.

[0165] M / NAWTR-030-PC NavVis GmbH

[0166] - 26 -

[0167] Advantageously, this minimizes SLAM drift and therefore achieves high accuracy. The forward-tilted position of the scanning device also ensures that surfaces are scanned densely as the user walks past them. This forward tilt minimizes adverse occlusions, allowing for dense scanning and ensuring that the resulting point cloud covers all relevant details.

[0168] According to a further development of the spatial detection system according to the invention, the central axis of the scanning device in the scan position encloses an angle with the vertical direction which lies in a range of 0° to +40°, wherein in this case a positive angle means that the scanning device is tilted forward.

[0169] In the inventive method for detecting the environment in an emission room with the described room detection device, signal beams are emitted into the opening angle by the scanning device in the scan pose of the room detection device, and reflection radiation generated by reflections of the signal beams on one or more objects of the object space is detected.

[0170] According to a further development of the inventive method, the reflections of the signal beams of the scanning device are evaluated and the space detection device is localized by LIDAR-SLAM methods and the localization is supplemented by camera-based visual odometry in areas where the reflections of the signal beams of the scanning device are insufficient.

[0171] Combining SLAM with visual odometry leads to a more accurate understanding of the environment. SLAM estimates the movement of the spatial sensing device in three dimensions with six degrees of freedom. If SLAM is unable to calculate the movement because there is no detection of walls, objects, or surfaces in all directions, visual odometry can assist in estimating the movement of the spatial sensing device.

[0172] Unlike SLAM methods, which measure distances on surfaces, visual odometry extracts features such as corners, edges, or similar elements from each frame of an incoming camera video stream to calculate the movement of the spatial sensing device. This motion estimation can then be combined with the SLAM method, thus maintaining the high accuracy of SLAM while visual odometry provides the motion estimation in a specific way.

[0173] M / NAVVTR-030-PC NavVis GmbH

[0174] - 27 -

[0175] The direction is stabilized in which the SLAM method has limited functionality. By combining complementary information—namely, the data from the scanning device (i.e., distance measurement data) and the visual information from the images captured by the camera—environmental detection is very robust and works in a wide variety of scenarios.

[0176] According to a further development of the method according to the invention, relocalization is performed using a LIDAR-based SLAM method. If such relocalization is not possible, relocalization is carried out using visual odometry.

[0177] According to a further development of the inventive method, point cloud data is recorded by the scanning device and images are recorded by the at least one camera, and in a post-processing step the point cloud data is colored using the images recorded by the camera, so that colored point cloud data is generated.

[0178] According to a further aspect of the method according to the invention, point cloud data is captured by the scanning device and images are captured by the at least one camera in such a way that the person wearing the spatial detection device is excluded. In post-processing, the images captured by the camera are stitched together, in particular to form panoramic images, preferably 360° panoramic images. In post-processing, the images captured by the camera are supplemented in the excluded area using point cloud data, in particular colorized point cloud data, and / or using image-based rendering methods from the available images of the surroundings. The supplementation is carried out in such a way that the space behind the person is displayed in the excluded area as if the person had not been in the excluded area and as if the area had not been excluded but had also been captured.

[0179] The aforementioned addition can be achieved alternatively or in combination using point cloud data, in particular colored point cloud data, using so-called "point-based rendering" methods, or with the help of image-based data using so-called "image-based rendering" methods from the available images of the environment.

[0180] Other known rendering and reconstruction methods can also be combined, for example, methods for creating 2D images from 3D data ("volume rendering") such as Gaussian splatting or methods for reconstructing 3D data from 2D data.

[0181] M / NAVVTR-030-PC NavVis GmbH

[0182] - 28 -

[0183] Images such as Neural Radiance Fields or methods using Neural Networks or AI technologies.

[0184] The masking can be achieved alternatively or in combination by discarding the image data from one or more cameras or by software-based selective masking of the image data.

[0185] According to a further development of the inventive method, automatic person detection is performed in the recorded images, and the image of the person closest to the camera is identified. This area of ​​the image, which shows the image of the person closest to the camera, is then masked. Optionally, this area can be filled, e.g., based on point cloud data.

[0186] The dimensions and orientation of the room sensing device and the room sensing system according to the invention are advantageously selected such that the room sensing device is as compact as possible, can be carried as easily as possible by one person, and does not collide with edges above the person's head when the person carrying the room sensing device moves. For this reason, the room sensing device is dimensioned so that the scanning unit does not protrude above the head when carried by a person. Furthermore, when the room sensing device is carried by a person, the scanning unit is arranged and oriented in the scanning position such that the lower beam, directed horizontally towards the lower mounting element, still extends over the person's shoulder.This has the advantage that a larger proportion of the data captured by the scanning device is usable for the SLAM process, and it also results in greater robustness and accuracy, since the ceiling behind the person wearing the room scanning device can still be captured. Ultimately, this provides better coverage of the ceiling for the point cloud.

[0187] The room sensing device is designed to be worn by a single user. The user holds the device by its handle and can also use their other hand to grip the lower section. To support the weight of the device, the user can utilize the harness, into which the support pad can be attached. Additionally, the room sensing device can be connected to the [unclear text] using the first and second coupling elements.

[0188] M / NAVVTR-030-PC NavVis GmbH

[0189] - 29 -

[0190] The harness is attached to limit the rotation of the room sensing device around the axis formed by the support and the lower mounting element. The harness can be adjusted to the user's height. The support attached to the harness has a support plate that is connected to the room sensing device via a hook. This distributes the weight across the user's body. It also ensures that the air inlets on the underside of the room sensing device are not obstructed by the user's clothing.

[0191] In the scan position, the room sensing device is positioned in front of the user, allowing it to capture most of the surroundings. Specifically, the lower beams emitted by the scan device are directed over the user's shoulders into the environment, thus avoiding obstruction by the user's chest. Because the scan device is positioned below the top of the user's head, the head obscures only a small angle of the emission area. In this scan position, the room sensing device can capture walls, floors, and ceilings in large, open environments such as entrance halls or factory floors.

[0192] The room scanning device can be removed from its base at any time, if necessary, to reorient it and capture specific areas that cannot be covered by the scan pose. This is necessary, for example, in small rooms or to capture areas under tables or behind objects and machines.

[0193] To capture images using the camera(s), a trigger can be activated manually or automatically by the control unit, for example, based on a distance along a mapping path. When the spatial scanning device is held in the scan position, an image is captured within a 270° field of view. 90° of the camera's field of view is excluded for the user. This also protects the user's privacy. In post-processing, the missing area in the panoramic images can be filled in using image information generated from the colored point cloud (point-based rendering). Alternatively, the missing areas can be created using image-based rendering methods that synthesize the missing areas from the available images of the scene.

[0194] According to a further development of the inventive method, the speed of movement, in particular the translational speed, of the

[0195] M / NAVVTR-030-PC NavVis GmbH

[0196] - 30 -

[0197] The room detection device detects the movement, and when the trigger button is pressed, the speed of movement, in particular the translational speed, is taken into account. Specifically, threshold values ​​at which the cameras take pictures when the trigger button is pressed are adjusted for the speed of movement, especially the translational speed, depending on the current lighting conditions and / or the current exposure time of the cameras.

[0198] By raising the room-capture device above the user's head in an overhead position, it is possible to create 360° panoramic images. In this case, the area where the user is located below the camera(s) of the room-capture device is masked out and then filled in using the methods described above.

[0199] The user can check the camera recordings by previewing the images on the display surface of the room detection device.

[0200] The spatial sensing device has a foldable hook on its underside, which is used to capture survey markers on floors or walls. Capturing such markers improves the overall data quality, as any drift in the SLAM trajectory can be corrected. These markers also serve for the subsequent alignment and georegistration of the datasets. During the capture of the survey markers, the spatial sensing device rests on three points: two on the device's frame and one on the foldable hook itself, thus ensuring a stable position during the capture process.

[0201] The invention will now be explained using an exemplary embodiment with reference to the drawings.

[0202] Figure 1 shows an embodiment of the space detection device according to the invention in a perspective view.

[0203] Figure 2 shows the embodiment in a different perspective view.

[0204] Figure 3 shows the embodiment of the space detection device according to the invention from the side.

[0205] M / NAVVTR-030-PC NavVis GmbH

[0206] - 31 -

[0207] Figure 4 shows the two-part cover in an assembled state and

[0208] Figure 5 shows the two parts of the cover in a disassembled state.

[0209] Figure 6 shows the harness worn by a person, to which the support is attached.

[0210] Figure 7 shows an embodiment of the room sensing system with the room sensing device and the support in a separate state and

[0211] Figure 8 shows the embodiment of the room detection system in a state in which the lower mounting element of the room detection device rests on the support.

[0212] Figure 9 shows the lower mounting element of the room detection device and the support in detail.

[0213] Figure 10 shows the embodiment of the room sensing system in which the room sensing device is carried by a person using the support and the harness, with the room sensing device in the scan position.

[0214] Figure 11 shows a second embodiment of the room detection system.

[0215] Figure 12 shows a third embodiment of the room detection system.

[0216] Figure 13 shows a fourth embodiment of the space detection system, in which the harness is connected to the space detection device by means of a retaining strap via coupling elements.

[0217] Figures 14 and 15 show this embodiment, in which the retaining strap has been detached from the space detection device.

[0218] Figure 16 shows the fourth embodiment of the spatial detection system, in which the first coupling element is connected to the second coupling element.

[0219] M / NAWTR-030-PC NavVis GmbH

[0220] - 32 -

[0221] Figure 17 shows a detail of the fourth embodiment of the spatial detection system in a sectional view, which shows the coupling of the first coupling element with the second coupling element.

[0222] Figures 18 and 19 show a detail of the first coupling element of the fourth embodiment of the space detection system, in which the fastening of the contact plate is visible.

[0223] Figures 20 and 21 show a detail of the fourth embodiment of the spatial detection system, which shows the coupling of the first coupling element with the second coupling element.

[0224] The individual elements of the space detection device and the space detection system according to the invention are described in the following list of reference numerals.

[0225] In the embodiment of the method according to the invention, the environment in an emission room is detected with the room detection system, which comprises the room detection device according to one of the embodiments, as described in the introduction.

[0226] First, with reference to Figures 1 to 3, the embodiment of the inventive space detection device 1 of the inventive space detection system is described:

[0227] The room detection device 1 has a frame 2 to which a scanning device 3, a lower mounting element 5, several cameras 6, a handle 7 and a display surface 13 are attached.

[0228] The scanning device 3 is designed to capture the environment in an emission chamber, as will be explained later. In the present embodiment, four cameras 6 for capturing panoramic images are arranged at intervals from one another on a circular ring, the axis of this ring being aligned parallel to a central axis of the scanning device 3. Furthermore, the cameras 6 include two global shutter cameras.

[0229] The frame 2 accommodates in its lower part, that is, in particular below the handle 7, an energy storage device, a data storage device, a control unit, and a processing unit. The energy and data storage devices, as well as a power switch, are accessible by opening a flap on the

[0230] M / NAWTR-030-PC NavVis GmbH

[0231] - 33 -

[0232] The underside of the room sensing device 1 is accessible. In the area where the energy storage, data storage, control unit and computing unit are integrated into the frame 2, air inlet openings 16 are provided so that the integrated units are actively cooled by air convection.

[0233] The control unit is linked to the release button and the cameras 6 via data transmission. Pressing the release button simultaneously triggers the image capture by the cameras 6, enabling a 360° panorama to be captured by the cameras 6 at a specific point in time. The release button is integrated into the handle 7 directly below the display area 13.

[0234] Furthermore, the room detection device 1 includes a data interface that is technically linked to an evaluation unit. The data recorded by the room detection device 1 can be output via this data interface to an external storage device for post-processing.

[0235] The spatial sensing device 1 further features a foldable hook with a reference mark in the form of a notch for determining the position and orientation of the spatial sensing device 1 relative to a fixed coordinate system. To determine this position and orientation, the spatial sensing device 1 also features an actuating button that can be pressed when the reference mark is aligned with a corresponding reference mark of the fixed coordinate system.

[0236] The room sensing device 1 further comprises a receiving antenna and an inertial measurement unit. The receiving antenna is designed for data exchange via a wireless local area network. The inertial measurement unit includes accelerometers and gyroscopes to determine the movement of the room sensing device 1 in space.

[0237] Furthermore, the room detection device 1 includes a status sensor configured to detect how the room detection device 1 is worn by a person, as will be explained later.

[0238] With reference to Figures 4 to 6, version 8 of the embodiment of the spatial detection system according to the invention is described:

[0239] M / NAWTR-030-PC NavVis GmbH

[0240] - 34 -

[0241] The support 8 corresponds to the lower mounting element 5 of the room detection device 1, so that the lower mounting element 5 can rest on the support 8 in a scan pose described later. A hook-like connection 14 is formed in the support 8, which corresponds to a hook-like connection 15 of the lower mounting element 5, described later. As shown in Figure 5, the support 8 consists of two parts that can be connected to each other via a snap mechanism.

[0242] Figure 6 shows how the support 8 can be worn by a person. The spatial sensing system includes a harness 9 for this purpose. The length of the straps of the harness 9 can be adjusted to fit the person wearing the support 8. The straps of the harness 9 are attached to mounting tabs on the support 8 and encircle the person's shoulders. On the back of the person, these shoulder straps are attached to a waist belt, which in turn encircles the person's upper body essentially horizontally and is attached to further mounting tabs on the support 8.

[0243] With reference to Figures 7 to 9, the connection of the space detection device 1 with the support 8 is described:

[0244] The lower mounting element 5 of the room sensing device 1 has a hook-like connection 15 for this purpose, which corresponds to the hook-like connection 14 of the support 8. The room sensing device 1 is connected to the support 8 via these hook-like connections 14 and 15 in such a way that it can pivot about a mounting axis. The connection via the hook-like connections 14 and 15 is designed such that pivoting about an axis perpendicular to the mounting axis is prevented. Furthermore, the hook-like connections 14 and 15 are designed such that the pivoting movement about the mounting axis is limited. The stops provided for this purpose are designed such that the room sensing device 1 is held in the scan position described later.

[0245] With reference to Figure 10, the exemplary embodiment of the spatial detection system according to the invention is described below:

[0246] In the illustration of Figure 10, the spatial detection device 1 is carried by a person using the support 8 and the harness 9. When coupled to the support 8, the spatial detection device 1 can pivot about the axis of attachment, which in this case is horizontally oriented. Therefore, various orientations of the

[0247] M / NAVVTR-030-PC NavVis GmbH

[0248] - 35 -

[0249] The room sensing device 1 is positioned in the room. Figure 10 shows the room sensing device 1 in the scan position.

[0250] The scanning device 3 is attached to an upper bracket of the spatial sensing device 1. In the present embodiment, the scanning device 3 is a multi-scanner, specifically a 32-layer LiDAR sensor. The emission units of the multi-scanner are lasers that sequentially emit signal pulses in the emission directions. This multi-scanner comprises several integrated emission units for generating a multitude of signal beams in defined emission directions, a receiver for detecting reflected radiation generated by reflections of the signal beams from one or more objects within a space, and a scanning device for changing the emission directions of the signal beams. The scanning device of the multi-scanner is configured to rotate the emission directions of the signal beams around the axis of rotation, so that the multi-scanner scans the volume of a fan-shaped solid of revolution.The sampled volume is the emission space.

[0251] The scanning device 3 emits signal beams into an emission chamber formed between two beams S1 and S2 rotating around a central axis 4, thus creating a double fan as the emission chamber. In the present embodiment, this opening angle of the emission fan is 40°.

[0252] During this rotation of the rays S1, S2 around the central axis 4, there is an angular position at which the rays S1, S2 are directed horizontally towards the lower support element 5. In this case, they are therefore directed towards the person wearing the room detection device 1. At the angular position 180° opposite, the rays S1, S2 are directed away from the lower support element 5. In this case, they are therefore also directed away from the person wearing the room detection device 1.

[0253] The scan pose of the spatial detection device 1 shown in Figure 10 was defined in the introduction. It is determined by the orientation of the beams S1 and S2 of the scanning device 3. When the beams S1 and S2 are directed towards the lower mounting element 5 during rotation with respect to the horizontal direction, the lower beam S1 forms an angle with a horizontal plane H in the scan pose within a range of -20° to +20°. In the present embodiment, this angle is 0°, so that the lower beam S1, directed towards the person, is horizontally oriented. The second beam S2, that is, the upper beam S2 when directed towards the person, forms an angle of +40° with the horizontal plane H.

[0254] M / NAVVTR-030-PC NavVis GmbH

[0255] - 36 - The opening angle between the two rays S1 and S2 is therefore also 40°. On the opposite side, that is, when rays S1 and S2 are directed away from the person, the second ray S2 is also directed upwards. It thus forms an angle of 0° with the horizontal plane H. The first ray S1 is directed downwards in this case. It forms an angle of -40° with the horizontal plane H. Consequently, the central axis 4 is tilted forwards in the scan pose. It forms an angle of 20° with the vertical.

[0256] The scan pose defines a specific swivel angle between the spatial sensing device 1 and the support 8, which is also limited by the stops of the hook-like connections 14 and 15. In particular, this results in a defined orientation of the spatial sensing device 1 in space and relative to the person wearing the spatial sensing device 1. Consequently, in the scan pose, there is also a distance between the scanning unit 3, which is attached to the frame 2 of the spatial sensing device 1, and the lower support element 5. Since this lower support element 5 is coupled to the support 8, which in turn is worn by the person by means of the harness 9, a vertical distance V1 of the lower support element 5 is obtained from the apex of the opening angle enclosed by the beams S1 and S2. This vertical distance V1 also defines the vertical position, i.e.,the height of the vertex relative to the person carrying the space detection device 1.

[0257] According to the invention, the vertical distance V1 is in a range of 18 cm to 68 cm. In the present embodiment, this vertical distance V1 is 43 cm.

[0258] Considering an average-sized person, 175 cm tall, the apex of the beams S1 and S2 of the scanning device 3 is below the person's height, i.e., the crown of their head. Furthermore, this apex lies above the top of the person's shoulders. The lower mounting element 5 is located vertically at the person's hips.

[0259] Furthermore, in the scan position, additional vertical and horizontal distances result between the room detection device 1 and the person wearing it. The handle 7 is positioned between the lower support element 5 and the scanning device 3 such that, in the scan position, it has a horizontal distance H2 from the lower support element 5 in the horizontal direction, which is in a range of 15 cm to 18 cm, with this horizontal distance H2 being measured at the center of the handle. In the vertical direction, the handle 7 has a

[0260] M / NAVVTR-030-PC NavVis GmbH

[0261] - 37 -

[0262] Vertical distance V2 from the lower mounting element 5, which lies in a range of 21 cm to 25 cm, where in this case too the measurement is to be taken from the center of the handle.

[0263] Between the handle 7 and the scan unit 3, the display surface 13 of a display unit is arranged vertically. In the scan position, the normal of the display surface 1 is inclined such that it forms an angle with the horizontal plane H in a range of 33° to 36°. The horizontal distance H3 from the center of the display surface in the scan position to the lower display element 5 is in a range of 21 cm to 25 cm.

[0264] Due to the arrangement and orientation of the handle 7 and the display surface 13, the person carrying the room sensing device 1 can hold the room sensing device 1 very comfortably in the scan pose on the handle 7, while at the same time being able to view the display surface 13 very well.

[0265] The cameras 6 are arranged on a circular ring such that the axis of the ring coincides with the central axis 4 of the scanning device 3. The viewing angle covered by the cameras 6 is 270°, with the uncovered area of ​​the cameras 6 pointing towards the person wearing the room detection device 1. This person is therefore located in an area not covered by the viewing angle of the cameras 6. The viewing angle of the cameras is also 270° in the vertical plane, and in this case as well, the person is located outside the viewing area.

[0266] The distance between the cameras 6 on the circular ring is chosen to be as small as possible. This distance on the circular ring therefore depends primarily on the size of the camera housings. The cameras 6 are triggered to take pictures by the shutter release button. In this case, a 360° image is captured, but it essentially excludes the area where the person is located. The multi-scanner of the scanning device 3 is positioned as close as possible above the camera 6.

[0267] In addition to the four cameras 6 for 360° imaging, two global shutter cameras are also mounted on the circular ring. These cameras are used for visual odometry to support the SLAM method of the scanning device 3. These global shutter cameras have lenses with a very wide viewing angle of more than 180°. They are oriented forward and to the side to capture as many visual features as possible for use in visual odometry. The global shutter cameras are designed to record a video stream that is transmitted via the interface to an external device.

[0268] M / NAVVTR-030-PC NavVis GmbH

[0269] - 38 -

[0270] Data can be transferred to storage in order to calculate the movement of the room detection device 1 during later data evaluation.

[0271] By tilting the central axis 4 of the scanning device 3 forward, it is ensured that closer surface sections of the floor can be captured by the scanning device 3 and also by the cameras 6 in the scan position. Simultaneously, the signal beams of the scanning device 3 are emitted upwards in the opposite direction, so that only a few areas in the space behind the person below the scanning device 3 are illuminated. Furthermore, the obstruction caused by the person wearing the room detection device 1 is very minimal in this area, since essentially only the user's head is within the emission zone. On the other hand, the room detection device 1 does not extend above the person wearing it, so there is no risk of the person colliding with objects above them when moving with the room detection device 1.Furthermore, this orientation is advantageous for real-time 3D SLAM processes.

[0272] Figure 11 shows another embodiment of the spatial detection system according to the invention. In this embodiment, the central axis 4 forms an angle of 30° with the vertical. The spatial detection device 1 is thus tilted further forward than in the embodiment shown in Figure 10. The geometry of the spatial detection device 1 and the scanning unit 3 is otherwise identical to that of the embodiment in Figure 10, so that the lower beam S1, when directed towards the lower support element 5, forms an angle of +10° with the horizontal plane H. Accordingly, the horizontal distances H2 and H3 are also greater than in the embodiment of Figure 10.

[0273] Figure 12 shows another embodiment of the spatial detection system according to the invention. In this case, the central axis 4 is not tilted forward as far as in the embodiment shown in Figure 10. The central axis 4 forms an angle of 15° with the vertical. Otherwise, the geometry of the spatial detection device 1 and the scanning device 3 is identical to the geometry of the embodiment shown in Figure 10. Accordingly, the lower beam S1, when directed horizontally towards the lower mounting element 5, is inclined downwards. It forms an angle of -5° with the horizontal plane. The horizontal distances H2 and H3 are also smaller than in the embodiment shown in Figure 10.

[0274] M / NAVVTR-030-PC NavVis GmbH

[0275] - 39 -

[0276] Figure 13 shows another embodiment of the spatial detection system according to the invention. The spatial detection device 1 corresponds to the spatial detection device 1 of the embodiment shown in Figure 10. The scan pose is also aligned in the same way. However, in this case, the spatial detection system comprises a second coupling element 11 with a retaining strap 12, which is designed as a neckband. The retaining strap 12 is in turn connected to the harness 9. The second coupling element 11 establishes a detachable connection with the first coupling element 10 of the spatial detection device 1.

[0277] Figure 13 shows a state in which the coupling elements 10 and 11 are coupled together and the collar is taut. In this case, the length of the collar is chosen such that the spatial detection device 1 is in the scan position.

[0278] In the present embodiment, the retaining strap 12 has an elastic section and a non-elastic webbing to which the elastic section is attached in its stretched state. For example, it is sewn on. When no load hangs from the retaining strap 12, the elastic section gathers. The elastic section is located only on one side, so that the second coupling element 11 does not hang centrally when the retaining strap 12 is gathered. Instead, it is pulled out of the way to the side, so that it does not obstruct the user.

[0279] Figure 14 shows a state in which the second coupling element 11 is detached from the first coupling element 10, and the user holds the room detection device 1 by means of the handle 7.

[0280] With reference to Figures 15 to 21, it is explained in detail how the coupling is carried out via the coupling elements 10 and 11:

[0281] The second coupling element 11 comprises a spherical element 17 arranged at the end of a pin-shaped element 18, which in turn transitions into a retaining element 21 (see Figure 17). The first coupling element 10 comprises a keyhole-shaped recess 20, wherein the spherical element 17 can be inserted into the keyhole-shaped recess and held in place by an undercut.

[0282] The spherical element 17 contains ferromagnetic material, whereas the first coupling element 10 comprises a magnet arranged such that the spherical

[0283] M / NAWTR-030-PC NavVis GmbH

[0284] - 40 -

[0285] Element 17 is held in an undercut position by magnetic force. In this case, a force-fit connection is established between the coupling elements 10 and 11 in all tensile directions that can act on the second coupling element 11 via the retaining band 12. The coupling of the coupling elements 10 and 11 can be released in this case by pushing the pin-shaped element 18 in a direction in which the spherical element 17 is moved in a direction in which it can be led out of the recess 20.

[0286] In the embodiment of the coupling elements 10 and 11 shown in Figure 17, the keyhole-shaped recess is formed by a metal contact plate 23. The contact plate 23 is embedded in a plastic shell 22. On its rear side, the contact plate 23 has a receptacle 24 corresponding to the spherical element 17 in an undercut position (see Figure 19). In this position, the surfaces of the spherical element 17 and the contact plate 23 lie in contact with each other. The spherical element 17 is held in this position by a magnet 25. After being inserted into the keyhole-shaped recess 20, the spherical element 17 is guided into the connection position by sliding action from the rear side of the contact plate 23.

[0287] As shown in Figure 16, the coupling elements 10 and 11 are also designed such that the maximum swivel angle a1 of the second coupling element 11 upwards is 8°, the maximum swivel angle a2 of the second coupling element 11 downwards is 60°, the maximum swivel angle a3 of the second coupling element 11 to one side is 7°, the maximum swivel angle a4 of the second coupling element 11 to the other side is 7°, and the rotation angle a5 about the axis of the second coupling element 11 is 360°.

[0288] An embodiment of the method according to the invention is explained below. In particular, the method uses the room detection system described above with the room detection device 1 according to the invention:

[0289] First, the room detection device 1 is aligned in the room and its position and orientation are recorded. For this purpose, the room detection device 1 is placed with its reference mark against a reference mark of a fixed coordinate system and the actuating button is pressed.

[0290] The user then puts on the support 8 with the harness 9 and the retaining strap 12. Next, the user grasps the room detection device 1 by the handle 7, lifts it up, and places the

[0291] M / NAVVTR-030-PC NavVis GmbH

[0292] - 41 - lower mounting element 5 onto the support 8, so that the hook-like connections 14 and 15 interlock. In this state, the room detection device 1 can be pivoted about a horizontal mounting axis.

[0293] The user now connects the second coupling element 11 of the harness 9 to the first coupling element 10 of the spatial sensing device 1 by grasping the second coupling element 11 at the retaining element 21, inserting the spherical element 17 into the keyhole-shaped recess 20, and guiding the spherical element 17 upwards until it is held by the magnet 25. The retaining strap 12 now defines a swivel angle of the frame 2 of the spatial sensing device 1 resting on the support 8 in the scan pose as shown in Figure 10.

[0294] The scanning of the room in which the user is located then begins. For this purpose, in this scan position, the room detection device 1 emits signal beams from the scanning unit 3 into the opening angle between the first beam S1 and the second beam S2, with the opening angle rotating around the central axis 4. The reflections of the signal beams are reflected by one or more objects in the room being scanned, and these reflected beams are detected by the scanning unit 3. While the signal beams are being emitted into the emission area, the user moves through the room. Simultaneously, the user can take pictures with the cameras by pressing the trigger button.

[0295] As the user moves through the room, the room detection device 1 is located. This localization is achieved firstly through a SLAM method, in which the evaluation unit processes the data generated by the scanning device 3. Secondly, localization is achieved through visual odometry, in which the evaluation unit processes the data generated by the global shutter cameras 6. Local processing of this data allows for the generation of display data, which shows the user on the display surface 13 which areas of the room have already been scanned and which areas still need to be scanned. This initial data processing takes place locally within the room detection device 1 by the evaluation unit in real time. Thus, the position of the room detection device 1 within the object space and its trajectory can be calculated locally.Furthermore, the area entered and the area accessible within the object space are graphically displayed on display area 13 based on the data already recorded.

[0296] M / NAVVTR-030-PC NavVis GmbH

[0297] - 42 -

[0298] If narrow passages or low doorways need to be navigated in the room being scanned, the user can tilt or remove the room scanning device 1 so that it is no longer in the scan position. This can be detected by the status sensor. The status data generated in this way is transmitted to the evaluation unit, which can later transfer it to the external storage unit so that it can be taken into account during post-processing. The status sensor can also detect when the room scanning device 1 has been placed stably on the floor. In this case, the control unit interrupts data acquisition. Furthermore, the status sensor data is collected to evaluate the quality of the data recorded by the scanning device 3 and the cameras 6, so that this evaluation can be considered later during post-processing.

[0299] After the room has been scanned using the room scanning device 1, the captured data, which is stored, for example, on a built-in or removable storage unit, is transferred, if necessary, via a wireless interface or cable to an external storage unit. Post-processing can then be carried out on this data using a stationary server. During post-processing, the point cloud data generated by the scanning device 3 and the images captured by the cameras 6 are aligned with respect to the scanned locations and colorized to create colorized point cloud data. The person wearing the room scanning device 1 is excluded from the captured data. Finally, during post-processing, the images captured by the cameras 6 are stitched together to create 360° panoramic images.In the area where cameras 6 did not capture any images because the person wearing the room detection device 1 was located there, the images are augmented using the point cloud data generated by the scanning device 3. Image-based rendering methods are used for this purpose. This augmentation is performed in such a way that the space behind the person is displayed in the omitted area as if the person had not been in that area, and as if the area had not been omitted but had been captured by cameras 6.

[0300] Further embodiments of the spatial detection system, the spatial detection device 1, and the method according to the invention are described below. These further embodiments complement the exemplary embodiments described above and thus result in new exemplary embodiments:

[0301] M / NAWTR-030-PC NavVis GmbH

[0302] - 43 -

[0303] The first supplementary aspect concerns relocalization during spatial acquisition. Relocalization is particularly necessary when the position of the spatial acquisition device 1 within the coordinate system of the acquired space is no longer known. Relocalization can be performed either in real time or during the post-processing of the acquired data.

[0304] In the SLAM method described above, relocalization is performed using the signal beams emitted by the scanning device 3. Specifically, a LiDAR-based SLAM method is used for relocalization. However, this method can produce false-positive relocalization results. Furthermore, it is possible that during post-processing of the data, it may no longer be possible to relocate the spatial detection device 1, for example, because the maximum time for relocalization has been exceeded. This leads to inaccuracies in the point cloud data and potentially to incomplete data processing.

[0305] For this reason, visual odometry is additionally used in a training course to support LiDAR-based relocalization. This advantageously avoids false-positive relocalizations, accelerates the relocalization process, and increases the reliability of relocalization both in real time and during post-processing. The addition of visual odometry makes relocalization more accurate and reduces processing errors.The support of LIDAR-based relocalization by visual odometry is particularly advantageous when LIDAR-based relocalization has failed because the room sensing device 1 has lost its position, for example in telephone or toilet cubicles, lost its localization in narrow stairwells or long corridors, or when the relocalization takes place at a different location than the location where the localization was lost.

[0306] If the localization is lost, the inventive method performs a relocalization using LIDAR-based relocalization and a relocalization using visual odometry independently of each other in order to achieve the relocalization as quickly as possible while ensuring high reliability in the relocalization.

[0307] M / NAVVTR-030-PC NavVis GmbH

[0308] - 44 -

[0309] The following procedure considers the case where localization has been lost using both the LIDAR-based method and visual odometry, and the evaluation unit performs a mislocalization:

[0310] 1. If a lidar-based relocalization was successful, but a relocalization using visual odometry was not, the procedure pauses the visual odometry relocalization, sets its status to "lost," and then restarts the visual odometry relocalization. In this case, the relocalization is performed using the lidar-based method.

[0311] 2. If relocalization using the LIDAR-based method is not possible, but relocalization using visual odometry was successful, the fusion SLAM method detects the successful relocalization by visual odometry. A location prediction based on visual odometry is then called upon to perform an ICP (Iterative Closest Point / nf) procedure. The visual odometry-based prediction is compared with the LIDAR-based relocalization hypotheses (primary / secondary), and relocalization is only performed if these hypotheses are consistent with the relocalization based on visual odometry.

[0312] 3. If, however, relocalization is not possible using either LIDAR-based methods or visual odometry, the relocalization process will, for example, result in a timeout after 60 seconds. If relocalization based on LIDAR-based methods and visual odometry is possible independently within the same timeframe, the method according to the invention performs the relocalization using the LIDAR-based method and resets the visual odometry to ensure the consistency of the map generated by visual odometry.

[0313] The following details the relocalization based on visual odometry:

[0314] If the tracking of the spatial detection device 1 using visual odometry is lost, the following steps are performed to be able to re-detect the location of the spatial detection device 1:

[0315] M / NAVVTR-030-PC NavVis GmbH

[0316] - 45 -

[0317] The most visually similar frames, or keyframes, to the current frame are identified. This can be done, for example, using a visual word vector. The following steps are performed for each keyframe candidate:

[0318] The features of the ORB (Oriented FAST and Rotated BR / EF) algorithm between the candidate key frame and the current frame are compared. Matches are used to attempt a pose estimation using a solution to an MLPnP (Maximum Probability Solution for the Perspective-n-Point Problem). This yields an estimated pose and a number of outliers. If few outliers are found, the procedure continues with the next key frame candidate. Otherwise, the pose estimated by the MLPnP algorithm solution is used for the current frame. A non-linear optimization is then performed, i.e., frame-by-frame tracking is carried out. If few outliers are found, the procedure continues with the next key frame candidate.

[0319] If the relocation was successful using this method, a single-image-to-map tracking is performed and the tracking status is set to OK.

[0320] According to a further embodiment of the inventive method, the triggering of the cameras 6 by means of the release button is further developed:

[0321] If a panoramic image is to be generated using the cameras 6, one embodiment of the inventive method requires that the user stop moving. The movement state of the spatial detection device 1 can be detected, for example, via the state sensors. This prevents motion blur in the recorded images.

[0322] Assuming a static environment, the degree of motion blur visible in the images depends on the speed of movement during recording. This speed can be linear or rotational. Furthermore, the motion blur depends on the exposure time used to capture the images from which the panoramic images are generated. Finally, the motion blur depends on the distance of the captured objects from the spatial detection device 1, i.e., from the cameras 6. This is particularly true for a translational speed of the spatial detection device 1, but not for a rotational speed.

[0323] M / NAVVTR-030-PC NavVis GmbH

[0324] - 46 -

[0325] It's possible that when the shutter button is pressed, only the speed of movement is taken into account. This means that the user has to stop for each shot, as the camera assumes the worst-case exposure time.

[0326] In brighter environments, such as outdoors, the exposure time will be so short that the images will not become blurry, even if the user is moving during the recording and has not stopped. In this embodiment of the method according to the invention, threshold values ​​for the movement speed are therefore adjusted depending on the current lighting conditions or the current exposure time of the cameras 6, so that the user does not have to stop in good lighting conditions and the shutter release button can also be pressed if the user is moving. In addition, the distance of the objects in the environment determined by the LIDAR data can be taken into account, and thus it can be calculated or estimated whether the image of the objects on the image sensor moves further than, for example, one pixel within the determined or set exposure time.If this is not the case, it can be assumed that an image without motion blur is possible despite the user's movement.

[0327] Advantageously, this method allows the process to be carried out more quickly without degrading the data used for spatial mapping. This particularly enables a higher density of panoramic images from the cameras of the scanned space, which is beneficial for certain rendering techniques such as Gaussian splatting.

[0328] According to a further embodiment of the inventive method, the generation of the panoramic images using the cameras 6 is improved by a dynamic and automatic recognition of the user:

[0329] According to the procedure described above, the panoramic images are only captured within a 270° angle to prevent the user from appearing in the panoramic image. Masking the user is necessary because the cameras 6 are located approximately at the height of the user's head, and the tilt of the room detection device 1 is insufficient to prevent the user's head from being captured. Therefore, in this case, it is necessary to supplement a relatively large area of ​​the panoramic images with the colored point cloud or with image-based rendering methods, regardless of the user's actual position. During dynamic user detection, this

[0330] M / NAVVTR-030-PC NavVis GmbH

[0331] - 47 -

[0332] In the embodiment of the inventive method, the masking only covers the area necessary for the user, thereby improving the overall quality of the panoramic image.

[0333] For this purpose, the inventive method incorporates automatic person recognition. This can be achieved, for example, using AI-supported image recognition and segmentation methods known in the prior art, employing speech-based segmentation, such as with the prompt: "Where is the tallest person in the image?" This enables more precise identification of the operator of the room detection device 1, thereby increasing the amount of camera image information actually generated and reducing the area to be supplemented in the panoramic images.

[0334] For this purpose, it is possible, for example, to recognize the user's mask in the camera images using a neural network that accepts the camera image as input. Before the images are fed into the neural network, they are mirrored so that the operator's masking is always located in a corner of the image, for example, the upper right corner. Downsampling, upsampling, and auxiliary loss can then be used to improve the masking for the user.

[0335] The training images are crucial for training the neural network. If the neural network is trained with images containing multiple different operators and background environments, the trained neural network will not improve operator masking. Therefore, the following is taken into account when training neural networks:

[0336] Color of Concealment: Concealment is particularly common with a medium-sized operator wearing a black or very dark shirt. However, the training data for the method according to the invention should also include unusual scenarios such as shirts of different colors and color patterns. Furthermore, with short-sleeved or sleeveless shirts, or shirts with a low neckline, some skin is visible. The scenario where the operator is wearing a brightly colored safety jacket must also be covered.

[0337] Size of the obscuration: The size, and consequently the type, of the obscuration varies depending on the operator's height and the shooting position. With taller individuals, a large portion of the neck and head may be visible in the camera images. This opens up numerous further possibilities, such as variations in hair and skin color, the operator's hair length, and any head covering, such as that worn by a...

[0338] M / NAVVTR-030-PC NavVis GmbH

[0339] - 48 -

[0340] Hoodies or helmets. The helmets, by the way, can be of various colors.

[0341] False positives in the background: In images taken in hallways, the edge between the wall and the floor often creates a user-like pattern in the image. This can lead to misleading user detection. Training images should therefore include such backgrounds to avoid false positives. Furthermore, unusual backgrounds, such as textured carpets, light-colored floors, and line markings on the floor (like those found in warehouses), should be considered in the training data, as they can otherwise impair user detection.

[0342] Accordingly, the training data should be compiled and, if necessary, expanded in light of the above explanations. The training images can then be divided into at least three categories:

[0343] 1. Manually labeled images: In this case, these are reverse-facing camera images where the occlusion has been manually labeled.

[0344] 2. Front camera images: To improve the performance of the neural network in terms of rejecting false positive results, it is helpful to include many training examples that do not contain any occlusion.

[0345] 3. Synthesized Images: For some of the problematic cases described above, only a limited amount of training data may be available. In this case, additional training patterns can be synthesized from the limited data by copying the pixel-accurate occlusion area and then pasting it onto a different background. In addition to offline-synthesized training patterns, several runtime extension techniques can also be applied to reduce overfitting and maximize the usefulness of the available training data. For example, random brightness variations, white Gaussian noise, random color channel swaps, and occlusion scaling by a random factor can be applied.

[0346] According to a further embodiment and development of the inventive method, the data from the inertial measuring unit are also used for user recognition.

[0347] M / NAVVTR-030-PC NavVis GmbH

[0348] - 49 - the point cloud data and / or the data from the state sensor are used. For example, if it has been determined that there is a high probability that no user is obscuring the panoramic image, the complete 360° panoramic image data can be used without masking any area due to user obscuration. The complete 360° panoramic images can be used, for example, if the state sensor has detected, or if data from the inertial measurement unit or trajectory data has been obtained in real time or in post-processing, that the room sensing device 1 has been raised above the user's head. This state of the room sensing device 1 can also be derived from the point cloud data.Conversely, point cloud data can also be used to determine whether the user was within the field of view, particularly of the rear-facing camera. In a simpler implementation, this can be achieved by simply identifying whether there are structures in the LiDAR data that match the size, distance, and shape of the user's head, torso, or body. Image-based person recognition can then be specifically focused on this area, or the various recognition methods can be combined or fused to achieve particularly reliable and accurate detection of areas obscured by the user. This means that only small areas need to be supplemented with point cloud data or image-based rendering methods.

[0349] Alternatively or additionally, the inertial measuring unit can detect that the room detection device 1 has been moved out of the support 8 and, for example, swiveled so that images were taken under an object, such as a table. In this case as well, the user will not obscure the 360° panoramic image, so that the complete data can be used without masking the user.

[0350] M / NAVVTR-030-PC NavVis GmbH

[0351] - 50 -

[0352] Reference symbol list

[0353] 1 room detection device

[0354] 2 frames

[0355] 3 Scan setup

[0356] 4 Central axis

[0357] 5 lower plant element

[0358] 6 cameras

[0359] 7 handle

[0360] 8th edition

[0361] 9 Harness

[0362] 10 first coupling element

[0363] 11 second coupling element

[0364] 12 Retaining strap

[0365] 13 Display area

[0366] 14 hook-like connections of the support

[0367] 15 hook-like connection of the lower system element

[0368] 16 air intake openings

[0369] 17 spherical element of the second coupling element

[0370] 18 pin-shaped element of the second coupling element

[0371] 20 keyhole-shaped recesses

[0372] 21 retaining element

[0373] 22 bowls

[0374] 23 Contact plate

[0375] 24. Mounting the contact plate for the second position of the spherical element

[0376] 25 Magnet

[0377] M / NAVVTR-030-PC NavVis GmbH

[0378] - 51 -

[0379] 51 first beam

[0380] 52 second beam

[0381] H Horizontal plane

[0382] V1 Vertical distance of the lower system element from the apex of the

[0383] Opening angle

[0384] V2 Vertical distance of the handle from the lower mounting element

[0385] H2 Horizontal distance of the handle from the lower mounting element

[0386] V3 Vertical distance of the display area from the lower mounting element

[0387] H3 Horizontal distance of the display surface from the lower mounting element a1 Maximum swivel angle of the second coupling element upwards a2 Maximum swivel angle of the second coupling element downwards a3 Maximum swivel angle of the second coupling element to one side a4 Maximum swivel angle of the second coupling element to the other side a5 Rotation angle about the axis of the second coupling element

[0388] M / NAVVTR-030-PC

Claims

1. NavVis GmbH - 52 - Patent claims 1. A spatial detection device with at least one scanning device for detecting the environment in an emission chamber, wherein the emission chamber is formed between two beams rotating about a central axis and enclosing an opening angle, and a frame comprising a lower mounting element and an upper support to which the scanning device is attached, characterized in that a scan pose is defined for the spatial detection device in which the scanning device is oriented such that the lower beam of the beams, when the beams are directed towards the lower mounting element during a rotation with respect to the horizontal direction, forms an angle with a horizontal plane in a range of -20° to +20°, and in the scan pose of the spatial detection device the distance of the scanning device from the lower mounting element is selected such thatthat the vertical distance of the lower mounting element from the apex of the opening angle lies in a range of 18 cm to 68 cm.

2. Space detection device according to claim 1, characterized in that in the scan position the scanning device is aligned such that the upper beam of the beams, when the beams are directed towards the lower mounting element during a rotation with respect to the horizontal direction, forms an angle with the horizontal plane in a range of 0° to +60°.

3. Space detection device according to claim 1 or 2, characterized in that in the scan position the scanning device is aligned such that the bisector of the rays, when directed towards the lower mounting element during a rotation with respect to the horizontal direction, forms an angle with the horizontal plane in a range of 0° to 40°.

4. Space detection device according to one of the preceding claims, characterized in that M / NAWTR-030-PC NavVis GmbH - 53 - in the scan position, in which an average-sized person carries the room sensing device in the scan pose such that the lower attachment element lies vertically in the area of ​​the hip, the scanning device is located below the top of the person's head.

5. Room detection device according to one of the preceding claims, characterized in that the frame has a handle which, in the scan position, is arranged horizontally between the lower mounting element and the scanning device at a horizontal distance from the lower mounting element which is in a range of 10 cm to 30 cm.

6. Room detection device according to claim 5, characterized in that the handle in the scan position is arranged vertically between the lower mounting element and the scanning device at a vertical distance from the lower mounting element which is in a range of 12 cm to 35 cm.

7. Room detection device according to one of the preceding claims, characterized in that the room detection device has a display device with a display surface whose normal in the scan position forms an angle with the horizontal plane in a range of 24° to 44°.

8. Room detection device according to claim 7, characterized in that the display surface in the scan position is arranged at a horizontal distance from the lower mounting element which lies in a range of 12 cm to 35 cm.

9. Room detection device according to one of the preceding claims, characterized in that the scanning device is a multi-scanner, in particular a LIDAR sensor.

10. Room detection device according to one of the preceding claims, characterized in that M / NAVVTR-030-PC NavVis GmbH - 54 - the room detection device further comprises at least one camera, in particular several cameras, which are attached to the frame between the scanning device and the handle.

11. Room detection device (30) according to claim 10, characterized in that the cameras in the scan pose can capture images in a recording angle which lies in a range of 180° to 360°, which is in particular 270°, and a person who carries the room detection device in the scan pose is arranged in the area not captured by the recording angle.

12. Space detection device according to one of claims 10 or 11, characterized in that the cameras are arranged spaced apart on a ring, wherein the axis of the ring is aligned parallel to the central axis of the scanning device.

13. Room sensing device according to one of the preceding claims, characterized in that the room sensing device comprises an energy storage device, a data storage device, a control unit and / or a computing unit, which are arranged in or on the frame in the scan position below the handle.

14. Space detection device according to one of the preceding claims, characterized in that the space detection device has at least one reference mark for detecting the position and / or orientation of the space detection device relative to a stationary coordinate system.

15. Space detection device according to claim 14, characterized in that the reference marking is formed by a foldable hook.

16. Room detection system with a room detection device according to one of claims 1 to 15 and a support which corresponds to the lower mounting element in such a way that the lower mounting element can rest on the support in the scan position. M / NAWTR-030-PC NavVis GmbH - 55 - 17. Room detection system according to claim 16, characterized in that, when the lower mounting element rests on the support, the room detection device is pivotable about a mounting axis which is horizontally aligned in the scan position.

18. Room detection system according to claim 16 or 17, characterized in that a hook-like connection can be established between the support and the lower mounting element, in which the room detection device is pivotable about the mounting axis.

19. Room detection system according to one of claims 16 to 18, characterized in that the support interacts with the system element in such a way that the system element is pivotable about the system axis when it rests on the support, but pivoting about an axis perpendicular to the system axis is hindered.

20. Room sensing system according to one of claims 16 to 19, characterized in that the room sensing system further comprises a harness for adjustable attachment of the support to a person who wears the room sensing device, wherein the distance of the scanning device from the lower support element is selected such that the scanning device is located lower than the top of the head of the person who wears the room sensing device.

21. Space detection system according to claim 20, characterized in that the space detection device has a first coupling element and the harness comprises a second coupling element with which a detachable connection between the harness and the space detection device can be established, wherein one of the coupling elements limits a pivoting movement of the space detection device about the mounting axis formed between the support and the mounting element in such a way that the space detection device is held in the scan pose.

22. Room detection system according to claim 21, M / NAVVTR-030-PC NavVis GmbH - 56 - characterized in that the second coupling element comprises a retaining band.

23. Room detection system according to claim 21 or 22, characterized in that the second coupling element comprises a spherical element arranged at the end of a pin-shaped element, and the first coupling element comprises a keyhole-shaped recess, wherein the spherical element can be inserted into the keyhole-shaped recess and is retained in an undercut.

24. Spatial detection system according to claim 23, characterized in that the spherical element contains ferromagnetic material and the first coupling element comprises a magnet arranged such that the spherical element is held in an undercut position by the magnetic interaction between the magnet and the ferromagnetic material of the spherical element.

25. Room detection system (30) according to one of claims 16 to 24, characterized in that the central axis of the scanning device in the scan position encloses an angle with the vertical direction which lies in a range of 0° to 40°.

26. Method for detecting the environment in an emission room with a room detection device according to one of claims 1 to 15, wherein in the scan pose of the room detection device signal beams are emitted by the scanning device into the opening angle and reflection radiation generated by reflections of the signal beams on one or more objects of the object space is detected.

27. Method according to claim 26 with a room detection device (30) according to any one of claims 9 to 15, wherein, while the environment in the emission room is detected, the room detection device is localized by LIDAR-SLAM methods and by camera-based visual odometry. M / NAVVTR-030-PC NavVis GmbH - 57 - 28. Method according to claim 27, characterized in that the reflections of the signal beams of the scanning device are evaluated and the spatial detection device is localized by LIDAR-SLAM methods and the localization is supplemented by camera-based visual odometry in areas where the reflections of the signal beams of the scanning device are insufficient.

29. Method according to one of claims 26 to 28, characterized in that point cloud data is recorded by the scanning device and images are recorded by the at least one camera, and in a post-processing step the point cloud data is colored by means of the images recorded by the camera, so that colored point cloud data is generated.

30. Method for capturing the environment in an emission room with a room detection device, characterized in that point cloud data is captured by a scanning device and images are captured by at least one camera in such a way that the person wearing the room detection device is excluded, and in a post-processing step the images captured by the camera in the excluded area are supplemented with point cloud data, in particular colored point cloud data, and / or with the help of image-based rendering methods from the available images of the environment. M / NAVVTR-030-PC

Images