Laser scanner with image derivation scan setting function

Abstract

The invention relates to a laser scanner with an image derivation scan setting function, in particular to a stationary ground laser scanner and an image processing method, which process a 2D image of a scanning sphere captured by a camera of the stationary laser scanner prior to measuring a scanning point on the basis of a machine learning algorithm, the image is divided into a plurality of clusters having different predefined measurement importance categories, and at least one adaptive scan parameter is set depending on the presence and / or absence of a cluster having an importance category in the first 2D image.

Description

TECHNICAL FIELD

[0001] The present invention relates to a terrestrial laser scanner and a method for scanning with camera image based scan setup adaptation. BACKGROUND

[0002] 3D scanning is a very effective technique for producing in a few minutes or seconds, millions of spatial measurement points of the surface of an object within a scanning sphere. Terrestrial laser scanning technology is used in particular for collecting static 3D data of stationary, non-moving artificial structures, such as buildings, construction sites, industrial plants, or of scenes of non-moving artificial objects, for example, crash sites. Typical measurement tasks are the recording of objects, such as industrial plants, house facades or historic buildings, but also accident sites and crime scenes, or of their surfaces.

[0003] To this end, a stationary laser scanner continuously directs a measurement beam of a distance measurement unit over the surface of an object within a measurement space or scanning sphere and simultaneously detects the direction and distance with respect to the respective measurement point in the process. The direction determination is based on an angle measurement. The distance determination is based on the reflection of the reflected measurement radiation from the illuminated target object, so that at least a portion of the measurement radiation is reflected back to the laser scanner and detected. For the detection, the scanner has a receiver embodied as a photodetector, which is designed for the time-resolved detection of the reflected measurement radiation, for example embodied as an APD diode. From the plurality of scan points, a so-called 3D point cloud is generated by data processing from the distance of each scan point and the direction information associated therewith.

[0004] Such a terrestrial measurement device for this purpose has at least one radiation source or emitter for generating optical measurement radiation, usually laser radiation, and an optical device, which includes a rotating deflector through which the generated measurement radiation can be emitted in free space onto a target or object to be measured, so these devices are also called so-called free-beam sensors. Optical measurement radiation is understood in this case as electromagnetic radiation, which is not only in the visible range of the spectrum, but also radiation in the ultraviolet, infrared and terahertz range. Known laser scanning devices use measurement radiation with a wavelength of 405 nm, 532 nm, 635 nm, 650-690 nm, 780 nm or 785 nm, 795 nm, 808-850 nm, 905 nm, 980 nm, 1064 nm or between 1500 nm and 1570 nm.

[0005] Such ground scanners are thus designed, in terms of their basic structure, to determine distances to object points as measurement or scanning points using electro-optical and usually laser-based distance measuring devices. The measuring beam of the distance measuring device is deflected in one spatial direction by rotating the aforementioned deflector about one axis (elevation axis) and independently thereof in another spatial direction by rotating a top portion or body comprising or supporting the deflector portion about a second axis (azimuth axis). The body is for example rotatably mounted to a base which comprises a tripod for positioning the laser scanner at a known reference position. The rotation angles are thus measured by respective angle encoders and the rotation speed of the deflector / around the elevation axis is usually much higher than the rotation speed of the body / around the azimuth axis.

[0006] As a result, a spatial measurement region (scanning sphere) can be recorded. The scanning width in horizontal direction is here often 360°, i.e. one complete circle around the azimuth axis, and in vertical direction for example 180°, i.e. a half circle around the elevation axis. The result is a half sphere which together with the maximum measurement distance defines the measurement space or scanning sphere.

[0007] The measurement is usually realized by determining distances and angles, i.e. in spherical coordinates which can also be transformed into Cartesian coordinates for display and further processing. For fast and accurate scanners, in particular a short measurement time as well as a high measurement accuracy, e.g. in the range of millimeters or below and a measurement time of individual points in the range of sub- to milliseconds are required. In this case, the range of the measurement region is from a few centimeters to several kilometers.

[0008] In this case, the spatial measurement resolution or scanning density is of particular importance. It determines what details can still be identified, but also influences the duration of the scanning process and the amount of data obtained in the process. In order not to miss relevant object features, the resolution requirement for such relevant features mainly determines the overall scanning density. Thus, a measurement project with modern high-speed laser scanners results in 3D point clouds with cardinalities of e.g. several million or several billion scanning points and more, many of which are actually superfluous because they do not provide useful information, e.g. surfaces representing objects of low or no interest (at least with too much detail). The storage, transmission and processing of the associated large amounts of data pose a huge challenge to hardware and software. For example, the execution speed of programs for evaluating 3D data depends to a large extent on the number of scanning points; thus, unnecessary points lead to a waste of processing time.

[0009] A specific problem in processing point clouds is so-called point cloud registration, i.e. the combination of point clouds of adjacent scan spheres taken from different stations based on overlapping scan areas into one combined point cloud. Thus, objects present in one scan / point cloud but not in the other point cloud or at a different position in the overlapping area, e.g. moving objects, vehicles or leaf shapes moved by the wind, can lead to registration errors.

[0010] Another problem of prior art scanners can arise from highly reflective surfaces, e.g. window panes, mirrors, car lights or street signs. Such object surfaces can lead to an oversaturation of the light receivers of e.g. a laser scanner or to a so-called multipath effect, which arises from detecting a return signal that has been reflected more than once. Windows are further problematic, because the measuring radiation can be reflected by the window itself or objects behind it, which can lead to ambiguities.

[0011] In addition to the distance measuring unit, modern laser scanners usually comprise one or more color / RGB cameras located inside the housing of the scanning device or mounted on the housing of the scanning device. The cameras can capture color images / photos of the scan sphere with a known spatial relationship to the scanned 3D data to enable mapping of the image data to the point cloud data, i.e. the color or other image-based rendering of the scanned environment. Each scan point of the point cloud is colored using the color value of the one or more data points in the image that are closest to the scan point, which is considered to correspond to the image point. Thereby, e.g. a color point cloud can be provided.

[0012] Furthermore, with regard to point cloud coloring (or 2D-imaging) of the scanned environment, problems can arise due to light conditions. For example, the above-mentioned highly reflective objects or strong natural or artificial light sources can lead to suboptimal or insufficient images, e.g. inconsistent color information.

[0013] In the worst case, when the repeated scanning or imaging process is too late or essentially impossible, any such undesired effects are noticed only later in the office during data processing. SUMMARY

[0014] It is therefore an object of the present invention to provide an improved stationary terrestrial laser scanner.

[0015] The present invention relates to a terrestrial laser scanner for coordinated measuring of a plurality of scan points representing a surface of an object within a scan sphere for generating a point cloud comprising the measured scan points.

[0016] The laser scanner comprises a base for fixing the laser scanner, a main body mounted on the base, a first motor configured to rotate the main body relative to the base about an azimuth axis at a first speed, and a first angle encoder configured to determine a first angle of the main body about the azimuth axis.

[0017] The laser scanner further comprises an emitter configured for emitting optical measurement radiation, a deflector carried by the main body and configured for deflecting the measurement radiation emitted in the form of a free beam onto a respective scan point of an object surface, and a receiver configured for detecting reflected measurement radiation reflected from the respective scan point back and deflected by the deflector onto the receiver.

[0018] The laser scanner further has a second motor configured for rotating the deflector relative to the main body about a pitch axis at a second speed, the second speed being higher than the first speed, and a second angle encoder configured for determining a second angle of the deflector about the pitch axis, wherein the first angle and the second angle together define a scan direction.

[0019] Further, the scanner has at least one camera for capturing 2D images, the at least one camera having an optical camera axis in a known spatial relationship to the scan direction.

[0020] Further, the scanner comprises a control and processing unit configured for measuring a scan point by determining a distance based on the detected measurement radiation reflected from the respective scan point, determining an intensity of the reflected measurement radiation, determining the first angle and the second angle of the scan direction of the respective scan point, and determining a coordinate of the respective scan point based on the determined first angle and second angle and the determined distance.

[0021] The control and processing unit is further configured for image processing a first 2D image of a scan sphere captured with the camera of the laser scanner prior to measuring the scan point based on a machine learning algorithm, such that the image is divided into a plurality of clusters having different predefined classes of measurement importance, and the control and processing unit is for setting at least one adaptive scan parameter depending on the presence and / or absence of clusters of importance classes in the first 2D image, in particular in real time.

[0022] In other words, based on the camera image of the scanner taken prior to the 3D measurement, the settings controlled by the scan parameters are adapted according to the image content, which is indicative of the relevance or meaning of the subsequent measurement process for the point cloud to be created, respectively.

[0023] Optionally, the control and processing unit is configured for capturing and processing a subsequent second 2D image, e.g. in the form of a real-time image stream, and continuously or dynamically adapting the at least one adaptive scanning parameter based on successively determined and / or updated image clusters. This allows for example continuously monitoring the presence or absence of a cluster class or a refinement or correction of a cluster and the scanning parameters associated therewith.

[0024] As a further option, the control and processing unit is configured for a scanning region specific setting of the adaptive scanning parameters, wherein the range of the respective scanning region corresponds to the range of the corresponding cluster. Thus, at least one parameter can be set region-specific, while other parameters can refer to general scanning settings and can be set for the scan as a whole.

[0025] As a further option, the control and processing unit is configured for preconfiguring the setting of the adaptive scanning parameters by selecting one scanning parameter setting mode from a plurality of preconfigured scanning parameter setting modes depending on the presence and / or absence of a cluster of the significance class in the first 2D image. Thus, for example, the perimeter range of the scanning parameters can be preselected based on the presence or absence of a certain "global" class (of clusters), or likewise a set of parameters to be adapted for further class-specific refinement can be preselected.

[0026] In some embodiments, the adaptive scanning parameter controls the emitter and / or the first motor and / or the second motor for controlling or adapting the scanning or measurement density, the scanning speed and / or the intensity and / or the pulse duration and / or the frequency of the emitted measurement radiation.

[0027] Optionally, the significance class comprises a scanning background class (in particular the sky) and / or comprises a vegetation class (more specifically a class of trees, shrubs, leafy wood and / or grass).

[0028] Optionally, the significance class comprises a non-stationary object and / or a currently moving object, more specifically a class of vehicles, humans and / or living beings.

[0029] Optionally, the significance class comprises a building structure, more specifically a class of walls, ceilings, floors, corners, edges, windows and / or doors.

[0030] Optionally, the significance class comprises a light source, more specifically an artificial light source and / or a natural light source; and / or a (highly) reflective surface, more specifically a window pane, a glass plate, a mirror, a car light, a digital board and / or a street sign; and / or a low reflective (or scattering or dark) surface.

[0031] In the above embodiments with parameter-controlled emitters or motors and using classes of scanning background, vegetation, non-stationary or moving objects, for a scanning region corresponding to a cluster of images of one of said classes, the scanning density is set to a (relative) low level (lower density or lower scanning resolution than in at least one other scanning region), which can include zero (no scanning of such scanning region at all; thus, scanning regions corresponding to clusters classified as scanning background, vegetation and / or non-stationary or moving objects are blanked out).

[0032] In the above embodiments with parameter-controlled emitters or motors and using classes of currently moving objects, the scanning is delayed by stopping the first motor (e.g. for a predefined length of time) and / or repeated when image processing of a second 2D image captured after the first 2D image indicates the absence of the (previously present) moving object class. Thus, in addition to or instead of blanking out the scanning region of the moving object class, the entire scan is interrupted until it has been identified in the images that the object has moved out of the scanning environment (determination of no “moving object” class) and / or then at least the respective scanning region is scanned again when no moving object obstructs the object behind it.

[0033] In the above embodiments with parameter-controlled emitters or motors and using different (sub-) classes of building structures, for a scanning region corresponding to a cluster of images of one of said classes, the scanning density is adapted to the specific class of building structure. For example, walls, ceilings, floors are scanned with a different, in particular lower, scanning density than applied to corners and edges and / or windows and / or doors. For example, for windows (panes), the scanning density parameter can be set to zero, i.e. the panes are not scanned at all.

[0034] Optionally, the adaptive scanning parameters are parameters of a camera of the laser scanner and control the camera, in particular the imaging resolution and / or the white balance of the camera, and / or control (apply, e.g. insert into the beam path) an attenuator and / or a filter in front of the detector suitable for the reflected measurement radiation when capturing the point cloud colored and / or textured image.

[0035] The invention also relates to a method of scanning a plurality of scan points representing a surface of an object within a scan sphere with a terrestrial laser scanner, in particular, as claimed for generating a point cloud, wherein the laser scanner comprises a base for mounting the laser scanner, a main body mounted on the base, a first motor configured for rotating the main body about an azimuth axis relative to the base with a first speed, and a first angle encoder configured for determining a first angle of the main body about the azimuth axis, an emitter configured for emitting optical measurement radiation, a deflector carried by the main body and configured for deflecting the measurement radiation emitted in the form of a free beam onto a respective scan point of the object surface, a receiver configured for detecting reflected measurement radiation reflected from the respective scan point back and deflected by the deflector onto the receiver, a second motor configured for rotating the deflector about a pitch axis relative to the main body with a second speed, the second speed being higher than the first speed, a second angle encoder configured for determining a second angle of the deflector about the pitch axis, wherein the first angle and the second angle together define a scan direction, and at least one camera for capturing 2D images, the at least one camera having an optical camera axis in a known spatial relationship to the scan direction.

[0036] The method comprises measuring a scan point by determining a distance based on the detected measurement radiation reflected from the respective scan point, wherein an intensity of the reflected measurement radiation is determined, determining the first angle and the second angle of the scan direction of the respective scan point, and determining a coordinate of the respective scan point based on the determined first angle and second angle and the determined distance.

[0037] The method further comprises image processing a first 2D image of the scan sphere captured with the at least one camera of the stationary laser scanner prior to measuring the scan points based on a machine learning algorithm, such that the image is divided into a plurality of clusters of different predefined importance categories for measurement and the scan is automatically adapted, in particular in real-time, depending on the presence and / or absence of clusters of importance categories in the first 2D image.

[0038] Optionally, the method comprises adapting the scan by adapting an automatic labeling of the measured scan points, in particular in terms of relevance and / or quality, and / or an automatic sorting of the measured scan points. By labeling the scan points, which can also be applied by labeling groups of scan points as a whole, additional information for subsequent (point cloud) processing can be provided, allowing for an adapted, improved further processing.

[0039] As a further option, the measurement radiation is emitted in pulses and the scanning is adapted by adapting the selection of the number of reflection pulses for distance determination to the respective scanning point for scanning regions corresponding to image clusters of the class of light sources, high or low reflective surfaces. For example, the number of reflection pulses or the selection criteria for distance determination can be adapted.

[0040] As a further option, the method comprises, prior to the measurement of the scanning points, automatically verifying the size of the clusters and / or the assignment of the clusters to the importance classes using information collected with the received measurement radiation, in particular the determined 3D coordinates and / or the quality of the received radiation. Thus, a confidence level of the determined cluster size and / or assignment is determined and targeted pre-measurement of low-level confidence regions is performed and / or data of already measured scanning points is used and the verification is performed in real time during the measurement of the scanning points.

[0041] The invention also relates to a computer program product comprising a program code stored on a machine readable medium or embodied by an electromagnetic wave comprising program code segments and having computer executable instructions for performing the claimed method, in particular when executed on a processing unit of the claimed laser scanner. BRIEF DESCRIPTION OF DRAWINGS

[0042] The laser scanner according to the invention and the method according to the invention will be described in more detail below by way of example purely by way of example with reference to exemplary embodiments which are depicted schematically in the drawings.

[0043] More specifically, in the drawings

[0044] Figure 1a , Figure 1b An exemplary example of a stationary terrestrial laser scanner is schematically shown; and

[0045] Figure 2a , Figure 2b A method for scanning an environment is schematically shown which can be performed automatically by a control and evaluation unit of such a stationary terrestrial laser scanner. DETAILED DESCRIPTION

[0046] Figure 1a , Figure 1b An exemplary example of a terrestrial laser scanner 1 for scanning a measurement space is shown. Figure 1a The laser scanner 1 within a scanned environment is depicted, whereas Figure 1b A scheme of an exemplary structure of a distance measurement portion 110 is shown.

[0047] In this example, the measurement space or scanning region is an outdoor setup (street scene) for coordinately measuring the surface of a building / property 11 as a measurement object. As is known in the art, such a laser scanner 1 comprises a base 101 (e.g. a tripod) which is stationed at a location L. An upper part or body 100 of the laser scanner 1 can be motor-drivenly rotated about a vertical or azimuth axis Al (rotation indicated in the figure by arrow Rl) relative to the base 101. A second rotation about a second or elevation rotation axis A2 (indicated in the figure by arrow R2) is provided by a deflector which is embodied as a rotating mirror 102, which can deflect the measurement radiation (e.g. in the form of a laser beam originating from a laser source 104) as a free beam E along a target or scanning direction S into the measurement space. The respective angular positions about the azimuth and elevation axes, respectively or simply the current scanning direction S, can be determined by respective angle encoders.

[0048] In this example, the radiation source 104 can emit a laser beam E as measurement radiation which is pre-focused by a collimator lens 105a as a first optical device and slightly widened incident on a deflection mirror 106 as a further optical device. The laser beam E is reflected from the deflection mirror in the direction of a main objective 105b and guided onto a deflection mirror 107 and from there onto the above-mentioned rotating deflector 102. However, alternatively, the deflection mirror 107 and the rotating deflector 102 can be identical.

[0049] By rotating the mirror 102 about the lateral axis A2 by R2, the elevation scanning direction can be changed and the measurement beam can be directed in a vertical manner on the object surface, while the azimuth direction can be changed and the light beam can be directed horizontally on the surface by rotating the entire upper part 100 about the azimuth axis Al by Rl. Thus, the scanning direction is continuously changed according to a known scanning pattern.

[0050] The degree of rotation about both axes Al, A2 and the (maximum) measurement distance define the measurement space or scanning sphere. Typically, a so-called full-dome scan is performed, which means a scanning sphere with a full rotation in the horizontal direction with an angular range of e.g. a = 270° in the vertical direction with a maximum distance of several tens of meters, one hundred meters or several hundred meters, so that a spherical range is provided which depicts almost the entire surroundings up to a maximum range in all spatial directions. However, any other angular range is also possible.

[0051] After reflection of the light beam E on the surface of the target object, the received light beam R is guided by the rotatable beam deflection unit 102 through the main objective 107 onto a further optical device (mirror 108). The received radiation beam R is reflected from the main objective 107 onto the deflection mirror 106 and from there further guided onto an optical sensor or receiver 109 (e.g. an APD diode) for time-resolved detection of the radiation.

[0052] The control and evaluation unit is data-connected to the distance measuring unit 110 (or to the radiation emitter 104 and the radiation receiver 109 of the scanner 1 and to the above-mentioned angle encoders). The control and evaluation unit is embodied in particular to determine the distance between the laser scanner 1 and the scanning point of the measured object for a plurality of measuring points. Together with the determined scanning direction, the three-dimensional coordinates of the respective scanning point can be determined.

[0053] In addition to the determined distance (or 3D coordinates) from the laser scanner 1 (or from the origin of the reference system (or with respect to the station L)), each scanning point can also have a brightness value, which is likewise determined by the control and evaluation unit. The brightness is a gray value determined, for example, by integrating the bandpass filtered and amplified signal of the radiation receiver over the measurement period assigned to the measuring point. These values form an intensity image of the scan. As an alternative to the representation using gray values, a mapping to colors is known in the art.

[0054] Thus, each object or scanning point is measured, wherein the respective distance to the scanning point is determined as well as the respective alignment of the scanning direction with respect to the two measurement axes Al, A2 (the current horizontal and vertical light beam emission directions) or the two angles. The scanning process thus generates a point set containing three-dimensional information about the surface of the scanned object, such as the building 11. The totality of the measured measuring points of such a measurement is, for example, referred to as a scan and can result in a point cloud. A display device (not shown here), which can be configured as a display on the laser scanner 1 directly or as a display of a connected computer, such as a tablet or smartphone, can be connected to the control and evaluation unit.

[0055] Optionally, a digital image, such as a high-resolution image of at least 1 MP, can also be additionally generated by means of one or usually a plurality of color or RGB cameras 103 of the scanner 1, which allows assigning additional color values to the measuring points to provide a textured or color point cloud. Such terrestrial laser scanners 1 can thus be equipped with one or more cameras 103 for taking images or providing a video stream of the surroundings during the scanning workflow. Furthermore, such surveying devices 1 with one or more cameras 103 are also known, which allow recording wide-angle or panoramic images of the measurement space.

[0056] Such a scan means that e.g. millions of surface points are measured by continuously sweeping the measurement beam E and continuously measuring instead of a directed measurement of a single point, the measurement space is recorded in a dense manner or completely, meaning that any object (surface) within the measurement space is scanned. This is especially true for stationary or non-moving objects such as the building 11 with the window 12, the tree 14 or the road sign 15, but also for movable or moving objects such as the depicted vehicle 13, as long as within the scan sphere (or hit by the measurement beam).

[0057] Not all of these objects 11-15 have the same relevance or importance or interest for the measurement. Some objects such as the car 13 or the tree 14 can even be in the line of sight from the scanner 1 to the area of interest and thus can obscure the object of interest.

[0058] The invention provides a laser scanner 1 (or scanning method) which provides a distinction between different objects or parts of the scan sphere in terms of the importance of the scan and enables to adapt to this during or before the actual scan / data acquisition 3d point measurement, as described below.

[0059] Figure 2a , Figure 2b A method for scanning an environment is schematically shown. The method can be performed automatically by the control and evaluation unit of a stationary terrestrial laser scanner as described above, in particular as part of the scanning function of the laser scanner.

[0060] Figure 2a A processed 2D image 10 of (at least a part of) the environment to be scanned, taken by the above-mentioned internal camera of the laser scanner before the 3D point measurement, preferably after the laser scanner is set up and directly before the scan point acquisition is initiated, is shown. Preferably, a full 360° vision is provided using a panoramic image and / or one or more cameras of the laser scanner. In the picture, the above-mentioned objects (see Figure 1a ) (i.e. the building, the tree, the street sign and the vehicle), as well as the (pre)ground and the sky in the background are imaged, which can be considered as different layers of the scan sphere.

[0061] For example, using a Neural Processing Unit (NPU) on the laser scanner as part of the scanner’s control and evaluation unit, the image 10 is image-processed based on a machine learning algorithm for partitioning or segmenting the image content according to predefined importance categories. The algorithm can be trained on 2D images of the scanned scene taken by the laser scanner camera, wherein 3D scan data of the respective scene can be considered, for example, to improve the accuracy or precision of the definition of cluster sizes or boundaries in the 2D images. These importance categories are machine-learned and used to partition or segment the image 10 into clusters 1, 2a, 2b, 3, 4, 5, 6, 7 as shown in Figure 2a

[0062] As shown in Figure 2b , the clusters 1-7 are associated with importance categories C1, C2a, C2b, C3, C4, C5, C6, and C7. In this example, the importance category C1 is “building” and the respective cluster 1 covers the part of the image 10 representing the building 11. The importance categories C2a is “window” and C2a is “street sign”, wherein both categories can be unified or summarized in a higher category “(highly) reflective object” such that all five clusters 2a, 2b can be related to the same category. However, in this example, these reflective objects (or image clusters 2a, 2b determined according to the related categories C2a, C2b) are further distinguished and treated separately as described below. Further categories can be represented as C3 “vehicle”, C4 “vegetation” (or “tree”), C5 “ground”, C6 “sky”, and C7 “sun”. Thus, the image 10 is processed in a way that determines the presence or absence of the image clusters 1-7 of the predefined scan importance categories C1-C7.

[0063] Adapting the subsequent scan / 3D measurement based on the determined presence or absence of the image clusters 1-7 (or imaged objects of the respective importance categories). In other words, the scanner controller provides at least one scan parameter that is adaptable according to the presence or absence of the classified image clusters.

[0064] In this example, one exemplary adaptive scan parameter according to the determined importance clusters is the scan density that is schematically indicated in Figure 2b by different graphical patterns. The scan density is adapted specific to scan regions (i.e. parts or partitions of the scan sphere) that are determined based on the known position / space relationship of the image (or scanner camera) and the beam emission (or scan) direction from the position and size of the image clusters 1-6.

[0065] As a result, the area of the high importance category C1 and thus the main measurement object (building 1) is scanned with a (relatively) high density (in Figure 2b ​other objects (or other regions) of lower importance (such as tree 14, street sign 15 and ground) are measured with lower scanning density (indicated in the middle by a coarse or less dense pattern). Figure 2b

[0066] The scanning can be further refined by defining and applying (sub) classes of (sub) regions of interest for larger objects / regions such as buildings, work pieces, etc. For example, classes such as "window (frame)", "door", "edge", "wheel", "frame", etc. can be defined / trained and used for image processing of the 2D images 10 to determine respective image clusters. Based thereon, such scanning regions can be dynamically and specifically scanned with higher density to obtain more detail.

[0067] In other words, the importance classification can have different levels of detail for a more differentiated scanning adaptation, allowing for example to scan specific structures of a building 11 defined by respective classes with higher resolution compared to the remaining part of the building 11 (these parts referring for example to the main or higher class "building" from which the sub-class clusters are subtracted).

[0068] For example, surfaces of low geometric detail such as "wall" or other flat regions can be classified by lower importance and thus scanned using a low scanning density adapted, although they are part of or located within a region of high importance (higher or larger) in itself. The identification and detailed classification of such (sub) regions allows for a more precise adaptation of the scanning density parameters, where even more importance differentiation or importance distinction of the scanning results is provided, for example with a higher number of relevant points (higher accuracy for important objects) and / or further reduced, less unnecessary scanning points or data.

[0069] On the other hand, the scanning density for unimportant objects / regions can even be reduced to zero, in this example, the vehicle 13 and / or the sky 16 are not scanned at all, or this scanning region C3 / C6 is omitted (indicated in the middle by a blank "pattern"). Figure 2b

[0070] The scanning parameters such as the scanning density can be adapted during the measurement. Here, by respective control of the scanning beam emission and steering, the scanning density can be dynamically set, for example for lower density, a wider angular measurement step or higher scanning speed can be set (the scanning beam is rotated faster with constant measurement rate), or for zero scanning density, the scanning beam can be switched off.

[0071] ​​Dynamic adaptation during the point measurement can also be applied to the cluster dimension and / or to the assignment of the classes themselves. That is, the information collected during the 3D point measurement (in particular the 3D coordinates of the points themselves) can be used to verify whether the clusters are correctly sized or whether their boundaries have to be refined. For example, the distance of an object can be used to classify the object (sub) as "far away" (beyond a defined distance, e.g. a fixed value and / or a range with respect to the measurement space) for a different adaptation compared to objects of the same class closer to the scanner (this distance-based differentiation can also be based on a distance implicitly estimated from the 2D image itself based on the size of the imaged object or the number of pixels). The classification can also be verified, e.g. by checking the reflectivity based on the intensity of the received radiation.

[0072] The verification or checking of the classification or clustering can also be done beforehand by utilizing a pre-measurement of the scanning beam. Here, a confidence level of the assignment of the important class of a cluster is determined and if the confidence level is below a certain confidence threshold, i.e. the class cannot be determined with sufficient confidence based on the 2D image data, a directed 3D measurement of the region in question is automatically triggered. Thereby, the generated scanning data (in particular the distance values and / or the intensity of the scanning surface) then allows the assignment of a class with an acceptable confidence level. Thus, the availability of the 3D imaging device is used in case of uncertainty of the classification based on the 2D image.

[0073] Generally, sensor data of other sensors of the scanner can also be used for adapting or checking the classification and / or the cluster construction. Although a fully automatic execution of the method is preferred, the uncertainty of the classification can also be resolved by a user who can for example confirm or reject a class or make a selection between classes proposed by the scanning software.

[0074] As a further option, further 2D images can be captured (in particular in real-time during the 3D measurement) for the verification or real-time adaptation or refinement of the classification or clustering. For example, a real-time video stream captured in parallel to the point measurement can also be used for the dynamic adaptation of the adaptive scanning parameters.

[0075] The redefinition or adaptation of the clusters / classes based on subsequent 2D images can also be used for non-stationary objects (or clusters of class C3) such as a vehicle 13 to verify whether such clusters / classes still exist. If for example in a subsequent image, the cluster of class C3 does not exist (because the vehicle has moved out of the scanning range), an updated classification without the class "vehicle" results. This adapted image cluster is then used to update the scanning parameters, e.g. if the area has not been directed before, what was previously the content of cluster 3 is not omitted / blanked but measured or a second measurement (e.g. a new full scan or a scan specifically directed to the area defined by cluster 3) is initiated to fill the previously omitted scanning area.

[0076] The capturing of the subsequent 2D images can itself be part of the adaptation based on the importance classification. That is, a decision can be made if further 2D images are to be captured or not depending on the determined class Ci-C7. The classes Ci-C7 can be considered to be indicative of a recent change in the scanned environment. For example, a vehicle can be classified (first and at least up to that point only 2D image) as a "parked vehicle" or a "(potentially) moving vehicle" (e.g. classified as the former when a car is identified on a parking spot and as the latter when a car is identified on a street lane and / or a driver is identified in the imaging vehicle). Then and only in case of the latter class (a vehicle that is actually or potentially moving soon) a further 2D imaging is initiated at some time later. Therein, this time parameter can also be adapted e.g. depending on a sub-class of "moving vehicle" or "moving object" such as "(fast) car", "(medium) bicycle" and "(slow) pedestrian".

[0077] Another example of an importance class that can lead to an adaptation can be a movable object like a door or a window: (only) if there is a cluster of the class "open door / window" a subsequent 2D imaging can be automatically triggered, in particular for observing whether the door or window has been closed. Additionally or alternatively, parameters of a control user messaging can be adapted. In this case, for example, the user is informed that there is an open door or window so that he can react and close the door / window if deemed necessary. Other examples of classes indicative of a moving or movable object are feasible, such as an animal or a portable object.

[0078] With regard to an adaptive scanning parameter such as the scanning density, as an alternative or in addition to the above, the scanning density can not be adapted before the collection of point measurement data but after the measurement of the scanning points. For example, during the measurement scanning points can be eliminated on the fly in that only every nth acquired scanning point is stored for a scanning region corresponding to a cluster of low importance. In other words, in this case there is no need to adapt the beam emission but a uniform or standard beam control can be applied since the adaptation is on the side of the data evaluation or processing. Thus, for example, a scanning region that is classified as "vegetation" or "ground" or the like is thinned by deleting or averaging the points in real time so that a point cloud of 3D point clusters of different densities is generated.

[0079] In addition or instead of an immediate dynamic or real-time adaptation, i.e. after the scan (or point cloud completion), a region-specific (or importance-specific) adaptation of the scanning density or other scanning (or point cloud) parameters can also be applied by the scanner to the already generated point cloud.

[0080] Within the scope of the method, this subsequent adaptation of the point cloud with respect to the image-based determination of the importance of regions or objects can also be postponed as "post-processing", for example for processing outside the scanner. Thus, the marking of the scanning points providing information for such post-processing can be applied as a further example of scanning adaptation which can be changed depending on the importance category. In this example, the scanning regions C2a, C3 and C4 representing the window 12, the vehicle 13 and the tree 14 are marked by the respective labels T1, T2, T3.

[0081] This adaptive automatic marking results for example in the labels T1, T2, T3 which can represent the respective scanning region or point cloud cluster as "window" or "reflective surface" (T2), "car", "vehicle" or "non-stationary object" (T1), "vegetation" or "tree" (T3), or the software can process it as any such label or symbol of information or meta-information for the point cloud processing.

[0082] For example, the marking of the point cloud region with the label T1 (or T3 also with respect to leaves which can move in a windy environment) with respect to scanning points of a movable or moving object can increase the robustness of the point cloud registration. For example, such a marking enables the registration software to ignore or weight such points of the "non-stationary" category, for example using an ICP algorithm (Iterative Closest Point). Other examples of labels can relate to such importance ("low", "high", for example also "unknown") without relating to the above type of object.

[0083] A further example of an importance category which improves the on-site and / or post-processing point cloud processing is the reflectivity. For example, the determined regions of the categories C2a, C2b representing "high reflectivity" (or, that is, implying such high reflectivity when classified as "window" or "street sign") provide information for implementing a scanning refinement. For example, this allows to adapt the point cloud filtering (or the point cloud filter) as a scanning parameter for reducing the reflection or multi-path effects or to adapt (in particular to reduce) the pulse energy. Thus, for example, the filter process can be adapted and applied to the point cloud in a directed or more specialized manner, or more specifically to dedicated parts of the point cloud, which would not be possible without the image-based classification.

[0084] Thus, the fine classification between "street sign" and "window" (instead of only one category "high reflective object") allows a further differentiated adaptation of the scanning. For example, different filter parameters can be set, or the same filter is set while differentiating the scanning of the different categories with respect to other scanning parameters such as the scanning density.

[0085] Another example of the importance of adaptive scan parameters relates to measuring radiation. In addition to setting the scan radiation intensity by reducing it to zero, as already mentioned, the intensity can also be adapted to the category of surface reflectivity.

[0086] On the other hand, in the case of a highly reflective surface (such as a "window"), for example, pulsed radiation can be used to adapt the radiation assessment, where the first reflection of a set of pulses is used for image determination, instead of using the strongest reflective pulse for other categories of scan points. In other words, the measurement pulse selection mode is adapted in response to the presence or absence of an object category (or importance category). In particular, WFD scanners are capable of evaluating multiple pulse returns. The present invention can apply importance / object category-related adaptations to parameters selected from these multiple returns.

[0087] exist Figure 2b The bottom (reference numeral 8) schematically illustrates another example of the presence or absence of an object using specific meanings. Here, the presence of category C7 (“(artificial or natural) light source” or “sun”) is determined and used for parameter adaptation. In this example, the presence of a cluster of category C7 is detected, and in response to this detection, a set of parameters is used. For example, parameters of a laser scanner’s camera are set to capture images for coloring or texturing point clouds, such as automatic white balance parameters W for consistent coloring across (more) scanning stations or settings.

[0088] As another example, in Figure 2b The bottom of the figure, schematically indicated by reference numeral 9, shows that parameter S1 is automatically selected from a set of parameter settings S1, S2, S3, as an adaptation for scanning based on the presence (or absence) of a category (such as C6) depending on the scanner's controller. For example, the presence of category C6 "sky" triggers the selection of camera parameters, such as the image resolution (e.g., LDR) used for the aforementioned point cloud coloring. On the other hand, the determination of category "indoor objects" can trigger a resolution adaptation for HDR to capture such a color image.

[0089] Selecting from different predefined settings S1-S3 based on categories such as C6 / “Sky” can also serve as a general pre-fit. That is, for example, parameter ranges, parameter thresholds, or different sets of parameters from one setting S1-S3 to another can be automatically pre-selected based on general importance categories such as “Sky” or “Sun (Light)”. Then, based on other (parameter)-specific categories, these parameters are further adapted or specified within the corresponding available or permitted ranges.

[0090] For example, the overall scan rate minimum (or maximum) S1 is selected from a plurality of scan rate minimums S1-S3 according to a global importance category, and then the scan rate is further refined individually for the further categorized clusters / scan areas, observing the minimum scan rate, such that all scan rates are higher than the lower value.

[0091] It goes without saying that these shown figures only schematically show possible exemplary embodiments. The various methods can also be combined with one another according to the application and, if not mentioned otherwise, with corresponding devices and methods of the prior art.

Claims

1. A terrestrial laser scanner for coordinatedly measuring a plurality of scan points representing a surface of an object within a scan sphere, the terrestrial laser scanner comprising a base for mounting the laser scanner, a main body mounted on the base, a first motor configured for rotating the main body relative to the base about an azimuth axis at a first speed, a first angle encoder configured for determining a first angle of the main body about the azimuth axis, an emitter configured for emitting optical measurement radiation, a deflector carried by the main body and configured for deflecting the emitted measurement radiation in the form of a free beam onto a respective scan point of an object surface, a receiver configured for detecting reflected measurement radiation reflected from the respective scan point back and deflected by the deflector onto the receiver, a second motor configured for rotating the deflector relative to the main body about a pitch axis at a second speed, the second speed being higher than the first speed, a second angle encoder configured for determining a second angle of the deflector about the pitch axis, the first angle and the second angle together defining a scan direction, a control and processing unit configured for measuring a scan point by determining a distance based on the detected reflected measurement radiation from the respective scan point, wherein an intensity of the reflected measurement radiation is determined, determining the first angle and the second angle of the scan direction of the respective scan point, and determining coordinates of the respective scan point based on the determined first angle and second angle and the determined distance, at least one camera for capturing 2D images, the at least one camera having an optical camera axis in a known spatial relationship to the scan direction, characterized in that the control and processing unit is configured for image processing a first 2D image of the scan sphere captured with the at least one camera of the laser scanner prior to measuring a scan point based on a machine learning algorithm, such that the image is divided into a plurality of clusters having different predefined classes of measurement importance, setting at least one adaptive scan parameter depending on the presence and / or absence of clusters of importance classes in the first 2D image.

2. The laser scanner according to claim 1, characterized in that the control and processing unit is configured for a scan region specific setting of the adaptive scan parameter, wherein a range of a respective scan region corresponds to a range of a corresponding cluster, and / or a pre-configuration of the setting of the adaptive scan parameter by selecting one scan parameter setting mode from a plurality of pre-configured scan parameter setting modes depending on the presence and / or absence of clusters of importance classes in the first 2D image.

3. The laser scanner according to any of the preceding claims, characterized in that the adaptive scan parameter controls at least one of: the emitter, the first motor and / or the second motor. ​ ​ ​ ​ ​ ​ ​ wherein, ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ for adapting at least one of: a scan density, a scan speed, an intensity and / or pulse duration and / or frequency of emitted measuring radiation.

4. Laser scanner according to any of the preceding claims, characterized in that the importance category comprises at least one of: a scan background, in particular sky, vegetation.

5. Laser scanner according to any of the preceding claims, characterized in that the importance category comprises at least one of: a non-stationary object, a currently moving object.

6. Laser scanner according to any of the preceding claims, characterized in that the importance category comprises a building structure.

7. Laser scanner according to any of the preceding claims, characterized in that the importance category comprises at least one of: a light source, in particular artificial and / or natural light source, a highly reflective surface, a lowly reflective surface.

8. Laser scanner according to claim 3 and 4 or claim 3 and 5, characterized in that for a scan region corresponding to an image cluster of one of the categories, the scan density is set to a relatively low level, in particular to zero, and / or in case of a currently moving object category, a scan is delayed and / or repeated by stopping the first motor when image processing of a second 2D image captured after the first 2D image indicates an absence of a moving object category.

9. Laser scanner according to claim 3 and 6, characterized in that the scan density is adapted to a specific category of building structure.

10. Laser scanner according to any of the preceding claims, characterized in that the adaptive scan parameter is a parameter controlling: the camera when capturing a point cloud colored and / or textured image, in particular controlling an imaging resolution and / or white balance, and / or an attenuator and / or a filter, the attenuator and / or the filter being applicable in front of the detector for reflected measuring radiation.

11. Method for scanning a plurality of scan points representing a surface of an object within a scan sphere with a terrestrial laser scanner, the laser scanner comprising a base for stationing the laser scanner, a main body mounted on the base, a first motor configured for rotating the main body relative to the base about an azimuth axis at a first speed, a first angle encoder configured for determining a first angle of the main body about the azimuth axis, an emitter configured for emitting optical measuring radiation, a deflector carried by the main body and configured for deflecting the measuring radiation emitted in the form of a free light beam onto a respective scan point of an object surface, a receiver configured for detecting reflected measuring radiation reflected back from the respective scan point and deflected by the deflector onto the receiver, a second motor configured for rotating the deflector relative to the main body about a pitch axis at a second speed, the second speed being higher than the first speed, a second angle encoder configured for determining a second angle of the deflector with respect to the tilt axis, wherein, the first angle and the second angle together defining a scan direction, at least one camera for capturing 2D images, the at least one camera having an optical camera axis in a known spatial relationship to the scan direction, the method comprising measuring a scan point by: determining a distance based on detected measurement radiation reflected from the respective scan point, wherein an intensity of the reflected measurement radiation is determined, determining a first angle and a second angle of the scan direction of the respective scan point, determining a coordinate of the respective scan point based on the determined first angle and second angle and the determined distance, characterized in that the method comprises: based on a machine learning algorithm, image processing of a first 2D image of the scan sphere captured with the at least one camera of the laser scanner prior to measuring a scan point, so that the image is divided into a plurality of clusters of different predefined importance categories for measurement, automatically adapting the scan depending on the presence and / or absence of clusters of importance categories in the first 2D image.

12. The method according to claim 11, characterized in that the scan is adapted by adapting: an automatic labeling of measured scan points, in particular with respect to relevance and / or quality, and / or an automatic sorting of measured scan points.

13. The method according to claim 11 or 12 for a laser scanner according to claim 7, characterized in that the measurement radiation is emitted in pulses, and for a scan region corresponding to an image cluster of one of the categories, the scan is adapted by adapting the selection of reflected pulses for the respective scan point used for distance determination.

14. The method according to one of claims 11 to 13, characterized in that prior to the measurement of the scan points, by a determination of cluster sizes and / or assigned confidence levels and an orientation prediction of low-level confidence regions, and / or during the measurement of scan points using data of already measured scan points, using information gathered with received measurement radiation, in particular determined 3D coordinates and / or quality of received radiation, the size of clusters and / or the assignment of clusters to importance categories is automatically verified.

15. A computer program product comprising program code stored on a machine-readable medium or embodied by an electromagnetic wave comprising program code segments and having computer-executable instructions for performing the method according to claim 11, in particular when executed on a processing unit of a laser scanner according to claim 1.

Images