Method and system for autonomous exploration and scanning
By defining a three-dimensional exploration map and dividing exploration blocks, UAV can independently explore and scan complex surface objects, solving the problems of low scanning shadows and power utilization efficiency in the existing technology, and achieving efficient and complete scanning exploration effects.
Patent Information
- Application Number
- CN202411607298.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-11-20
- Filing Date
- 2024-11-12
- Publication Date
- 2025-05-20
AI Technical Summary
When existing unmanned aerial vehicles (UAVs) independently scan and explore complex surface objects, they are prone to scan shadows and excessive power consumption, making it difficult to effectively utilize the power provided by the battery.
By defining a three-dimensional exploration map, the object is divided into multiple three-dimensional exploration blocks, and the mobile robot is traveling along the exploration path. Scan data related to point clouds is generated through the laser scanning module, the exploration map is updated and the exploration path is defined, ensuring that UAV can independently explore and scan objects of interest.
It realizes autonomous and effective scanning exploration of UAV on complex surface objects, reduces scanning shadows, improves power utilization efficiency, and ensures the integrity and efficiency of UAV in the exploration process.
Smart Images

Figure CN120020669A_ABST
Abstract
Description
Technical Field
[0001] The present invention generally relates to a mobile robot, in particular an unmanned aerial vehicle (UAV), which includes a laser scanner module for scanning the surface of an object of interest. More specifically, the present invention relates to a method and system for autonomously exploring and scanning an object of interest by one or more mobile robots. Background Art
[0002] UAVs are being developed to manage various tasks in technical and non-technical fields: recording movie scenes, transporting goods, inspecting buildings and technical facilities, and surveying, measuring, and / or digitizing the physical environment. For example, WO 2022 / 268316A1 discloses a rotary wing drone type UAV having a laser scanner module for inspecting, surveying, measuring, and digitizing the environment of the UAV. Such a UAV is a flying laser scanner that can reach locations inaccessible to fixed scanners (such as rooftops, trees, building facades, etc.) and can scan its surrounding environment during flight. It would be advantageous to facilitate the inspection, surveying, measuring, and digitizing of an object of interest. It would be particularly advantageous if the UAV could explore the object completely autonomously, i.e., without any user interaction after initially defining the operating area of the UAV.
[0003] Existing methods for autonomous or semi-autonomous scanning exploration by UAVs generally work well for simple (e.g., box-shaped) objects but are difficult to handle objects with complex surfaces (e.g., bridges or buildings, including canopies or complex facades). Typically, scanning such complex objects will leave scanning shadows and / or take an excessive amount of time to complete. Since the battery of the UAV is a limiting factor, it is important to effectively utilize the provided power during exploration. Summary of the Invention
[0004] Accordingly, it is an object of the present invention to provide an improved method and system for scanning the surface of an object of interest.
[0005] A particular object is to provide such a method and system that allows a UAV or other mobile robot to perform scanning completely autonomously.
[0006] A particular object is to provide such a method and system that allows a UAV or other mobile robot to autonomously explore a user-defined volume including an object of interest.
[0007] At least one of these objectives is achieved by the method according to the first aspect of the present invention, the UAV according to the second aspect of the present invention, and / or other embodiments of the present invention.
[0008] The first aspect of the present invention relates to a computer-implemented method for a mobile robot to autonomously explore one or more objects of interest. The mobile robot includes a computing unit and a laser scanner module for scanning the surface of one or more objects of interest. The laser scanner module has a field of view (FOV). The method includes the following steps:
[0009] - Defining a three-dimensional exploration map, where one or more objects of interest are located in the exploration map;
[0010] - Dividing the exploration map into a plurality of three-dimensional exploration blocks; and
[0011] - Autonomously exploring the exploration map by the mobile robot, which includes: generating scan data related to the point cloud through the laser scanning module while the mobile robot travels along the exploration path.
[0012] According to this aspect of the present invention, exploring an exploration block at least includes determining whether the corresponding exploration block includes one or more points of the point cloud, and the computing unit of the mobile robot updates the exploration map and defines the exploration path.
[0013] According to some embodiments, the mobile robot is an unmanned aerial vehicle (UAV), such as a quadcopter drone.
[0014] According to some embodiments, the FOV depends on the position and orientation of the UAV and is limited by the position and orientation of the laser scanner module relative to the UAV and a predefined radius less than the maximum scan range of the laser scanner module. For example, the maximum scan range of the laser scanner module can be between 15 meters and 150 meters, particularly between 40 meters and 80 meters.
[0015] In some embodiments, the FOV is also limited by the position and orientation of the laser scanner module relative to other features of the UAV (such as rotors and wings).
[0016] According to some embodiments, the exploration map is updated at time intervals selected according to at least one of the scanning speed of the laser scanner unit, the computing speed of the computing unit, and the current speed of the mobile robot. For example, these time intervals can be between 100 ms and 1 s.
[0017] According to some embodiments, updating the exploration map includes, at each of a plurality of different time points:
[0018] - Obtain the scan data generated by the laser scanning module;
[0019] - Based on the retrieved scan data, identify an exploration block that is at least partially located in the past FOV and includes one or more points of the point cloud (the past FOV is the FOV of the laser scanner module at a previous time point before the corresponding time point among multiple different time points);
[0020] - Define the identified exploration block as an occupied block;
[0021] - Extract cone information of each occupied block that is at least partially located in the past FOV, where the cone information is related to the cone defined by the position of the laser scanner module at the corresponding previous time point and the boundary of the corresponding occupied block;
[0022] - Extract cone information of each unexplored block that is completely located in the past FOV, where the cone information is related to the cone defined by the position of the laser scanner module at the corresponding previous time point and the boundary of the corresponding unexplored block; and - For each unexplored block among the unexplored blocks that are completely located in the past FOV, based on the cone information of the corresponding unexplored block and the cone information of the occupied blocks that are at least partially located in the past FOV, determine whether the line of sight between the center of the past FOV and at least a part of the corresponding unexplored block is blocked by one or more occupied blocks.
[0023] In some embodiments, determining whether the line of sight is blocked includes determining whether the cone of the corresponding unexplored block overlaps with one or more cones of the occupied blocks, and if it overlaps, determining whether the distance from the center of the past FOV to the center of the corresponding unexplored block is greater than the distance from the center of the past FOV to the center of the corresponding occupied block.
[0024] According to some embodiments, store information about the minimum distance and direction to the nearest occupied block or the nearest point of the point cloud for each exploration block, and updating the exploration map further includes:
[0025] - For each block that has been defined as an occupied block at the most recent different time point, determine the distance and direction between the corresponding occupied block and each unoccupied exploration block within the defined radius around the corresponding occupied block; and
[0026] - If the determined distance is less than the stored minimum distance for the block, update the stored minimum distance and direction with the determined distance and direction.
[0027] For example, the radius can be defined based on the maximum or optimal scan range of the laser scanner module and / or based on the size of multiple 3D exploration blocks.
[0028] According to some embodiments, updating the exploration map further includes defining unexplored blocks that are completely within the past field of view as free blocks, and defining free blocks adjacent to the unexplored blocks as boundary blocks, where the cones of the unexplored blocks do not overlap with one or more cones of the occupied blocks. In these embodiments, defining an exploration path includes assigning scores to the boundary blocks and assigning the next exploration target to the mobile robot based on the scores assigned to the boundary blocks. For example, the boundary block with the highest score is assigned as the next exploration target. If the cones overlap, but the unexplored block is in front of the occupied block, the block can also be set as free.
[0029] For example, scores are assigned based on the following items:
[0030] - The distance between the corresponding boundary block and one or more occupied blocks,
[0031] - The total distance and / or relative vertical distance between the corresponding boundary block and the current position of the UAV, and / or
[0032] - Based on the number of occupied blocks within a defined radius around the corresponding boundary block.
[0033] According to some embodiments, defining an exploration path includes defining a subset of free blocks that are not boundary blocks as scan target blocks, and also assigning scores to the scan target blocks. In these embodiments, assigning the next exploration target to the mobile robot is based on the scores assigned to the boundary blocks and the scores assigned to the scan target blocks, where the block with the highest score is assigned as the next exploration target. For example, scores are assigned to the scan target blocks based on the position of the scan target blocks relative to the surface of the object of interest and their potential to provide good scan results.
[0034] According to some embodiments, assigning scores includes assigning boundary scores to the boundary blocks and assigning scan target scores to the scan target blocks, where a first weight is multiplied by each of the boundary scores in the boundary scores, and a second weight is multiplied by each of the scan target scores in the scan target scores. Optionally, the first weight and the second weight can be user-selectable.
[0035] According to some embodiments, defining scan target blocks includes defining multiple surface scan cones for each occupied block, and for each free block whose center point is within the surface scan cone, determining whether the line of sight to a part of the corresponding occupied block is blocked. Then those free blocks whose center points are within the surface scan cone and whose line of sight to the corresponding occupied block is not blocked are defined as scan target blocks. In some embodiments, when the center of the FOV enters the corresponding surface scan cone, the surface scan cone is marked as explored.
[0036] According to some embodiments, the 3D volume is defined by the user as a 3D exploration map in a graphical user interface (GUI), which can be displayed, for example, on the screen of a mobile computing device, showing a 2D representation of one or more objects of interest. For example, the GUI can allow the user to define a polyhedron, specifically a cuboid, as a three-dimensional exploration map by marking two corner points of the cuboid.
[0037] According to some embodiments, exploring an exploration block includes scanning the surface of an object present in the exploration block by a laser scanner module, for example, with a scanning accuracy defined by the user. For example, one or more objects of interest include buildings or other man-made structures (the surface includes at least one of facades, roofs, pillars, and road surfaces) and / or natural objects or scenes, such as caves.
[0038] A second aspect of the present invention relates to a UAV, which includes a computing unit and a laser scanner module having a FOV, wherein,
[0039] - the computing unit is configured to receive a 3D exploration map or information on allowing the calculation of a 3D exploration map (i.e., at least volume coordinates), wherein one or more objects of interest are located in the exploration map, and wherein the exploration map is divided into a plurality of three-dimensional exploration blocks;
[0040] - the UAV is configured to perform autonomous exploration of the exploration map based on the received exploration map, for example, according to the method of the first aspect, wherein exploring the exploration map includes the laser scanner module generating scan data related to a point cloud while the UAV travels along an exploration path,
[0041] - exploring an exploration block at least includes determining whether the corresponding exploration block includes one or more points of the point cloud based on the scan data; and
[0042] - the computing unit of the UAV is configured to update the exploration map and define an exploration path.
[0043] A third aspect of the present invention relates to a system for scanning the surface of one or more objects of interest, which includes a UAV according to the second aspect and a mobile computing device, wherein the mobile computing device is configured to receive user input defining a 3D exploration map, wherein one or more objects of interest are located in the exploration map, and provide information on the exploration map to the UAV. The system can be configured to perform the method according to the first aspect of the present invention.
[0044] A fourth aspect of the present invention relates to a computer program product comprising program code having computer-executable instructions for performing the method according to the first aspect, in particular for performing the method according to the first aspect when running in a computing unit of a UAV according to the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The present invention will be described in detail below by way of exemplary embodiments with reference to the accompanying drawings, in which:
[0046] Figure 1 An exemplary embodiment of a UAV according to the present invention is shown;
[0047] Figure 2 Shows Figure 1 the field of view of the sensors of the UAV;
[0048] Figure 3 A mobile computing device forming part of an exemplary embodiment of a system according to the present invention is shown, allowing a user to define a volume as a 3D exploration map;
[0049] Figure 4 Illustrates the start of an exemplary exploration and scanning process of a UAV within a defined volume;
[0050] Figure 5a and Figure 5b shows the UAV moving through the volume;
[0051] Figures 6a to 6c Illustrates an exemplary process of updating the exploration map;
[0052] Figure 7 Illustrates an exemplary exploration planning process;
[0053] Figure 8 Is a flowchart illustrating an exemplary embodiment of a method for autonomously exploring an object of interest by a UAV according to the present invention;
[0054] Figure 9 Is Figure 8 a flowchart illustrating an exemplary process of updating the exploration map as part of the method;
[0055] Figure 10 Is Figure 8 a flowchart illustrating an exemplary exploration planning process as part of the method;
[0056] Figure 11 Illustrates a first example of a scanning position resulting in a poor scan result;
[0057] Figure 12Illustrates a second example of a scanning position that results in a poor scanning result;
[0058] Figure 13 Illustrates selecting the next flight target based on the assigned scores and weights; and
[0059] Figures 14a to 14c Illustrates defining a scanning target block. DETAILED DESCRIPTION
[0060] Figure 1 Illustrates an exemplary embodiment of an unmanned aerial vehicle (UAV) 1 as an example of a mobile robot according to the present invention. The illustrated UAV 1 is a rotary-wing drone type UAV having four rotors. The UAV 1 includes a laser scanner module 10 (such as a lidar module) that is configured to inspect, survey, measure, and digitize the environment of the UAV. As shown here, the laser scanner module 10 may include a single laser scanner. Alternatively, two or more laser scanners may be provided at different positions on the UAV 1.
[0061] Additional sensors may be provided on the UAV 1. Multiple sensors can be used to prevent collisions with objects. These sensors may include the laser scanner module 10. In addition, radar sensors (e.g., mounted on the left, right, and rear of the shroud) can be used. For example, a camera sensor that allows navigation through the environment without colliding with obstacles (e.g., other objects in the area of interest or exploration area) can be provided. The sensors can be configured to perform a simultaneous localization and mapping (SLAM) function as the UAV moves through the exploration area. For example, the camera can be used for state estimation of the UAV (visual inertial odometry). Optionally, a camera that captures color images can also allow adding more information to the scan, particularly the coloring of the points in the point cloud.
[0062] Figure 2 Illustrates the UAV 1 with several fields of view of its sensors (not to scale). In the example shown, three camera sensors are provided on the UAV 1, each camera sensor having a camera field of view 17a to 17c. Other camera setups can include more cameras, for example also pointing up and down. Depending on its setup and position on the UAV, the laser scanner unit can have a spherical or substantially spherical field of view 15. However, certain portions of the field of view will generally be blocked by other parts of the UAV, such as the fuselage and rotors. Optionally, such occlusion can be prevented or reduced by a laser scanner unit that includes two or more separate laser scanners.
[0063] Figure 3 Illustrates in connection with Figure 1A mobile computing device 2 for use with a UAV. For example, the device 2 can be configured as a tablet computer, a smart phone, or a laptop computer. It includes means for wireless data exchange with the UAV. These means can include one or more of Bluetooth, WiFi, and mobile (cellular) radio. The device 2 includes a screen 20 that provides a graphical user interface (GUI) (e.g., configured as a touch screen). In the illustrated embodiment, the GUI shows a 2D map of the surrounding environment, where the surrounding environment includes one or more objects of interest that the user wants the UAV to scan, such as a house, a bridge, or a church.
[0064] The GUI allows the user to define a three-dimensional exploration volume Figure 3 as a volume that encloses one or more objects of interest. Information about this volume 3, in particular its 3D coordinates, is provided to the UAV via a wireless data connection, and then the UAV autonomously explores volume 3 while scanning the scene that includes the object of interest in an optimal manner. For example, a software application (app) installed on the device can provide the GUI, receive user input about volume 3, and establish a wireless data connection with the UAV.
[0065] Volume 3 can be defined, for example, by marking the top-view area that contains the object of interest on the 2D map view and then defining the height of the volume with a slider. Volume 3 can be polygonal. Additionally, any other form of volume definition can be used. For example, a volume can be selected based on an existing point cloud or CAD model or any other 3D model. For example, after selecting the boundaries of the volume on the 2D map (i.e., latitude and longitude coordinates), the user can then define the height of the volume. By default, the bottom of the volume can be set to the height before the UAV takes off. If needed, the user can also move the volume up or down to change the global vertical position of the volume (e.g., using a slider).
[0066] Of course, the exploration volume can also be defined in a 3D view. For example, when loading a 3D pre-scan of one or more objects, the exploration volume can be defined around them (i.e., in the 3D view) for scanning. The same method can also be applied when loading a BIM or CAD model.
[0067] Figure 4 Shows the start of the exploration and scanning process of the UAV 1. For example, once the volume has been defined, the user presses the "Start Exploration" button on the mobile device, and the UAV 1 will autonomously explore the volume content and scan the objects inside it in an optimal manner without any user interaction.
[0068] By using its laser scanner unit, the UAV 1 can sense its environment in a spherical volume around it. Thus, it can build a map 19 of its own environment. The volume (“exploration map”) 3 defined by the user is automatically divided into a plurality of blocks (“exploration blocks”). In the example shown, the UAV 1 is outside the volume 3 at the start of the process. From Figure 4 As can be seen, the features within the field of view 15 of the UAV's laser scanner unit have been captured as a map 19 of the environment. The UAV 1 determines its actual position - for example its absolute position or its position relative to the volume 3 - and calculates a flight path 13 to approach the volume 3.
[0069] Initially, the volume 3 is unknown space to the UAV 1. Once the field of view 15 of the UAV's laser scanner enters the volume, the unknown space is set to free or occupied, depending on whether the laser scanner detects an object within the corresponding exploration block. In this context, the term “field of view” refers to the 3D region visible by the laser scanner from a specific position and orientation at a given point in time. During flight, the exploration map is continuously updated. The “exploration path planner” uses the continuously updated exploration map Figure 3 to decide the best next flight destination for exploring the volume content and scanning objects of interest.
[0070] Although the embodiments illustrated here show the use of a UAV, other types of mobile robots can also be used with the method according to the present invention. For example, a scanner unit (e.g., including a Leica BLK ARC autonomous laser scanning module) can be provided on a wheeled vehicle (unmanned ground vehicle, UGV) or a walking robot. In particular, if the object of interest is flat or can be accessed by a ramp or stairs, it may not be necessary to use a UAV. If the object of interest is inside a building, a UGV or a walking robot may even be superior to a UAV. To allow collaborative exploration, the exploration map can be hosted on the cloud and updated by the scanner units of multiple UAVs or other mobile robots. Then, multiple UAVs or other mobile robots will periodically retrieve the latest exploration map state and plan their trajectories accordingly.
[0071] Figure 5a and Figure 5b illustrates the process after the UAV 1 has reached the volume 3. For clarity, the exploration map is shown here in a 2D cross-sectional view Figure 3 , where a layer of the exploration block 30 appears as a grid square. The UAV 1 is shown in a top view moving along its path 13 through the exploration map Figure 3 , capturing 3D data of its surrounding environment within the field of view 15 of the laser scanner, and calculating the 3D positions of the objects based on the captured 3D data. Note that for updating the exploration map Figure 3The field of view does not necessarily need to be exactly aligned with the physical field of view of the lidar scanner; the field of view used must be included within the physical field of view.
[0072] Exploration area Figure 3 Overlay on the "global main occupancy map" (i.e., the map established during the flight of UAV 1), and includes 3D information about the detected objects that have been captured by the lidar scanner module of the UAV. This occupancy map can also be used to avoid obstacles. Depending on whether 3D data has been detected within block 30, each exploration block 30 can be marked as "occupied", "free" or "unknown". Initially, all exploration blocks are set to unknown.
[0073] Figure 5b Illustrates updating the exploration map using information collected at a previous time point. UAV 1 continuously moves through volume 3, i.e., without stopping to capture 3D data. The lidar scanner performs approximately 240,000 measurements per second, so it takes a few milliseconds to cover the entire spherical field of view 15. Additionally, it also takes a few milliseconds to process the scan data and combine it into the global occupancy map.
[0074] Internally, the measurements are forwarded to the occupancy map framework. In the case of valid lidar measurements, the corresponding voxels of the occupancy map are set to occupied, thus constructing the global occupancy map. This occupancy map is consistent, i.e., it keeps the latest occupancy information visible.
[0075] Sampling the entire spherical volume of the lidar field of view 15 and the internal processing of the lidar data for updating the occupancy map take a few milliseconds. If the current field of view 15 is used for each exploration map update, the occupancy map in the field of view 15 will generally not be up-to-date. Therefore, the field of view 15' from the past (e.g., a few milliseconds ago) is used to perform the exploration map update. This ensures that the occupancy map is up-to-date in the volume of the past field of view 15'.
[0076] When the occupancy map and the exploration map overlap, the occupancy map information is used to define whether the exploration block is occupied. To ensure that the occupancy map is up-to-date and contains complete occupancy information, the field of view 15' from a few hundred milliseconds ago (past FOV) is used to evaluate and perform the exploration map update. That is, the newly received 3D position may not be the position seen from the pose of the UAV at the actual time point t0, but the position observed at the previous time point t - 1. The time lag between these two time points t0, t - 1 can be selected according to the measurement and calculation speed. The time lag results in a position deviation 14 between the actual position of UAV 1 and its position 1' at the previous time point t - 1.
[0077] Figures 6a to 6c Illustrates updating the exploration areaFigure 3 The process. The exploration map update process described below is performed periodically during the exploration process, for example, once every 500 ms. Alternatively, the exploration map update process can be performed asynchronously (e.g., at a frequency depending on the current flight speed), where if the UAV does not move, the update is not performed, and the faster the UAV moves, the more frequently the update is performed.
[0078] As described above regarding Figure 5a it takes some time (e.g., up to one second) to calculate the latest map (the latest occupancy map) of the environment, during which time UAV 1 does not stop but continues to travel along its path 13 through the Figure 3 terrain. Therefore, the step of updating the exploration Figure 3 map must also be performed based on data from a previous time point (i.e., captured at different positions). This is illustrated in Figure 6a .
[0079] Receive information about the past field of view 15' of the laser scanner at a previous time point (e.g., 500 ms ago). This information can include pose information about the position and orientation and / or range of the past field of view 15'. This ensures that the global main occupancy map of UAV 1 is up-to-date within the field of view volume. Since the laser scanner scans from the center of its field of view 15 in a dome-shaped manner, the individual exploration blocks 30 within the (past) fields of view 15, 15' define cones 41, 42, in particular right circular cones. Each cone is defined by the direction dir of its central axis and the half-aperture angle α.
[0080] Calculating the cones is only an approximation, but it can be used for efficient calculation and allows real-time evaluation, even on platforms with limited computing power. Of course, instead of using cones, volumes aligned with rays to the corner points of the blocks can also be defined. Although more accurate, this solution will require more computational effort.
[0081] All those exploration blocks 30 that overlap with the occupancy blocks of the global occupancy map (main map) and are fully or partially within the field of view 15' are marked as occupancy blocks 32. In addition to other information, each exploration block 30 can store the distance and direction to the nearest occupancy block 32 in its vicinity.
[0082] As Figure 6bAs illustrated, during each exploration map update process, for each exploration block 30, the stored distance and direction to the nearest occupied block 32 can be updated. Alternatively, whenever a block is marked as occupied in a particular surrounding environment, the stored distance and direction can be updated. Additionally, during each update process, the stored distance and direction of already existing occupied blocks 32 do not need to be updated; only the newly marked blocks need to be considered to update their surrounding environment. If a block has been previously set as occupied, there is no need to update the minimum distance and direction of the surrounding exploration blocks.
[0083] For all exploration blocks 322 within a defined radius 320 around one or more occupied blocks 32, the distance 325 to these one or more occupied blocks 32 is determined. If the distance 325 so determined is less than the minimum distance stored in the exploration block, the minimum distance and direction are updated. Subsequently, each exploration block near a detected object will store the approximate distance and direction to the nearest detected object. This information is later used to find the best path for object scanning and to orient the drone towards the object to achieve a high scanning density. Optionally, multiple directions and distances (nearest, next nearest, etc.) or other information about the occupied blocks in the neighboring area (e.g., roughness of the object, surface details, etc.) can be stored to further optimize path planning and scanning, for example, by adjusting the flight speed and orientation.
[0084] As Figure 6c illustrated, for each occupied exploration block that is fully or partially within the past field of view 15’, the corresponding cone information is extracted and stored. For clarity, Figure 6c not shown in Figure 3 . This cone information includes the half-aperture angle α occ of the cone and the direction dir occ from the center of the field of view 15’ to the center of the occupied block 32, occ as well as the corresponding distance d. occ Next, for all unknown exploration blocks that are fully within the past field of view 15’, the corresponding cone information (including the half-aperture angle α, direction dir, and distance d) is extracted. Based on the cone information, it is determined whether the corresponding block is fully or partially hidden by any previously stored occupied exploration block 32. If, for any occupied block 32, d > d occ and δ < (α + α occ ), then the exploration block is considered “hidden” or “occluded”, where δ is the angle between the cone directions dir and dir occ . In other words, if the corresponding unknown block is further away from the laser scanner than the occupied block and if their cones overlap, it is hidden behind the occupied block 32. Then only the non-hidden exploration blocks are marked as free blocks 31, while the fully or partially hidden exploration blocks remain unknown blocks 33.
[0085] Figure 7 An exploration planning process that directly follows the above exploration map update process is illustrated. Figure 7 The UAV 1 is shown, and the field of view 15 of the UAV 1 is within the exploration map (volume). For clarity, the free blocks (not including any scanned features) are not shown here. At the bottom, the laser scanner has detected features and captured these as a map 19 of the environment ("global main occupancy map"). Within the exploration map, these features are not visible because they are covered by exploration blocks that have been marked as occupied blocks 32 (occupied because they include these detected features). The unknown blocks 33 are either outside the field of view 15 or hidden behind the occupied blocks 32.
[0086] For all free blocks 31, it is determined whether they are adjacent to one or more unknown blocks 32. If so, the free blocks 31 are marked as "boundary" blocks 34 - 36. Each boundary block 34 - 36 is free and at the boundary of the unknown space. The indices of each boundary block 34 - 36 are stored, for example, in the current boundary exploration block index array.
[0087] Since the boundary blocks 34 - 36 are free (and thus reachable) and adjacent to the unknown space, they are used as possible flight targets for the UAV1. The main task of the exploration planner is to select the best boundary block 36 as the next flight target for the UAV 1. The selection can be made by assigning costs / scores to each of the boundary blocks 34 - 36 based on, for example, the distance to the current UAV position, the distance to the object, etc.
[0088] Optionally (as described in further detail below regarding Figures 11 to 14c the list of possible boundary blocks 34 - 36 can also be extended with free exploration blocks that do not necessarily touch the unknown space but are in a certain relative position to the occupied exploration blocks 32. This improves path planning and allows for a more uniform scan resolution of the object. Optionally, additional metrics can be employed to select blocks as flight targets. For example, only blocks that provide a good view on the surface of the object within a certain distance can be considered as possible flight targets.
[0089] The update of the exploration map is triggered as described above. For example, the exploration planner may trigger the update. Alternatively, the exploration map can also perform its update independently, and the planner only needs to extract a snapshot of the map whenever needed. For example, the exploration map can be hosted in the cloud, and multiple sensors / robots can update it in parallel. Then, each planner of the robots will only need to extract the current state of the exploration map from the cloud for their respective path planning.
[0090] Each time an update to the exploration map is triggered, the exploration planner runs the following process. For example, the exploration map update can trigger this process in a continuous, regular loop (e.g., every 500 ms).
[0091] First, it is determined whether UAV 1 has reached its previous exploration planner goal, i.e., has completed (or is about to complete) its flight to the center of the previously given boundary blocks 34 - 36. If this is the case, the current boundary blocks 34 - 36 are retrieved from the updated exploration map and graded based on different weighting metrics. For example, these metrics can include the current distance between the respective boundary blocks 34 - 36 and UAV 1 and / or the next occupied block 32. To obtain a smooth flight trajectory, the next target to fly to can be slightly calculated before reaching its goal to avoid stopping.
[0092] Next, the highest - graded boundary block 36 is extracted from all the graded boundary blocks 34 - 36, i.e., the boundary block with the highest total score. The task of flying to the center point of the highest - graded boundary block 36 is sent to the global path planner engine of UAV 1. The global path planner engine plans path 13 to move UAV 1 to the highest - graded boundary block 36 while avoiding obstacles on the way. Preferably, a smooth and continuous scan trajectory should be calculated to avoid stopping during flight. The sent task can also advantageously define the desired target orientation of UAV 1 at the target location. Since the direction to the nearest occupied block 32 has been stored in the block during the exploration map update, the desired target orientation can be set to the direction to the nearest occupied block 32.
[0093] In addition, the relative vertical distance between the respective boundary blocks 34 - 36 and UAV 1 can affect the score, because a shorter vertical distance between the respective boundary blocks 34 - 36 and UAV 1 usually results in more horizontal exploration before vertical exploration. The vertical height level 18 of UAV 1 is shown in Figure 7 The grading of boundary blocks 35, 36 located at the same level 18 can be higher than that of boundary blocks above or below this level 18 to save the battery power of UAV 1.
[0094] Optionally, the exploration Figure 3 and / or planning can be hosted in the cloud and updated by multiple autonomous UAVs and other mobile robots. In addition, fixed laser scanners can be used in addition to mobile robots. This solution allows for autonomous collaborative exploration - based volume scanning.
[0095] Similarly, the interior of an indoor environment such as a building, a manufacturing hall, a cave system, etc. can be explored. For example, a UAV can be placed in a cave system, and a large exploration volume can be defined around the cave system. When exploration is performed, the UAV 1 will then attempt to explore the unknown spaces within the cave system while scanning the cave system. The volume outside the cave will remain unknown as it will not be reachable and observable by the UAV 1 within the cave. The volume outside the cave (e.g., on the ground or in the mountains) is unknown, and the UAV does not attempt to observe it because there are no boundary blocks (idle blocks adjacent to the unknown space) outside the cave in the mountains / ground. The unknown space outside the cave is only adjacent to occupied blocks.
[0096] Optionally, the autonomous exploration process can be paused so that the user can manually control the UAV, e.g., via joystick control, and explore the scene. Since the exploration map is continuously updated (e.g., also during the paused state), the autonomous mode can be re-entered at any time, taking into account the current state of the exploration map. The UAV 1 can also land during exploration, e.g., to replace the battery, and then resume exploration.
[0097] Figure 8 FIG. 1 is a flow chart illustrating an exemplary method 100 for autonomously exploring one or more objects of interest by an unmanned aerial vehicle (UAV) according to the present invention. As described above, the UAV includes a computing unit and a laser scanner module for scanning the surface of one or more objects of interest. The field of view of the laser scanner may be limited by the maximum scanning range and the position and orientation of the laser scanner module (e.g., the rotor). Generally, the maximum scanning range is between 15 meters and 150 meters, particularly between 40 meters and 80 meters.
[0098] In a first step, a three-dimensional exploration map, i.e., a volume, is defined 110 by the user around one or more objects of interest. Then, the user-defined exploration map is partitioned 120 into a plurality of three-dimensional exploration blocks. For example, the size of the exploration blocks can be selected based on the size of the object of interest and the size of its smallest exploitable features. The size of the exploration blocks is generally a trade-off between computational efficiency and the completeness of the scans obtained: the smaller the exploration blocks, the more complete the scans. For example, if each exploration block will have an edge length of 3 meters, this will be too large for exploration within a small building or a cave because essentially all reachable exploration blocks will be occupied blocks. In the case of an opening such as a hole in a wall, the size of the exploration blocks must be chosen small enough so that the exploration of the unknown blocks behind the opening can be ensured. For example, if a circular hole has a diameter of 1 meter, the block size should be chosen to be less than 0.447 meters.
[0099] Optionally, the user may be allowed to change or update the volume selection during an ongoing exploration or during a paused exploration. This may include moving the volume selection, e.g., vertically up or down. This may also include expanding or reducing the volume, e.g., if the user realizes that additional volume is needed to scan an important part of the object that would otherwise not be included. Existing exploration blocks that remain within the newly created volume retain their actual state prior to the update, e.g., idle, occupied, unknown, etc. Parts of the volume that were not covered by the previous volume are filled with exploration blocks having an unknown state.
[0100] The UAV then performs a fully autonomous exploration 130 of the defined and partitioned exploration map. This exploration 130 includes the UAV's laser scanner module generating scan data of objects of interest (and other objects in the user-defined volume) as the UAV flies along the exploration path through the volume until all exploration blocks of all volumes have been explored or determined to be unreachable. In particular, if there are no more blocks in the list of possible flight targets, i.e., no remaining boundary blocks and all other alternative flight targets have been resolved, the exploration is complete.
[0101] Exploring an exploration block at least includes determining whether the corresponding exploration block includes a surface based on the scan data.
[0102] While the exploration 130 is in progress, the UAV's computing unit continuously updates 140 the exploration map and continuously (re)defines 150 the exploration path.
[0103] Figure 9 The flowchart of illustrates Figure 8 An exemplary embodiment of the map update process 140 of the method of. The update process 140 is repeated at multiple time points, e.g., every 500 ms.
[0104] In the exemplary embodiment shown here, it first retrieves 141 information about the field of view of the laser scanner of the past UAV (FOV information). For example, a part of this information can be provided as a predefined value set in the memory of the UAV's computing unit, or retrieved as actual data during a test run before the actual scanning process (e.g., the actual volume definition of the FOV can be predefined via parameters and stored in the memory). The FOV defined as a spherical volume is constant. However, as the laser scanner moves, the FOV has different poses (positions and orientations) over time. The algorithm buffers these poses of the laser scanner within the last x seconds. This allows access to the poses x seconds before the current moment. The FOV information can include information about the maximum scanning range. However, the field of view used in the algorithm can have a smaller radius than the true field of view, i.e., a smaller radius than the maximum scanning range (the field of view used must be included within the physical field of view). Optionally, information about the occlusion of the field of view can also be part of the FOV information, i.e., about those parts of the UAV within the field of view that prevent the scanner from providing a full-dome scan.
[0105] Next, it retrieves 142 information from the occupancy map. The volume of the field of view a few milliseconds ago (past FOV) is known. For each exploration block that is fully or partially within this past FOV, it is determined whether the occupancy map holds an object (i.e., an occupied block) within the volume of the corresponding exploration block. If so, the corresponding exploration block is marked 143 as an occupied block.
[0106] For each block that has been marked as occupied and is within the field of view, it extracts 144 the corresponding cone information. Similarly, for each block that is still unknown (and fully within the past field of view), i.e., each block that has not been marked as free or occupied, it extracts 144 the corresponding cone information. The cone information relates to a cone defined by the position of the UAV's laser scanner module at the corresponding time point and the boundaries of the corresponding block. In particular, the cone information includes at least the direction and the aperture (or semi-aperture) angle.
[0107] Then, based on the extracted cone information (i.e., the cone information of the corresponding unexplored blocks and the cone information of all occupied blocks that are fully or partially within the field of view), it is determined 146 for each unexplored block within the field of view (i.e., fully within the field of view) whether the corresponding unexplored block is at least partially hidden by one or more occupied blocks. If the unknown block is not hidden, it is marked 147 as a free block, otherwise it remains an unknown block for the time being. After these two steps 146, 147 have been performed for each unexplored block within the field of view (i.e., once it is clear which blocks are occupied, free, or unexplored), the free blocks adjacent to the unknown blocks are also marked 148 as boundary blocks.
[0108] Each exploration block can store its (minimum) distance and direction to the nearest occupied block. These values can be updated 149 during each update process 140. However, only the values of those exploration blocks that are close to the most recently set occupied exploration block need to be updated. Generally, it is not necessary to always update the surroundings around all currently occupied exploration blocks. Specifically, for each exploration block within a specific radius around one or more occupied blocks, the stored distance and direction to the nearest occupied block will be updated 149. This includes, for all exploration blocks within a defined radius around one or more occupied blocks, determining the distance to these one or more occupied blocks (see Figure 6b ). If the distance thus determined is less than the minimum distance stored in the exploration block, the minimum distance and the new direction are updated. The radius can be defined, for example, based on the maximum or optimal scanning range of the laser scanner module and / or based on the size of the exploration block. In short: Whenever an exploration block is set to occupied (and was not occupied before), the exploration blocks around that occupied block are updated. If the distance to the current occupied block is less than the minimum distance already stored, it is updated.
[0109] Moreover, the radius can be defined as a part of the maximum scanning range. For example, the radius can be defined as approximately four times the length of the edge of the exploration block (e.g., between three and five times) (i.e., where a is the edge length of the block). Preferably, the radius should at least include the optimal scanning distance of the object. This ensures that the values of all exploration blocks are updated, which serve as possible flight targets for the optimal scanning results. For example, if the optimal scanning distance of the object is 4m and the maximum distance between the centers of adjacent exploration blocks is 2.5m, the radius should be at least 5m, preferably even larger.
[0110] Figure 10 The flowchart of Figure 8 illustrates an exemplary implementation of the path definition process 150 of the method of
[0111] This process can start after each map update process 140, checking 151 whether there are previously assigned exploration targets, i.e., boundary blocks, that the UAV is still traveling to but has not reached. If the UAV has not reached the target, the path definition process 150 is aborted and the next map update process 140 is started.
[0112] If the UAV has (almost) reached the target (or if there is no such target), the path definition process 150 continues by assigning 152 scores to the boundary blocks. The boundary block that has been assigned the highest score is defined 153 as the next exploration target for the UAV, i.e., the UAV is next sent to fly to that boundary block. Optionally, other free blocks that serve as good scanning positions can also be used as the next exploration targets.
[0113] The next exploration target optionally further includes defining the orientation of the UAV at the target location. For example, the orientation can be set to be consistent with the direction to the nearest occupied exploration block, which has been stored in the block during the exploration map update. Alternatively, the UAV can be oriented to provide the best possible view of the object to the mounted scanner, i.e., such that the side of the UAV including the scanning module faces the object to be scanned.
[0114] For example, scores 152 can be assigned based on the following:
[0115] a) The current distance between the respective boundary block and the UAV;
[0116] b) The distance between the respective boundary block and the nearest occupied block; and / or
[0117] c) The relative vertical distance between the respective boundary block and the UAV.
[0118] A shorter distance from the UAV to the next target generally results in more efficient exploration and an overall shorter flight path. Regarding the distance between the respective boundary block and the nearest occupied block, the highest score is assigned to the distance closest to the optimal scanning distance of the laser scanner. Depending on the selected block size, this generally means that the shorter the respective distance, the higher the assigned score. Only when the distance is below the optimal scanning distance can a higher score be assigned to a longer distance. A shorter distance to the next occupied block generally results in a flight path closer to the scannable object and thus takes precedence over exploring unknown space far from known objects. The distance to the nearest exploration block has been stored in the block during the exploration map update and can now be used to assign scores 152.
[0119] A shorter vertical distance between the respective boundary block and the UAV (i.e., boundary blocks at the same altitude level have a higher score than those above or below that level) generally results in more horizontal exploration before vertical exploration. This prevents random up-and-down flights, thus saving battery power and extending the maximum flight time of the UAV.
[0120] Optionally, as illustrated with respect to Figures 11 to 14a to Figure 14c the method for autonomous exploration can be further improved to ensure optimized scanning of the surface of one or more objects of interest. Although the above method ensures complete exploration of the volume, certain surfaces of the objects in the volume may not be scanned optimally, i.e., because they have been scanned at a very flat angle of incidence and / or from a distance that is not the optimal scanning distance.
[0121] Figure 11 and Figure 12 show two examples illustrating these limitations. In Figure 11In this case, the UAV is located at position P 0 at the edge of the L-shaped object 40. The planner selects the boundary block 36 as the next flight target, and the UAV flies there. When the UAV enters the boundary block 36, the three previously unknown exploration blocks 31', 31'', 31''' (assuming they are within the LiDAR FOV) above the boundary block 36 have been set to free. However, when the laser beam hits the surface 41 at a relatively large incident angle, the position at the center of the boundary block 36 does not provide a good scanning position for the surface 41 of the object 40. Assuming a constant angle, e.g., 0.5 degrees (sampling resolution), between consecutive measurements of the LiDAR, the scanning resolution of the surface is low. The position at the boundary block 36 can also be relatively far from the part of the surface 41. Therefore, selecting the nearest boundary block as the next flight target does not always result in a close flyover of the corresponding surface, leading to a low scanning resolution of that surface. In Figure 12 this case, the UAV 1 flies to the center of the boundary block 36 that has been selected as the next flight target. When flying there, the UAV 1 passes over a depression of the object 40. At position P 0 all the previously unknown exploration blocks 31', 31'', 31''' in the depression will be set to free, and the previously unknown block 32 will be set to occupied (assuming the block 32 is still within the field of view). In this case, the surface 41 in the occupied block 32 will not be scanned in an optimal way because it is not at the optimal scanning distance.
[0122] Figure 11 and Figure 12 The problems illustrated in can be solved by adding a list of scanning target exploration blocks to the list of boundary exploration blocks. This is illustrated in Figure 13 The list of scanning target exploration blocks includes free blocks and provides good and occlusion-free scanning positions for scanning the object. In each list, the blocks receive a score depending on how well they are suitable as the next flight target for the UAV. The planner selects the block with the highest score from the two lists, i.e., the block that is considered the best next exploration block to fly to.
[0123] Optionally, when selecting the next exploration block to fly to from the two lists, the scores in the respective lists can be multiplied by weights assigned to the corresponding lists (here: "weight A" and "weight B"). By setting different values for these weights, the behavior can be adjusted to focus more on exploration or more on scanning quality. For example, the behavior of the basic method can be restored by setting "weight B" to zero.
[0124] Figures 14a to 14cIllustrates the use of surface scan cone objects to define scan target blocks. Whenever an exploration block is set to occupied in an exploration map update, a set of surface scan cones 50 is defined for that occupied block 32. The surface scan cones 50 can be placed in a variety of ways. For example, the cones can be placed according to one or more surface normals of an object detected within the occupied exploration block 32. This method requires continuously observing the actual structure within the occupied exploration block 32 and extracting the surface normals that have not been optimally scanned, as the detected structure may not be complete from the start.
[0125] Alternatively, as Figure 14a shown, the surface scan cones 50 can be placed without considering the actual structure within the occupied block 32. Once an exploration block is set to occupied, a fixed set of surface scan cones 50 with defined directions can be created for the occupied exploration block. The defined directions are always the same for all occupied exploration blocks. In the case of a cubic exploration block, as Figure 14a shown, the set of surface scan cones typically consists of six cones 50, although only one cone is shown here for clarity. As shown, for each occupied exploration block 32, six surface scan cones 50 can be defined, which are aligned with the exploration block grid axes, i.e., four horizontally (90 degrees apart), one upward, and one downward. Of course, more surface scan cones 50 (e.g., every 45 degrees) can be added to further increase the scan results (or the possible quality of the scan results). Each surface scan cone 50 is defined by a normal vector 51, a maximum incident angle α (alpha), and a maximum scan distance s (s and α are parameters, where s must be less than the radius of the field of view of the laser scanner).
[0126] Figure 14b Illustrates the marking of the surface scan cones 50 that have been scanned. Each surface scan cone 50 includes a flag indicating whether the scan condition has been met. Initially, the scan condition flag is set to false, i.e., "not yet scanned". If the center 55 of the field of view 15 of the laser scanner enters the surface scan cone 50 and has an unobstructed view of the relevant occupied exploration block 32, the surface scan cone 50 is marked as scanned. In each exploration map update, the scan condition is checked for all non-scanned surface scan cones 50 of the occupied exploration blocks 32 that are at least partially within the current field of view 15 of the scanner module of the UAV. Also as Figure 14b shown, an empty block whose center point lies within the surface scan cone 50 and provides an unobstructed view of the relevant occupied exploration block 32 is defined as a "scan target exploration block" (or "scan target block") 37. At the end of each exploration map update (or when the planner otherwise requires), a list of possible scan target exploration blocks 37 for all non-scanned surface scan cones 50 within the exploration map is collected.
[0127] Figure 14c Illustrates in more detail the definition of the scan target block 37. In this example, the second occupied block 32' is located within the surface scan cone 50 of the occupied block 32. The scan target exploration blocks 37 are the nine blocks marked with a black center. For the particular surface scan cone 50 shown here, these three blocks with a cross center are not defined as scan target exploration blocks. Although their centers are located within the surface scan cone 50, they do not necessarily provide an unobstructed view of the relevant occupied exploration block 32 because the second occupied exploration block 32' at least partially blocks the view.
[0128] In each of the scan target exploration blocks 37, the distance and direction to a specific occupied exploration block 32 (which can also be multiple occupied exploration blocks) are stored. Additionally, a counter is stored and incremented to count the number of surface scan cones 50 in which the specific scan target exploration block is located. Alternatively, a vector / list of the relevant occupied exploration block indices can be stored in the scan target exploration block 37. Based on this index, the distance and direction can be calculated. Based on the length of the vector / list of indices, the number of surface scan cones in which the specific scan target exploration block 37 is located can be calculated.
[0129] Since the surface scan cone 50 is geometrically the same for each occupied block 32, the relative indices of the exploration blocks within the surface scan cone 50 can be predefined (i.e., it is known which exploration block centers are within the cone), which improves the algorithm efficiency. When a list of possible scan target blocks 37 is collected, this iterates over all the as-yet-unscanned surface scan cones and the relevant predefined exploration block indices, checking whether they are free and provide an unobstructed view towards the relevant occupied block 32. Those blocks that meet the criteria are placed on the list.
[0130] Each of the scan target exploration blocks 37 can be scored similarly to the boundary blocks. However, in addition, they are scored with the stored distance to the occupied exploration block, with the best scan distance having the highest score. Additionally, if a scan target exploration block 37 is located in multiple surface scan cones 50, the higher the score of the scan target exploration block 37, i.e., the more cones it covers, the higher the assigned score. The above counter value can be used for this score. The UAV flying to the scan target blocks 37 included in multiple surface scan cones 50 marks them all as scanned simultaneously, thus improving the efficiency of exploration.
[0131] Although the present invention has been illustrated above in part with reference to some preferred embodiments, it must be understood that various modifications and combinations of different features of the embodiments can be made. All such modifications are within the scope of the appended claims.
Claims
1. A computer-implemented method (100) for autonomously exploring one or more objects of interest by a mobile robot, the mobile robot comprising a computing unit and a laser scanner module (10) for scanning a surface of the one or more objects of interest, the laser scanner module (10) having a field of view (15), It is characterized in that - defining (110) a three-dimensional exploration map (3), wherein the one or more objects of interest are located in the exploration map (3); - dividing (120) the exploration map (3) into a plurality of three-dimensional exploration blocks (30); and - autonomously exploring (130) the exploration map (3) by the mobile robot, wherein exploring (130) the exploration map (3) comprises the mobile robot exploring at least a subset of the exploration blocks along an exploration path (13), and the laser scanner module (10) generates scanning data when the mobile robot moves along the exploration path (13), wherein: - exploring the exploration block (30) comprises at least determining whether the respective exploration block comprises one or more points of the point cloud; and - The computing unit of the mobile robot updates (140) the exploration map (3) and defines (150) the exploration path (13).
2. The method (100) according to claim 1, wherein: The mobile robot is a UAV (1), in particular a quadcopter drone.
3. The method (100) according to claim 2, wherein: The field of view (15) depends on the position and orientation of the UAV (1) and is limited by: - the position and orientation of the laser scanner module (10) relative to the UAV (1), and - a predefined radius, the predefined radius being smaller than the maximum scanning range of the laser scanner module (10), In particular, - the field of view (15) is also limited by the position and orientation of the laser scanner module (10) relative to other features of the UAV (1), in particular wherein the other features include at least the rotor (11) and / or the wing of the UAV (1); and / or - the maximum scanning range of the laser scanner module (10) is between 15 meters and 150 meters, in particular between 40 meters and 80 meters.
4. The method (100) according to any one of the preceding claims, wherein: Updating (140) the exploration map (3) is performed at a time interval selected according to at least one of the following: - the scanning speed of the laser scanner unit (10), - the calculation speed of the calculation unit, and - the current speed of the mobile robot, In particular, The time interval is between 100 ms and 1 s.
5. The method (100) according to any one of the preceding claims, wherein: Updating (140) the exploration map (3) includes at each of a plurality of different time points: - retrieving (142) scan data generated by the laser scanner module (10); - based on the retrieved scan data, identifying an exploration block (30) at least partially located in a past field of view (15') and comprising one or more points of the point cloud, wherein the past field of view (15') is the field of view (15) of the laser scanner module at a previous time point (t-1) before a corresponding time point (t0) among the plurality of different time points; - defining (143) the identified explored block as an occupied block (32); - extracting (144) cone information for each occupied block (32) at least partially located in the past field of view (15'), the cone information being related to a cone (42) defined by the position of the laser scanner module (10) at the respective previous point in time (t-1) and the boundaries of the respective occupied block (32); - extracting (145) cone information of each unexplored block (33) completely located in the past field of view (15'), the cone information being related to the cone (41) defined by the position of the laser scanner module (10) at the respective previous point in time (t-1) and the boundary of the respective unexplored block (33); and - for each unexplored block (33) completely located in the past field of view (15'), determining (146) whether a line of sight between a center of the past field of view (15') and at least a portion of the corresponding unexplored block (33) is blocked by one or more occupied blocks (32) based on the cone information of the corresponding unexplored block (33) and the cone information of the occupied block (32) at least partially located in the past field of view (15'), In particular, determining (146) whether the line of sight is blocked includes determining whether the cone (41) of the corresponding unexplored block (33) overlaps with one or more cones (42) of the occupied block (32), and if it overlaps, determining whether the distance (d) from the center of the past field of view (15') to the center of the corresponding unexplored block (33) is greater than the distance (d) from the center of the past field of view (15') to the center of the corresponding occupied block (32). occ ).
6. The method (100) according to claim 5, wherein: Storing information about the minimum distance and direction to the nearest occupied block (32) or the nearest point of the point cloud for each exploration block (30), and updating (140) the exploration map (3) further comprises: - for each block that has been defined (143) as an occupied block (32) at a different recent point in time, determining the distance (325) and direction between the respective occupied block (32) and a respective unoccupied exploration block (322) within a defined radius (320) around the respective occupied block (32); and - if the determined distance (325) is less than the minimum distance stored for the block, updating (149) the stored minimum distance and direction with the determined distance (325) and direction, In particular, the radius (320) is defined based on a maximum or optimal scanning range of the laser scanner module (10) and / or based on the sizes of the plurality of three-dimensional exploration blocks (30).
7. The method (100) according to claim 5 or claim 6, wherein: Updating (140) the exploration map (3) further comprises: - defining (147) as a free block (31) an unexplored block (33) which is completely located in the past field of view (15'), the cone (41) of which does not overlap with one or more cones (42) of the occupied block (32); - defining (148) the free block (31) adjacent to the unexplored block (33) as a boundary block (34-36), The exploration path (13) defined in (150) includes: - assigning (152) scores to said boundary blocks (34-36); and - assigning (153) a next exploration target to the mobile robot based on the scores assigned to the boundary blocks (34-36), in particular wherein the boundary block (36) with the highest score is assigned as the next exploration target, In particular, the score is assigned (152) based on: - the distance between the corresponding boundary block (34-36) and one or more occupied blocks (32); and / or - the total distance and / or the relative vertical distance between the corresponding boundary block (34-36) and the current position of the UAV (1); and / or - the number of occupied tiles (32) within a defined radius around the respective boundary tile (34-36).
8. The method (100) according to claim 7, wherein: The exploration path (13) defined (150) includes: - defining a subset of free blocks (31) that are not boundary blocks (34-36) as scan target blocks (37), and - also assigning a score to the scan target block (37), wherein assigning (153) a next exploration target to the mobile robot is based on the scores assigned to the boundary blocks (34-36) and the scores assigned to the scanning target blocks (37), wherein the block (34-37) with the highest score is assigned as the next exploration target, and in particular wherein the score is assigned to the scanning target block (37) based on the position of the scanning target block (37) relative to the surface of the object of interest and its likelihood of providing a good scanning result.
9. The method (100) according to claim 8, wherein: Allocating scores includes allocating boundary scores to the boundary blocks (34-36) and allocating scan target scores to the scan target blocks (37), wherein a first weight is multiplied by each of the boundary scores and a second weight is multiplied by each of the scan target scores, and in particular wherein the first weight and the second weight are user selectable.
10. The method (100) according to claim 8 or claim 9, wherein: Defining the scanning target block (37) includes: - defining a plurality of surface scanning cones (50) for each occupied block (32), and - for each free block (31) whose centre point is within the surface scanning cone (50), determining whether the line of sight to a portion of the corresponding occupied block (32) is blocked, Wherein, those free blocks (31) whose center points are located in the surface scanning cone (50) and whose line of sight to the corresponding occupied blocks (32) is not blocked are defined as scanning target blocks (37), and in particular, when the center (55) of the field of view (15) enters the corresponding surface scanning cone (50), the surface scanning cone (50) is marked as explored.
11. The method (100) according to any one of the preceding claims, wherein: The three-dimensional volume is defined by the user as the three-dimensional exploration map (3) in the graphical user interface (20), in particular wherein the graphical user interface - displayed on a screen of the mobile computing device (2); and / or - allowing the user to define a polyhedron, in particular a cuboid, as the three-dimensional exploration map (3) by marking two corner points of the cuboid; and / or - A two-dimensional representation of the one or more objects of interest is shown.
12. The method (100) according to any one of the preceding claims, wherein: Exploring the exploration volume (30) comprises scanning, by the laser scanner module (10), surfaces of objects present in the exploration volume, in particular with a user-defined scanning accuracy, in particular wherein the one or more objects of interest comprise: - a building or other man-made structure, and the surface comprises at least one of a facade, a roof, a support and a pavement; and / or -Natural objects or scenes, especially caves.
13. A UAV (1), comprising a computing unit and a laser scanner module (10), wherein the laser scanner module (10) has a field of view (15), It is characterized in that The computing unit is configured to: a) receiving a three-dimensional exploration map (3) divided (120) into a plurality of three-dimensional exploration blocks (30), or b) receiving a three-dimensional exploration map (3) and dividing (120) the exploration map (3) into a plurality of three-dimensional exploration blocks (30), or c) receiving exploration map information, the exploration map information particularly comprising coordinates, to generate a three-dimensional exploration map (3) based on the exploration map information, and to divide (120) the exploration map (3) into a plurality of three-dimensional exploration blocks (30), wherein one or more objects of interest are located in the exploration map (3), and wherein the computing unit is configured to control autonomous exploration (130) of the exploration map (3) by the UAV (1), wherein exploring (130) the exploration map (3) comprises the UAV (1) exploring at least a subset of the exploration blocks along an exploration path (13), and wherein the laser scanner module (10) generates scan data when the UAV (1) travels along the exploration path (13), the scan data relating to a point cloud, wherein: - exploring the exploration block at least comprises determining, based on the scan data, whether the corresponding exploration block includes one or more points of the point cloud; and - The computing unit of the UAV (1) is configured to update (140) the exploration map (3) and define (150) the exploration path (13).
14. A system for scanning a surface of one or more objects of interest, the system comprising a UAV (1) according to claim 13 and a mobile computing device (2), wherein: The mobile computing device (2) is configured to receive user input defining (110) the three-dimensional exploration map (3), wherein the one or more objects of interest are located in the exploration map (3), and to provide the exploration map (3) or exploration map information to the UAV (1), the exploration map information particularly comprising coordinates.
15. A computer program product comprising a program code having computer executable instructions for executing the method (100) according to any one of claims 1 to 12, in particular for executing the method (100) according to any one of claims 1 to 12 when running in a computing unit (10) of a UAV (1) according to claim 13.
Citation Information
Patent Citations
Unmanned aerial vehicle
WO2022268316A1