SURVEYING A GEOGRAPHICAL AREA USING MULTIPLE AUTONOMOUS ROBOTS

Abstract

A procedure includes receiving a request to generate a map of a geographic area, obtaining parameters of a geographic area representing properties of the area, subdividing the area into a plurality of sub-areas based on the parameters of a geographic area, and obtaining corresponding robot parameters representing properties of the robot for each of a plurality of robots.Furthermore, the procedure includes, for each specific sub-area, determining corresponding sub-area parameters representing the properties of that sub-area based on the parameters of a geographic area; assigning a specific robot to survey the specific sub-area based on the sub-area parameters and the robot parameters; and obtaining corresponding survey data acquired by the specific robot, where the survey data represents a survey of the specific sub-area by the specific robot. The procedure also includes combining the corresponding survey data acquired by the assigned robots to generate a highly realistic map of the geographic area.

Description

INTRODUCTION

[0001] The information given in this section serves to provide a general overview of the context of the disclosure. The work of the inventors mentioned herein, to the extent described in this section, as well as aspects of the description that cannot otherwise be considered prior art at the time of filing, are neither explicitly nor implicitly recognized as prior art with respect to the present disclosure.

[0002] A map of a geographical area can be created by surveying that area. In some examples, equipment such as a camera is used to survey the area. In other examples, a person surveys the area manually.

[0003] The present disclosure relates generally to the surveying of a geographical area using a plurality of autonomous robots. SUMMARY

[0004] One aspect of the revelation provides a system comprising a plurality of robots configured to survey assigned sub-areas of a geographic territory, and a server communicating with these multiple robots. The system is configured to perform operations that include receiving a request to generate a highly realistic map of the geographic territory, obtaining a plurality of parameters representing the geographic territory's properties, subdividing the geographic territory into a plurality of sub-areas based on these parameters, and obtaining a corresponding plurality of robot parameters representing the specific properties of each robot within the plurality of robots.The operations include, for each specific sub-area of ​​the plurality of sub-areas, determining a corresponding plurality of sub-area parameters representing properties of the specific sub-area, based on the plurality of parameters of a geographic area; assigning a specific robot from the plurality of robots to survey the specific sub-area based on the corresponding plurality of sub-area parameters and the corresponding plurality of robot parameters; and obtaining corresponding survey data acquired by the specific robot, where the corresponding survey data represents a survey of the specific sub-area by the assigned specific robot. Furthermore, the operations include combining the corresponding survey data acquired by the assigned robots to generate the highly realistic map of the geographic area.

[0005] Implementations of the disclosure may include one or more of the following optional features. According to some implementations, each assigned robot is configured to perform operations that include determining a path and / or a procedure for surveying the assigned sub-area based on the appropriate plurality of sub-area parameters for an assigned sub-area and the appropriate plurality of robot parameters, navigating to the assigned sub-area, acquiring the appropriate survey data representing a survey of the designated sub-area based on the path and / or procedure, and providing the appropriate survey data to the server.

[0006] According to some examples, assigning a specific robot to a plurality of robots to survey a specific sub-area, based on the corresponding plurality of sub-area parameters and robot parameters, reduces the amount of energy expended by the assigned robots. Additionally or alternatively, assigning a specific robot to a plurality of robots to survey a specific sub-area, based on the corresponding plurality of sub-area parameters and robot parameters, maximizes coverage of the geographic area. Additionally or alternatively, assigning a specific robot to a plurality of robots to survey a specific sub-area, based on the corresponding plurality of sub-area parameters and robot parameters, reduces surveying time.

[0007] According to some implementations, the corresponding plurality of robot parameters, which represent properties of a particular robot, includes an ability to navigate on the ground and / or an ability to navigate on water and / or an ability to navigate in water and / or an ability to navigate in the air and / or a battery status and / or a charging time and / or a robot type and / or a robot location and / or a coverage area and / or a cost to traverse a distance unit and / or a shape and / or one or more sensor types and / or one or more communication interfaces.According to some examples, the corresponding plurality of sub-area parameters representing properties of a particular sub-area includes the presence of shade and / or the presence of sunshine and / or a topology and / or the presence of water and / or an elevation above sea level and / or one or more geographical hazards and / or a traversable surface and / or a surface type.

[0008] According to some examples, combining the relevant survey data acquired by the assigned robot to generate the highly realistic map of the geographic area involves extracting one or more features and one or more locations corresponding to the one or more features from the relevant survey data for each specific sub-area of ​​the plurality of sub-areas, determining an orientation of the relevant survey data to the relevant survey data acquired by another robot based on the one or more features and the one or more locations, and combining the relevant survey data and the relevant survey data acquired by the other robot using the orientation.Combining the relevant survey data and the corresponding survey data acquired by the other robot, using the orientation, may involve the use of a weighted feature score. One or more features may be weighted using a first realism score assigned to the relevant survey data and a second realism score assigned to the corresponding survey data acquired by the other robot.

[0009] Another aspect of the disclosure provides a computer-implemented procedure, executed by data processing hardware, which causes the data processing hardware to perform operations. These operations include receiving a request to generate a highly realistic map of the geographic area, obtaining a plurality of parameters of a geographic area representing its properties, subdividing the geographic area into a plurality of sub-areas based on these plurality of parameters, and obtaining a corresponding plurality of robot parameters representing the properties of a particular robot, for each particular robot of the plurality of robots.The operations include, for each specific sub-area of ​​the plurality of sub-areas, determining a corresponding plurality of sub-area parameters representing properties of the specific sub-area, based on the plurality of parameters of a geographic area; assigning a specific robot from the plurality of robots to survey the specific sub-area based on the corresponding plurality of sub-area parameters and the corresponding plurality of robot parameters; and obtaining corresponding survey data acquired by the specific robot, where the corresponding survey data represents a survey of the specific sub-area by the assigned specific robot. Furthermore, the operations include combining the corresponding survey data acquired by the assigned robots to generate the highly realistic map of the geographic area.

[0010] Implementations of the disclosure may include one or more of the following optional features. According to some implementations, combining the relevant survey data acquired by the assigned robot to generate the highly realistic map of the geographic area involves extracting one or more features and one or more locations corresponding to the one or more features from the relevant survey data for each specific sub-area of ​​the plurality of sub-areas, determining an orientation of the relevant survey data to the relevant survey data acquired by another robot based on the one or more features and the one or more locations, and combining the relevant survey data and the relevant survey data acquired by the other robot using the orientation.

[0011] According to some examples, combining the relevant survey data and the corresponding survey data acquired by the other robot using orientation involves the use of a weighted feature score. One or more features can be weighted using a first realism score assigned to the relevant survey data and a second realism score assigned to the corresponding survey data acquired by the other robot.According to some implementations, assigning a specific robot to the plurality of robots to survey the specific sub-area based on the corresponding plurality of sub-area parameters and the corresponding plurality of robot parameters involves reducing the amount of energy expended by the assigned robots and / or maximizing coverage of the geographic area and / or reducing survey time.

[0012] Yet another aspect of the revelation provides a system that includes data processing hardware and storage hardware communicating with the data processing hardware. The storage hardware stores instructions which, when executed in the data processing hardware, cause the data processing hardware to perform operations. These operations include receiving a request to generate a highly realistic map of the geographic area, obtaining a plurality of parameters of a geographic area representing its properties, subdividing the geographic area into a plurality of sub-areas based on these plurality of parameters, and obtaining a corresponding plurality of robot parameters representing the properties of a particular robot, for each particular robot among the plurality of robots.The operations include, for each specific sub-area of ​​the plurality of sub-areas, determining a corresponding plurality of sub-area parameters representing properties of the specific sub-area, based on the plurality of parameters of a geographic area; assigning a specific robot from the plurality of robots to survey the specific sub-area based on the corresponding plurality of sub-area parameters and the corresponding plurality of robot parameters; and obtaining corresponding survey data acquired by the specific robot, where the corresponding survey data represents a survey of the specific sub-area by the assigned specific robot. Furthermore, the operations include combining the corresponding survey data acquired by the assigned robots to generate the highly realistic map of the geographic area.

[0013] Implementations of the disclosure may include one or more of the following optional features. According to some implementations, combining the relevant survey data acquired by the assigned robot to generate the highly realistic map of the geographic area involves extracting one or more features and one or more locations corresponding to the one or more features from the relevant survey data for each specific sub-area of ​​the plurality of sub-areas, determining an orientation of the relevant survey data to the relevant survey data acquired by another robot based on the one or more features and the one or more locations, and combining the relevant survey data and the relevant survey data acquired by the other robot using the orientation.

[0014] According to some examples, combining the relevant survey data and the corresponding survey data acquired by the other robot using orientation involves the use of a weighted feature score. One or more features can be weighted using a first realism score assigned to the relevant survey data and a second realism score assigned to the corresponding survey data acquired by the other robot.According to some implementations, assigning a specific robot to the plurality of robots to survey the specific sub-area based on the corresponding plurality of sub-area parameters and the corresponding plurality of robot parameters involves reducing the amount of energy expended by the assigned robots and / or maximizing coverage of the geographic area and / or reducing survey time.

[0015] The details of one or more implementations of the disclosure are set forth in the accompanying drawings and in the following description. Further aspects, features, and advantages will become apparent from the description, the drawings, and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described here serve only to illustrate selected configurations and are not intended to limit the scope of protection of the present disclosure; they show: Fig. 1 a schematic representation of an exemplary surveying system comprising a plurality of autonomous robots, in accordance with the principles of the present disclosure; Fig. 2. A flowchart of an exemplary sequence of operations for a procedure for using a plurality of autonomous robots to survey a geographical area; Fig. 3. A flowchart of an exemplary arrangement of operations for a procedure for dividing a geographical area into sub-areas and assigning autonomous robots to the sub-areas; Fig. 4. A flowchart of an exemplary sequence of operations for a procedure for combining survey data acquired by a plurality of autonomous robots; Fig. 5 a flowchart of another exemplary arrangement of operations for a procedure for combining survey data acquired by a plurality of autonomous robots; Fig. 6. A flowchart of another exemplary arrangement of operations for a procedure using a plurality of autonomous robots to survey a geographical area.

[0017] Corresponding reference symbols in all drawings denote corresponding parts. DETAILED DESCRIPTION

[0018] Exemplary configurations are now described in more detail with reference to the accompanying drawings. Exemplary configurations are given to ensure that this disclosure is thorough and fully conveys the scope of protection of the disclosure to the person skilled in the art. To provide a thorough understanding of the configurations of this disclosure, specific details such as examples of specific components, devices, and processes are set forth. It is clear to the person skilled in the art that specific details need not be used, that exemplary configurations can be embodied in many different forms, and that the specific details and the exemplary configurations are not to be understood as limiting the scope of protection of the disclosure.

[0019] The terminology used here serves only to describe certain exemplary configurations and is not intended to be restrictive. Unless the context clearly indicates otherwise, the singular articles "a," "an," and "that," as used here, are intended to include the plural forms. The terms "includes," "comprehensive," "containing," and "exhibiting" are inclusive and thus specify the presence of features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more further features, steps, operations, elements, components, and / or groups thereof. Unless a specific order of execution is given, the procedural steps, processes, and operations described herein should not be understood as requiring their execution in the particular order discussed or presented.Additional or alternative steps can be used.

[0020] When an element or layer is described as "on," "interacting with," "connected with," "attached to," or "coupled with" another element or layer, it may be directly on, interacting with, connected with, attached to, or coupled with the other element or layer, or there may be intermediate elements or layers. Conversely, no intermediate elements or layers may be present when an element is described as "directly on," "directly interacting with," "directly connected with," "directly attached to," or "directly coupled with" another element or layer. Other words used to describe the relationship between elements (e.g., "between" versus "directly between," "adjacent to" versus "directly adjacent to," etc.) are to be interpreted in the same way.As the term “and / or” is used here, it includes any combination of one or more of the associated listed objects.

[0021] The terms "first," "second," "third," etc., may be used here to describe different elements, components, areas, layers, and / or sections. These elements, components, areas, layers, and / or sections are not intended to be limited by these terms. These terms may only be used to distinguish one element, component, area, layer, or section from another. Unless otherwise clearly indicated by the context, terms such as "first," "second," and other numerical terms do not imply a sequence or order.Thus, a first element, a first component, a first area, a first layer or a first section discussed below could be referred to as a second element, a second component, a second area, a second layer or a second section without deviating from the lessons of the exemplary configurations.

[0022] In this application, including in the following definitions, the term "module" may be replaced by the term "circuit". The term "module" may refer to, be a part of, or include an application-specific integrated circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combination logic circuit; a free programmable logic array (FPGA); a processor (shared, dedicated, or group) that executes code; memory (shared, dedicated, or group) that stores code executed by a processor; other suitable hardware components that provide the described functionality; or a combination of some or all of the above, such as in a system-on-a-chip.

[0023] The term "code," as used above, can include software, firmware, and / or microcode, and can refer to programs, routines, functions, classes, and / or objects. The term "shared processor" includes a single processor that executes some or all of the code from multiple modules. The term "group processor" includes a processor that executes some or all of the code from one or more modules along with additional processors. The term "shared memory" includes a single memory that stores some or all of the code from multiple modules. The term "group memory" includes memory that stores some or all of the code from one or more modules along with additional memory. The term "memory" can be a subset of the term "computer-readable medium."The term "computer-readable medium" excludes transitory electrical and electromagnetic signals propagating through a medium and can therefore be considered a concrete and non-transient storage medium. Non-restrictive examples of non-transient storage include a concrete computer-readable medium that encompasses non-volatile storage, magnetic storage, and optical storage.

[0024] The devices and methods described in this application can be implemented in whole or in part by one or more computer programs executed by one or more processors. The computer programs contain instructions executable by a processor, stored on at least one non-transitory, concrete, computer-readable medium. Furthermore, the computer programs can contain and / or rely on stored data.

[0025] A software application (i.e., a software resource) can refer to computer software that causes a computer device to perform a task. Depending on the context, a software application may be called an "application," an "app," or a "program." Examples of applications include, but are not limited to, system diagnostics applications, system management applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and gaming applications.

[0026] Non-transitory memory can be physical devices used to store programs (e.g., sequences of instructions) or data (e.g., program state information) on a temporary or permanent basis for use by a computer device. Non-transitory memory can be volatile and / or non-volatile addressable semiconductor memory. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (which is commonly used, for example, for firmware such as boot programs).Examples of volatile memory include, but are not limited to, read / write memory (RAM), dynamic read / write memory (DRAM), static read / write memory (SRAM), phase change memory (PCM), disks or tapes.

[0027] These computer programs (also known as programs, software, software applications, or code) contain machine instructions for a programmable processor and can be implemented in a higher-level procedural and / or object-oriented programming language and / or in assembly language / machine language. As used here, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, any non-transitory computer-readable medium, any device, and / or any apparatus (e.g., magnetic disks, optical disks, memory, programmable logic devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including machine-readable medium that receives machine instructions as a machine-readable signal.The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0028] Various implementations of the systems and techniques described herein can be realized in a digital electronic and / or optical circuit arrangement, in an integrated circuit arrangement, in specially designed ASICs (application-specific integrated circuits), in computer hardware, in computer firmware, in computer software, and / or in combinations thereof. These various implementations may include implementations in one or more computer programs that are executable and / or interpretable in a programmable system that contains at least one programmable processor, which may be a special-purpose or general-purpose processor, coupled to a storage system, at least one input device, and at least one output device for receiving data and instructions from and sending data and instructions to a storage system.

[0029] The processes and logic sequences described in this description can be executed by one or more programmable processors, also known as data processing hardware, which run one or more computer programs to perform functions by processing input data and generating output. Alternatively, the processes and logic sequences can be executed by a specialized logic circuit arrangement, such as an FPGA (free programmable logic assembly) or an ASIC (application-specific integrated circuit). Processors suitable for executing a computer program include, for example, general-purpose and specialized microprocessors, as well as any type of digital computer processor(s). Generally, a processor receives instructions and data from read-only memory, read / write memory, or both.The essential elements of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Generally, a computer also includes one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or is functionally coupled to them to receive data from them, send data to them, or both. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.The processor and memory can be supplemented by or integrated into a special logic circuit arrangement.

[0030] To provide interaction with a user, one or more aspects of the disclosure can be implemented in a computer that has a display device, such as a CRT (cathode ray tube) monitor, an LCD (liquid crystal display) monitor, or a touchscreen, for displaying information to the user, and optionally a keyboard and pointing device, such as a mouse or trackball, by which the user can provide input to the computer. Other types of devices can also be used to provide interaction with a user; for example, feedback provided to the user can be any form of sensory feedback, such as...This feedback can be visual, audible, or tactile; and input can be received from the user in any form, including acoustic, speech, or keystroke input. Furthermore, a computer can interact with a user by sending documents to and receiving documents from a device used by the user; for example, by sending web pages to a web browser on a user's client device in response to requests received by the web browser.

[0031] Unless explicitly stated otherwise, the phrase "at least one of A, B, or C" refers to any combination or subset of A, B, or C, such as (1) at least one A alone; (2) at least one B alone; (3) at least one C alone; (4) at least one A with at least one B; (5) at least one A with at least one C; (6) at least one B with at least one C; and (7) at least one A with at least one B and at least one C. Furthermore, unless explicitly stated otherwise, the phrase "at least one of A, B, and C" refers to any combination or subset of A, B, or C, such as (1) at least one A alone; (2) at least one B alone; (3) at least one C alone; (4) at least one A with at least one B; (5) at least one A with at least one C; (6) at least one B with at least one C; and (7) at least one A with at least one B and at least one C.Unless explicitly stated otherwise, “A or B” shall refer to any combination of A and B such as: (1) A alone; (2) B alone; and (3) A and B.

[0032] A map of a geographic area can be created by surveying that area. In some cases, equipment such as a camera, a light detection and distance measurement (LiDAR) system, a depth camera, etc., is used for surveying. In other cases, a person surveys the area manually. However, especially for geographic areas that are difficult to survey, conventional surveying techniques may not be able to adequately measure such areas. Examples of difficult-to-survey geographic areas include a military installation or site; a terrain area; a jungle area; a desert area; an agricultural area; an area with altered roads, traffic patterns, topography, etc., following a natural disaster (e.g., an earthquake or hurricane).; an area following a man-made disaster (e.g., an explosion, a release of radioactive material, a military operation, etc.); an area that is potentially unsafe for people or animals, etc., but not limited to these. It is useful to survey such difficult areas to help expedite the delivery of aid or supplies; to reduce traffic congestion; to save lives by assessing how aid can be delivered; to accelerate rescue efforts; to prioritize road cleaning; to determine routes for traversing a geographic area; etc. Thus, there is a need for improved procedures and systems for surveying geographic areas, especially those that are difficult, unsafe, or impossible to survey using conventional techniques.According to disclosed examples, robots are optimally assigned to specific sub-areas of a geographic region, and these robots then autonomously survey their assigned sub-area(s). The survey data collected by the autonomous robots are then optimally combined to generate a highly realistic map of the geographic region. The autonomous robots can survey the sub-areas before they are deemed safe for human access. In some examples, autonomous robots survey a geographic region in real time to facilitate real-time decision-making regarding that region.

[0033] Fig. Figure 1 is a schematic view of an exemplary surveying system 100, comprising a server 102 and a plurality of autonomous robots 104, 104a-n (also referred to as robot 104), which are assigned to autonomously survey sub-areas 304, 304a-n of a geographical area 302 (see Fig. 3) are configured. As shown, a carrier or astronaut vehicle 106 can be used to transport one or more of the robots 104 closer to the geographic area 302 or assigned sub-areas 304. An autonomous robot 104 can be any type of vehicle (e.g., passenger car or truck), robot, drone, component, etc., capable of autonomous movement, action, or data acquisition. The robots 104 can include different types of robots, so that different types of robots can be used to survey different types of areas.

[0034] Server 102 divides a geographic area 302 to be surveyed into sub-areas 304, assigns one or more robots 104 to survey each sub-area 304, receives survey data 211 from the assigned robots 104, and combines the survey data 211 to generate a highly realistic map of the geographic area 302. Server 102 divides the geographic area 302 based on geographic area parameters that represent properties of the geographic area, so that each sub-area 304 can be surveyed by a specific type of robot 104. Sub-areas 304 can correspond, for example, to areas of water, areas of dry land, areas of wet land, mountainous or hilly areas, jungle areas, shaded areas, sunny areas, etc. Each sub-area 304 has corresponding sub-area parameters 306 that represent properties of the sub-area 304.According to some examples, the subrange parameters 306 are taken from or derived from the parameters of a geographic area. Server 102 assigns each subrange 304 based on subrange parameters 306, which represent properties of subrange 304, and robot parameters 308 (see ). Fig. 3) representing the properties of each robot 104, assign one or more robots 104. According to some examples, the server 102 assigns robots 104 to sub-areas 304 in order to, for example, reduce or minimize the amount of energy expended by the assigned robots 104, increase or maximize the coverage of the geographic area 302, or reduce or minimize surveying time.

[0035] Exemplary parameters of a geographic area and sub-area parameter 306 include, but are not limited to, the presence of shade, the presence of sunshine, a topology, the presence of water, an elevation above sea level, one or more geographic hazards, a traversable surface (e.g., a drivable surface) or a surface type (e.g., vegetation, terrain, gravel road, etc.).

[0036] Exemplary robot parameters 308 include, but are not limited to, an ability to navigate on ground, an ability to navigate on water, an ability to navigate in water, an ability to navigate in air, a battery status, a charging time, a robot type, a robot location, a coverage area, the cost to traverse a unit of distance, a shape, one or more sensor types, or one or more communication interfaces.

[0037] The robot(s) 104, which are assigned to survey a specific sub-area 304, are configured to autonomously determine a path and / or a procedure for surveying the assigned sub-area 304 based on the sub-area parameters 306 for sub-area 304 and their robot parameters 308. The robot(s) 104 then navigate to the assigned sub-area 304 and acquire survey data 211 based on the path and / or the procedure (see Fig. 2), which represent a survey of sub-area 304. According to some examples, the robots 104 navigate and survey their assigned sub-areas 304 autonomously. Alternatively, one or more robots 104 can be remotely controlled by the server 102 and / or by a person. According to some implementations, a robot 104 can also measure the distance it takes to reach an assigned sub-area 304.

[0038] Then the robot(s) 104 provide the survey data 211 to the server 102 and the server 102 combines the survey data 211 to create a very realistic map of the geographical area 302.

[0039] According to some implementations, Server 102 subdivides the geographic area 302 based on an initial map or a less realistic map of the geographic area 302. Furthermore, the process of generating the highly realistic map can be repeated, as in some examples. For instance, the highly realistic map can be fed back into System 100 to generate better sub-areas 304 and / or to create a better deployment strategy for robots 104 within the areas 304. For example, an impassable area might be discovered in a particular sub-area 304, requiring a robot 104 with different capabilities to be redeployed within that same sub-area 304. Thus, the realism of the generated map can increase over time with each deployment by repeating the process.

[0040] Fig. Figure 2 is a flowchart of an exemplary sequence of operations for a computer-implemented procedure 200 for using a plurality of autonomous robots 104 to survey a geographic area 302. The operations can be executed by data processing hardware of the server 102 or the robots 104 based on the execution of instructions stored in memory hardware of the server 102 or of the robots 104. Many other ways can be used to implement the procedure 200. For example, the order in which the operations are executed can be changed, and / or one or more of the operations and / or interactions can be modified, removed, subdivided, or combined. Furthermore, the operations can be derived from Fig. 2. e.g., by separate process streams, processors, devices, discrete logic, circuits, etc., executed sequentially and / or in parallel.

[0041] In operation 202, procedure 200 includes the server 102 performing robot route planning. According to some examples, performing robot route planning involves dividing the geographic area 302 to be surveyed into sub-areas 304, for example, based on a high-level or low-resolution map, assigning one or more of the robots 104 to survey each sub-area 304, and determining a route that each robot 104 is to take to its assigned sub-area or sub-areas 304. As in Fig. As shown in Figure 3, the assignment of one or more robots 104 to measure each sub-area 304 is based on sub-area parameters 306, which represent properties of the sub-area 304, and on robot parameters 308, which represent properties of each robot 104. As in the exemplary method 300 from Fig. As shown in Figure 3, an optimization module 310 can optimize the allocation of robots 104 to sub-areas 304 in order to, for example, reduce or minimize the amount of energy expended by the allocated robots 104, increase or maximize the coverage of the geographic area 302, or reduce or minimize surveying time. More than one robot 104 can be allocated to survey a sub-area 304, a specific robot 104 can be allocated to survey multiple sub-areas 304, and a specific robot 104 cannot be allocated to survey any sub-area 304 of the geographic area 302.

[0042] In operation 204, procedure 200 specifies that each assigned robot 104 determines a path and / or a method for measuring the assigned sub-area 304 based on the corresponding multiple sub-area parameters 306 for an assigned sub-area 304 and its robot parameters 308. According to some implementations, each robot 104 autonomously determines its path and / or method for measuring its assigned sub-area(s) 304.

[0043] In Operation 206, Procedure 200 specifies that each assigned robot 104, based on the route and / or the procedure, acquires data 207 representing a survey of the assigned sub-area 304. In Operation 208, Procedure 200 specifies that each assigned robot 104, based on the acquired data 207, generates a local, highly realistic map 209 of an assigned sub-area 304. Subsequently, in Operation 210, Procedure 200 specifies that each assigned robot 104 transmits or otherwise transfers survey data 211 representing the data 207 and / or the local, highly realistic map 209 of the assigned sub-area 304.

[0044] In operation 212, procedure 200 includes the server 102 combining the survey data 211 from the one or more assigned robots 104 to produce a highly realistic map of the geographic area 302.

[0045] Any method can be used to combine the survey data 211, 211a-n acquired by a plurality of autonomous robots 104 to produce a highly realistic map of a geographic area 302. An exemplary method is described in “Highly Efficient Image Stitching Based on Energy Map” by Tang et al., published in the 2009 2nd International Congress on Image and Signal Processing, IEEE, 2009, available at https: / / ieeexplore.ieee.org / abstract / document / 5304214, the full disclosure of which is incorporated herein by reference. Another exemplary method is described in “Automatic Panoramic Image Stitching using Invariant Features”, published in International Journal of Computer Vision 74 (2007): 59-73, available at https: / / link.springer.com / article / 10.1007 / s11263-006-0002-3, the full disclosure of which is included here by reference.Yet another example is described in “Adaptive As-Natural-As-Possible Image Stitching” by Chung-Ching et al., published in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, available at https: / / www.cv-foundation.org / openaccess / content_cvpr_2015 / html / Lin_Adaptive_As-Natural-As-Possible_Image_2015_CVPR_paper.html, the full disclosure of which is included here by reference. A further method is described in “Color Image Fidelity Metrics Evaluated Using Image Distortion Maps” by Zhang et al., published in Signal Processing 70.3 (1998): 201-214, available at https: / / www.sciencedirect.com / science / article / abs / pii / S 016516849800125X, the full disclosure of which is included here by reference.

[0046] Fig. Figure 4 is a flowchart of another exemplary arrangement of operations for a method for combining survey data 211, 211a-n acquired by a plurality of autonomous robots 104. The operations can be executed by data processing hardware of the server 102 or the robots 104 based on the execution of instructions stored in memory hardware of the server 102 or of the robots 104. Many other ways can be used to implement the method 400. For example, the order of execution of the operations can be changed and / or one or more of the operations and / or interactions can be modified, removed, subdivided, or combined. Furthermore, the operations can be derived from Fig. 4. e.g., by separate process streams, processors, devices, discrete logic, circuits, etc., executed sequentially and / or in parallel.

[0047] In operation 402, procedure 400 involves processing the survey data 211, 211a-n acquired for a specific sub-area 304 using a feature extraction device to generate, identify, or extract a plurality of features 404, 404a-n and a corresponding plurality of locations 406, 406a-n of the features 404. According to some examples, the feature extraction device includes one or more machine learning (ML) models trained to generate, identify, or extract features from image data. According to other examples, the feature extraction device uses a scale-invariant feature transformation to extract the features from image data.

[0048] In operation 408, procedure 400 involves processing the features 404 and the locations 406 using a feature comparison device to compare features between different sets of survey data 211 and to combine the survey data 211 into a high-quality map 410 of the geographic area 302 based on the compared features. According to some examples, the feature comparison device compares features 404 using a most projective image merging, a shape-preserving semi-projective image merging, or a combination thereof.

[0049] In Operation 412, Procedure 400 includes processing the high-quality map 410 and the survey data 211 and using a realism testing device to generate one or more realism ratings 414. Exemplary realism ratings 414 include, but are not limited to, an average Euclidean distance between compared features and an index of spatial information. According to some examples, one or more machine learning (ML) models trained to generate a realism rating for a stitched image are included.

[0050] Fig. Figure 5 is a flowchart of another exemplary arrangement of operations for a method for combining survey data 211, 211a-n acquired by a plurality of autonomous robots 104. The operations can be executed by data processing hardware of the server 102 or of robots 104 based on the execution of instructions stored in memory hardware of the server 102 or of robots 104. Many other ways can be used to implement the method 500. For example, the order of execution of the operations can be changed and / or one or more of the operations and / or interactions can be modified, removed, subdivided, or combined. Furthermore, the operations can be derived from Fig. 5 e.g. by separate process streams, processors, devices, discrete logic, circuits, etc., executed sequentially and / or in parallel.

[0051] When a specific sub-area 304 is measured by several robots 104, the procedure 500 ensures that the measurement data 211 collected by the robots 104 are checked for consistency and grouped accordingly.

[0052] According to the example shown from Fig. 5 refers to RG x B refers to the xth sub-area 304. n on the nth robot 104 and refers to B SCn on the robot parameters 308 of the nth robot 104. In operation 502, procedure 500 includes the selection of an area of ​​interest (ROl) RG. x 304 of a geographical area to be surveyed 302. In operation 504, procedure 500 includes the identification of the robots B n 104, which selected the ROI RG x 304 have been measured.

[0053] In operation 506, the procedure includes obtaining the robot parameters B. SCn308 for each of the robots identified in Operation 504 B n 104 and the assignment of ranks B Ri to the sensors of each robot B n 104. In particular, the robots identified in Operation 504 B n They exhibit 104 diverse sets of abilities. For example, the robots B can n 104 differ with respect to the type(s) and / or number of sensors (where, for example, robot A may have a high-precision camera and LiDAR, while robot B has only a high-precision camera). Accordingly, the information collected by robots 104 is ranked based on their sensor sets by assigning a higher rank B. Ri and weighted with a higher degree of realism to the survey data 211 collected by robot 104 with more sophisticated sensor sets.

[0054] In Operation 508, Procedure 500 includes obtaining survey data 211 from each of the robots B identified in Operation 504. n 104 and the processing of the survey data 211 to generate, identify, or extract a plurality of features and a corresponding plurality of locations. According to some examples, the features are extracted using one or more machine learning (ML) models trained to generate, identify, or extract features from image data. Alternatively, features can be extracted using a scale-invariant feature transformation to extract the features from image data.

[0055] In operation 510, procedure 500 involves calculating a weighted reality rating S of feature ranks. According to some examples, the weighted reality rating can be expressed as: S=(BRi⋅F)+⋯+(BRn⋅F) are calculated, where F is a matrix of weights reflecting the fidelity of different sensor types. The weighted fidelity rating S arbitrates discrepancies about features 404 of a specific sub-area between the robots 104 that have surveyed a sub-area 304. This arbitration logic resolves discrepancies about the extracted features 404 of a sub-area 304 based on corresponding robot fidelity ratings S. Once the features or properties of a sub-area 304 have been resolved, the survey data 211 are used to update the local map 410 for the corresponding sub-area 304.

[0056] In operation 512, procedure 500 involves combining the survey data 211 from the assigned robot(s) 104 to produce a highly realistic map of the geographical area 302.

[0057] Fig. Figure 6 is a flowchart of an exemplary sequence of operations for a computer-implemented procedure 600 for using a plurality of autonomous robots 104 to survey a geographic area 302. The operations can be executed by data processing hardware of the server 102 or the robots 104 based on the execution of instructions stored in memory hardware of the server 102 or of the robots 104. Many other ways can be used to implement the procedure 600. For example, the order in which the operations are executed can be changed, and / or one or more of the operations and / or interactions can be modified, removed, subdivided, or combined. Furthermore, the operations can be derived from Fig. 6. e.g., by separate process streams, processors, devices, discrete logic, circuits, etc., executed sequentially and / or in parallel.

[0058] In operation 602, procedure 600 includes receiving a request to generate a highly realistic map of a geographic area 302. In operation 602, procedure 600 includes obtaining a plurality of parameters of a geographic area that represent properties of the geographic area 302. In operation 604, procedure 600 includes subdividing the geographic area 302 into a plurality of sub-areas 304 based on the plurality of parameters of a geographic area. In operation 606, procedure 600 includes obtaining a corresponding plurality of robot parameters 308 that represent properties of the specific robot 104, for each specific robot 104 of a plurality of robots 104.

[0059] In operation 610, procedure 600 includes determining a corresponding plurality of sub-area parameters 306, representing properties of a specific sub-area 304, for each specific sub-area 304 of the plurality of sub-areas 304, based on the plurality of parameters of a geographical area. In operation 612, procedure 600 includes assigning a specific robot 104 of the plurality of robots 104 to survey the specific sub-area 304 for each specific sub-area 304 of the plurality of sub-areas 304, based on the corresponding plurality of sub-area parameters 306 and the corresponding plurality of robot parameters 308.In operation 614, procedure 600 includes obtaining corresponding survey data 211, acquired by the designated robot 104, for each designated sub-area 304 of the plurality of sub-areas 304, wherein the corresponding survey data 211 represent a survey of the designated sub-area 304 by the assigned designated robot 104.

[0060] In operation 616, procedure 600 includes combining the corresponding survey data 211 acquired by the assigned robots 104 to produce the highly realistic map 410 of the geographical area 302.According to some examples, combining the corresponding survey data 211 acquired by the assigned robot 104 to produce the highly realistic map 410 of the geographic area 302 involves extracting one or more features 404 and one or more locations 406 corresponding to the one or more features 404 from the corresponding survey data 211 for each specific sub-area 304 of the plurality of sub-areas 304; determining an orientation of the corresponding survey data 211 to the corresponding survey data 211 acquired by another robot 104 based on the one or more features 404 and the one or more locations 406; and combining the corresponding survey data 211 and the corresponding survey data 211 acquired by the other robot 104 using the orientation.According to some implementations, combining the corresponding survey data 211 and the corresponding survey data 211 acquired by the other robot 104 using the orientation involves the use of a weighted feature score. Here, one or more features are weighted using a first realism score assigned to the corresponding survey data 211 and a second realism score assigned to the corresponding survey data 211 acquired by the other robot 104.

[0061] Several implementations have been described. However, it should be understood that various modifications can be made without deviating from the inventive concept and scope of protection as disclosed. Accordingly, other implementations fall within the scope of protection of the following claims.

[0062] The foregoing description is given for illustrative and descriptive purposes only. It is not intended to be exhaustive or to limit the disclosure. Individual elements or features of a particular configuration are generally not restricted to that particular configuration, but are, where applicable, interchangeable and may be used in a selected configuration, even if it is not specifically shown or described. Furthermore, it may be modified in many ways. Such modifications are not considered a derogation from the disclosure, and all such modifications are intended to be included within the scope of protection of the disclosure. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited non-patent literature

[0000] “Highly Efficient Image Stitching Based on Energy Map” by Tang et al., published in 2009 2nd International Congress on Image and Signal Processing, IEEE, 2009, available at https: / / ieeexplore.ieee.org / abstract / document / 5304214

[0045] “Automatic Panoramic Image Stitching using Invariant Features”, published in International Journal of Computer Vision 74 (2007): 59-73, available at https: / / link.springer.com / article / 10.1007 / s11263-006-0002-3

[0045] Adaptive As-Natural-As-Possible Image Stitching“ von Chung-Ching u. a., veröffentlicht in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, verfügbar unter https: / / www.cv-foundation.org / openaccess / content_cvpr_2015 / html / Lin_Adaptive_As-Natural-As-Possible_Image_2015_CVPR_paper.html

[0045] Color Image Fidelity Metrics Evaluated Using Image Distortion Maps“ von Zhang u. a., veröffentlicht in Signal Processing 70.3 (1998): 201-214, verfügbar unter https: / / www.sciencedirect.com / science / article / abs / pii / S 016516849800125X

[0045]

Claims

[1] System that includes: a plurality of robots configured to survey assigned sub-areas of a geographic region; and a server in communication with the majority of robots, configured to perform operations that include: Receiving a request to generate a highly realistic map of the geographical area; Obtaining a plurality of parameters of a geographical area that represent properties of the geographical area; Subdividing the geographical area into a plurality of sub-areas based on the plurality of parameters of a geographical area; Obtain a corresponding plurality of robot parameters representing properties of the specific robot, for each specific robot of the plurality of robots; for each specific sub-area of ​​the plurality of sub-areas: Determining a corresponding plurality of sub-area parameters that represent properties of the specified sub-area, based on the plurality of parameters of a geographical area; Assigning a specific robot from the plurality of robots to measure the specific sub-area based on the corresponding plurality of sub-area parameters and the corresponding plurality of robot parameters; and Obtaining relevant survey data acquired by the designated robot, wherein the relevant survey data represent a survey of the designated sub-area by the assigned designated robot; and Combining the relevant survey data collected by the assigned robots to create a highly realistic map of the geographical area. [2] System according to claim 1, wherein each assigned robot is configured to perform operations comprising: Determining a distance and / or a procedure for measuring the assigned sub-area based on the corresponding plurality of sub-area parameters for an assigned sub-area and the corresponding plurality of robot parameters; Navigate to the assigned sub-area; Recording the relevant survey data representing a survey of the specific sub-area, based on the route and / or the procedure; and Providing the relevant survey data to the server. [3] System according to claim 1, wherein the assignment of a particular robot to the plurality of robots for measuring the particular sub-area on the basis of the corresponding plurality of sub-area parameters and the corresponding plurality of robot parameters comprises reducing an amount of energy expended by the assigned robots. [4] System according to claim 1, wherein the assignment of a particular robot to the plurality of robots for measuring the particular sub-area on the basis of the corresponding plurality of sub-area parameters and the corresponding plurality of robot parameters comprises maximizing coverage of the geographical area. [5] System according to claim 1, wherein the assignment of a particular robot to the plurality of robots for measuring the particular sub-area on the basis of the corresponding plurality of sub-area parameters and the corresponding plurality of robot parameters comprises reducing a measurement time. [6] System according to claim 1, wherein the corresponding plurality of robot parameters representing properties of a particular robot comprises at least one of the following: an ability to navigate on the ground; an ability to navigate on water; an ability to navigate in water; an ability to navigate in the air; a battery status; a loading time; a type of robot; a robot location; a coverage area; Cost of traveling one distance unit; a form; one or more sensor types; or one or more communication interfaces. [7] System according to claim 1, wherein the corresponding plurality of sub-area parameters representing properties of a particular sub-area comprises at least one of the following: a presence of shadows; a presence of sunshine; a topology; the presence of water; a height above sea level; one or more geographical hazards; a drivable surface; or a surface type. [8] System according to claim 1, wherein combining the corresponding survey data acquired by the assigned robot to generate the highly realistic map of the geographical area comprises: Extracting one or more features and one or more locations corresponding to the one or more features from the relevant survey data for each specific sub-area of ​​the plurality of sub-areas; Determining an alignment of the relevant survey data with the relevant survey data acquired by another robot, based on one or more features and one or more locations; and Combining the relevant survey data and the relevant survey data acquired by the other robot using the orientation. [9] System according to claim 8, wherein combining the corresponding measurement data and the corresponding measurement data acquired by the other robot using the orientation comprises using a weighted feature evaluation. [10] System according to claim 9, wherein one or more features are weighted using a first realism rating assigned to the corresponding measurement data and a second realism rating assigned to the corresponding measurement data acquired by the other robot.

Images