Collaborative exploration of environment space by mobile devices

Abstract

A mobile device obtains depth information from a depth sensor, and updates an exploration graph data structure to explore an exploration space. Operations determine relative pose information indicating pose of the mobile device relative to another mobile device, and send to the other mobile device a graph message including the relative pose information and the first exploration graph data structure. Operations receive from the other mobile device, a response graph message including a merged exploration graph data structure. Operations merge the merged exploration graph data structure with the first exploration graph data structure to generate a further merged exploration graph data structure. Operations determine a next exploration goal prioritizing individual nodes for exploration by the mobile device based on relative distance from nodes assigned to the other mobile device. Operations control movement of the mobile device based on the next exploration goal to explore the exploration space.

Description

COLLABORATIVE EXPLORATION OF ENVIRONMENT SPACE BY MOBILE DEVICESTECHNICAL FIELD

[0001] The present disclosure relates to controlling movement of mobile devices exploring an environment using depth sensor information.BACKGROUND

[0002] Mobile devices, such as robots, vehicles, and electronic devices transported by people, can perform autonomous exploration of an environment through operations where the mobile device navigates a previously unknown (i.e., unmapped) environment to create a map representation of the environment. The operations involve processing depth sensor data, updating a map, calculating the next best region to navigate to for exploration, planning and following a path to that region, and repeating this process until no more unmapped regions are left.

[0003] One algorithm for autonomous exploration is disclosed in the publication of D. Duberg and P. Jensfelt, “Ufoexplorer: Fast and scalable sampling based exploration with a graph-based planning structure,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 2487-2494, 2022, hereinafter "[3]." The algorithm executes the following actions whenever an update from the depth sensor(s) is received:1. Merge data from sensor(s) with map.2. Determine volume of map which has been updated.3. Update exploration graph nodes in relation to the updated volume.4. Stage updated nodes to have their utility re-calculated.5. Add new nodes to the exploration graph.6. Find and start execution of path to closest goal.

[0004] The exploration graph is an undirected graph where each node is encoded with utility in the form of an estimation of how much of the map may be uncovered by visiting that node. This metric will be referred to as the node's gain.

[0005] A multi-device exploration algorithm is disclosed in the publication of B. Zhou, H. Xu, and S. Shen, “Racer: Rapid collaborative exploration with a decentralized multi -uav system,” 2022. [Online], Available: https: / / arxiv.org / abs / 2209.08533, hereinafter "[!]."Several mobile devices collaborate in creating one map of the environment. The disclosed operations are a decentralized approach including segmenting the mobile devices' maps grid representations into sparser versions and dividing the segments as exploration tasks between mobile devices.

[0006] Another multi-device exploration algorithm is disclosed in the publication Y. Chang, L. Ballotta, and L. Carlone, “D-lite: Navigation-oriented compression of 3d scene graphs under communication constraints,” 2022. [Online], Available: https: / / arxiv.org / abs / 2209.06111, hereinafter "[2]." The disclosed operations provide a graph-based implementation of multi-device SLAM where the environment is modeled using 3D scene graphs. Mobile devices may query each other for paths to certain positions and receive them in the form of sparse pruned graphs.

[0007] These multi-device exploration algorithms can have operational inefficiencies and problems, which can include one or more of the following:• A centralized node / agent is required to gather all sensor data and coordinate the mobile devices. This requires constant connection, which may not be feasible, and particularly indoor environments can be difficult to guarantee. Furthermore, the bandwidth requirements of such solutions are very high since depth sensor data (usually between 10-30Hz) needs to be constantly streamed to the centralized node.• Mobile devices may share their maps with each other whenever they meet. This is a bottleneck for the network, since the amount of data to be shared grows with the size of the environment and can easily become prohibitively large to share in real time. This might lead to devices having to remain stationary while information is exchanged and communication is not broken before completion, and which results in longer exploration times and higher battery consumption that may prevent the mobile devices from completing their exploration of the entire environment.• Mobile devices share only frontier nodes, i.e., positions in the border between known and unknown space. Although these operations reduce the bandwidth requirements drastically, mobile devices have no information as to which areas are / were explored by other mobile devices.SUMMARY

[0008] Some embodiments disclosed herein are directed to a first mobile device that includes a wireless transceiver operative to provide a communication connection with a second mobiledevice, a depth sensor configured to provide depth information comprising data points indicating locations of object features sensed by the depth sensor within an exploration space, and at least one processor to perform operations. The operations obtain the depth information from the depth sensor, and update a first exploration graph data structure used by the first mobile device to explore the exploration space, based on the depth information. The first exploration graph data structure has nodes representing locations in the exploration space and edges representing paths that the first mobile device can be moved between the nodes. For each of the nodes, an information gain metric is assigned to the node based on an amount of new information about the exploration space which can be acquired by the depth sensor of the first mobile device exploring at the location represented by the node. For each of the edges, a weight is assigned to the edge based on the distance the first mobile device would move on the path between the nodes. Operations determine relative pose information indicating pose of the first mobile device relative to the second mobile device. Operations send through the wireless transceiver to the second mobile device, a graph message comprising the relative pose information and the first exploration graph data structure containing an indication of a first group of assigned nodes which are assigned to the first mobile device for exploration. Operations receive through the wireless transceiver from the second mobile device, a response graph message including a merged exploration graph data structure containing an indication of a third group of assigned nodes which are assigned to the first mobile device for exploration and an indication of a fourth group of assigned nodes which are assigned to the second mobile device for exploration. Operations merge the merged exploration graph data structure with the first exploration graph data structure to generate a further merged exploration graph data structure. Operations determine a next exploration goal prioritizing individual nodes in the third group of assigned nodes for exploration by the first mobile device based on relative distance from nodes of the fourth group of assigned nodes. Operations control movement of the first mobile device based on the next exploration goal to explore the exploration space.

[0009] Some other embodiments are directed to a corresponding second mobile device that includes a wireless transceiver operative to provide a communication connection with a first mobile device, a depth sensor configured to provide depth information comprising data points indicating locations of object features sensed by the depth sensor within an exploration space, and at least one processor to perform operations. The operations obtain the depth information from the depth sensor, and update a second exploration graph data structure used by the second mobile device to explore the exploration space, based on the depth information. Thesecond exploration graph data structure has nodes representing locations in the exploration space and edges representing paths that the second mobile device can be moved between the nodes. For each of the nodes, an information gain metric is assigned to the node based on an amount of new information about the exploration space which can be acquired by the depth sensor of the second mobile device exploring at the location represented by the node. For each of the edges, a weight is assigned to the edge based on a distance the second mobile device would move on the path between the nodes. Operations receive through the wireless transceiver, a graph message including relative pose information indicating pose of the first mobile device relative to pose of the second mobile device, and including a first exploration graph data structure containing an indication of a first group of assigned nodes which are assigned to the first mobile device for exploration. Operations merge the first exploration graph data structure with the second exploration graph data structure defining a second group of assigned nodes assigned to the second mobile device for exploration, to generate a merged exploration graph data structure. Operations assign a third group of assigned nodes in the merged exploration graph data structure to the first mobile device for exploration, and a fourth group of assigned nodes in the merged exploration graph data structure to the second mobile device for exploration. Operations send to the first mobile device a response graph message including the merged exploration graph data structure and indicating the third and fourth groups of assigned nodes. Operations determine a next exploration goal prioritizing individual nodes in the fourth group of assigned nodes for exploration by the second mobile device based on relative distance from nodes of the third group of assigned nodes.Operations control movement of the second mobile device based on the next exploration goal to explore the exploration space.

[0010] Some other embodiments are directed to a corresponding method by a first mobile device. The method includes obtaining from a depth sensor, depth information comprising data points indicating locations of object features sensed by the depth sensor within an exploration space. The method further includes updating a first exploration graph data structure used by the first mobile device to explore the exploration space, based on the depth information. The first exploration graph data structure has nodes representing locations in the exploration space and edges representing paths that the first mobile device can be moved between the nodes. For each of the nodes, an information gain metric is assigned to the node based on an amount of new information about the exploration space which can be acquired by the depth sensor of the first mobile device exploring at the location represented by the node. For each of the edges, a weight is assigned to the edge based on distance the firstmobile device would move on the path between the nodes. The method further includes determining relative pose information indicating pose of the first mobile device relative to a second mobile device, and sending through the wireless transceiver to the second mobile device, a graph message comprising the relative pose information and the first exploration graph data structure containing an indication of a first group of assigned nodes which are assigned to the first mobile device for exploration. The method further includes receiving through the wireless transceiver from the second mobile device, a response graph message including a merged exploration graph data structure containing an indication of a third group of assigned nodes which are assigned to the first mobile device for exploration and an indication of a fourth group of assigned nodes which are assigned to the second mobile device for exploration. The method further includes merging the merged exploration graph data structure with the first exploration graph data structure to generate a further merged exploration graph data structure. The method further includes determining a next exploration goal prioritizing individual nodes in the third group of assigned nodes for exploration by the first mobile device based on relative distance from nodes of the fourth group of assigned nodes, and controlling movement of the first mobile device based on the next exploration goal to explore the exploration space.

[0011] Some other embodiments are directed to a corresponding method by a second mobile device. The method includes obtaining from a depth sensor, depth information comprising data points indicating locations of object features sensed by the depth sensor within an exploration space. The method further includes updating a second exploration graph data structure used by the second mobile device to explore the exploration space, based on the depth information. The second exploration graph data structure has nodes representing locations in the exploration space and edges representing paths that the second mobile device can be moved between the nodes. For each of the nodes, an information gain metric is assigned to the node based on an amount of new information about the exploration space which can be acquired by the depth sensor of the second mobile device exploring at the location represented by the node. For each of the edges, a weight is assigned to the edge based on a distance the second mobile device would move on the path between the nodes. The method further includes receiving through the wireless transceiver, a graph message comprising relative pose information indicating pose of a first mobile device relative to pose of the second mobile device, and comprising a first exploration graph data structure containing an indication of a first group of assigned nodes which are assigned to the first mobile device for exploration. The method further includes merging the first explorationgraph data structure with the second exploration graph data structure defining a second group of assigned nodes assigned to the second mobile device for exploration, to generate a merged exploration graph data structure. The method further includes assigning a third group of assigned nodes in the merged exploration graph data structure to the first mobile device for exploration, and a fourth group of assigned nodes in the merged exploration graph data structure to the second mobile device for exploration. The method further includes sending to the first mobile device a response graph message including the merged exploration graph data structure and indicating the third and fourth groups of assigned nodes. The method further includes determining a next exploration goal prioritizing individual nodes in the fourth group of assigned nodes for exploration by the second mobile device based on relative distance from nodes of the third group of assigned nodes, and controlling movement of the second mobile device based on the next exploration goal to explore the exploration space.

[0012] As will be explained in further detail below, a potential advantage which may be provided by these and other embodiments is that mobile devices coordinate through sharing of at least their exploration graph data structures or, in some embodiments, only sharing their exploration graph data structures. These operations can involve enhancing the exploration graphs with indications of which mobile device is expected (assigned) to explore which nodes. By doing so, the second (observed) mobile device can infer which regions have been explored and where the first (observer) mobile device (second mobile device in Figure 2) plans to travel and explore. This related exploration can avoid duplication of effort by two or more mobile devices exploring overlapping areas, can reduce communication resources which are used to perform coordinated exploration, and can speed up the exploration.

[0013] Other mobile devices, methods, and computer program products according to embodiments will be or become apparent to one with skill in the art upon review of the following drawings and detailed description. It is intended that all such mobile devices, methods, and computer program products be included within this description, be within the scope of the present disclosure, and be protected by the accompanying claims. Moreover, it is intended that all embodiments disclosed herein can be implemented separately or combined in any way and / or combination.BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Aspects of the present disclosure are illustrated by way of example and are not limited by the accompanying drawings. In the drawings:

[0015] Figure 1 illustrates a combined flowchart of operations and communications by an observer mobile device and an observed mobile device performing coordinated exploration of an exploration space in accordance with some embodiments;

[0016] Figure 2 illustrates another combined flowchart of operations and communications by first and second mobile devices which are cooperative exploring an exploration space in accordance with some embodiments;

[0017] Figures 3a and 3b illustrate assigned nodes in an exploration graph data structure and corresponding indications in a graph message for communication in accordance with some embodiments;

[0018] Figures 4a and 4b illustrate grouping of assigned nodes in an exploration graph data structure in accordance with some embodiments;

[0019] Figure 5 illustrates a flowchart of operations by a first mobile device functioning as an observer in accordance with some embodiments;

[0020] Figure 6 illustrates a flowchart of operations by a second mobile device functioning as an observed in accordance with some embodiments; and

[0021] Figure 7 illustrates components of mobile devices and components of a computing node which can operate in accordance with some embodiments.DETAILED DESCRIPTION

[0022] Inventive concepts will now be described more fully hereinafter with reference to the accompanying drawings, in which examples of embodiments of inventive concepts are shown. Inventive concepts may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of various present inventive concepts to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. Components from one embodiment may be tacitly assumed to be present / used in another embodiment.

[0023] Embodiments of the present disclosure are directed to operations for collaborative exploration and mapping of an environment by a plurality of mobile devices. The operations can be performed without requiring continuous connection between the mobile devices and can reduce the communication bandwidth that is utilized for coordination between the mobile devices. Through the coordination, the total distance travelled by each mobile device to complete exploration of the environment can be reduced.

[0024] Through various operational embodiments, mobile devices that are exploring the same environment space share their exploration graph data structures instead of sharing map representations of the environment space and / or sharing frontier nodes. The shared exploration graph already contains information that is indicative of traversable areas and frontier areas. Some further embodiments provide enhancements to what is indicated by the shared exploration graphs, how the mobile devices communicate the exploration graphs and coordinate the exploration. For example, exploration graph nodes of an exploration graph can be enhanced with indication of which mobile device is assigned (intended) to explore which group of nodes. Then, when a mobile device shares its exploration graph, it can group together exploration goals and couple these groups with which mobile device is likely to explore which group. These enhancements inform the receiving mobile device which areas have been explored by the transmitting mobile device, but also where it plans to explore. Finally, the mobile devices can choose their groups of nodes which are located more distance from the other mobile device, thus maximizing coverage and reducing the number of messages exchanged.

[0025] Figure 1 illustrates a combined flowchart of operations and communications by an observer mobile device and an observed mobile device performing coordinated exploration of an exploration space in accordance with some embodiments.

[0026] Referring to Figure 1, the observer mobile device is exploring the exploration space using a depth sensor that is configured to provide depth information comprising data points indicating locations of object features sensed by the depth sensor within an exploration space. The depth sensor is transported by the observer mobile device while exploring. Example types of depth sensors include, without limitation, Lidar sensors, stereo cameras, monocular cameras, radars, and ultrasonic sensors.

[0027] The observer mobile device updates an observer exploration graph data structure used by the observer mobile device to explore the exploration space, based on the depth information. The observer exploration graph data structure has nodes representing locations in the exploration space and edges representing paths that the observer mobile device can be moved between the nodes to explore the exploration space. Each of the nodes can be assigned an information gain metric which may be based on an amount of new information about the exploration space which can be acquired by the depth sensor of the observer mobile device exploring at the location represented by the node. For each of the edges a weight is assigned to the edge which may be based on a distance the observer mobile device would move on the path between the nodes.

[0028] The observer mobile device observes 100 another mobile device (called the "observed mobile device") such as by operation of the depth sensor (e.g., camera) observing the observed mobile device, by operation of a radio receiver sensing signaling from the observed mobile device, or by other operations. The observer mobile device determines relative pose information indicating pose (e.g., 2D or 3D location and / or angular orientation, such as 3 axial directions and 3 rotational angles about axes) of the observer mobile device relative to the observed mobile device. The observer mobile device initiates the establishment of a communication connection to the observed mobile device. The observer mobile device creates a graph message including the relative pose information and the observer exploration graph data structure containing an indication of a first group of assigned nodes which are assigned to (intended for) the observer mobile device to explore. The observer mobile device sends the graph message to the observed mobile device.

[0029] The observed mobile device is similarly transporting a depth sensor while exploring the exploration space. The observed mobile device updates an observed exploration graph data structure used by the observed mobile device to explore the exploration space. The observed exploration graph data structure includes an indication of a second group of assigned nodes assigned to (intended for) the second mobile device to explore. The observed mobile device receives the graph message and merges 112 the observer exploration graph data structure with the observed exploration graph data structure to generate a merged exploration graph data structure.

[0030] The observed mobile device re-determines 114 assignments of which nodes are to be explored by which of the mobile devices. More particularly, the observed mobile device can assign a third group of assigned nodes in the merged exploration graph data structure to the observer mobile device for exploration, and a fourth group of assigned nodes in the merged exploration graph data structure to the observed mobile device for exploration. The terms group and cluster may be used interchangeably herein to refer to nodes which are to be treated in a similar manner, such as being intended for exploration by a particular mobile device.

[0031] The observed mobile device selects 116 a new exploration goal based on the merged exploration graph data structure. Selection of the new exploration goal can correspond to determining a next exploration goal that prioritizes individual nodes in the fourth group of assigned nodes for exploration by the observed mobile device based on relative distance from nodes of the third group of assigned nodes to be explored by the observer mobile device. Theobserved mobile device can then control its movement based on the next exploration goal to explore the exploration space.

[0032] The observed mobile device creates 118 a response graph message that includes the merged exploration graph data structure and indicates the third and fourth groups of assigned nodes, and sends response graph message to the observer mobile device.

[0033] The observer mobile device receives the response graph message, and merges 104 the merged exploration graph data structure with the observer exploration graph data structure to generate a further merged exploration graph data structure. The observer mobile device re-determines 106 assignments of which nodes are to be explored by which of the mobile devices. The observer mobile device selects 108 new exploration goal based on the further merged exploration graph data structure. Selection of the new exploration goal can correspond to determining a next exploration goal prioritizing individual nodes in the third group of assigned nodes intended for exploration by the observer mobile device based on relative distance from nodes of the fourth group of assigned nodes intended for exploration by the observed mobile device. The observer mobile device can then control its movement based on the next exploration goal to explore the exploration space.

[0034] Figure 2 illustrates another combined flowchart of operations and communications by first and second mobile devices which are cooperative exploring an exploration space in accordance with some embodiments.

[0035] In the scenario of Figure 2, there are a plural number N mobile devices which are cooperatively exploring an unknown environment. Each mobile device performs the exploration by using depth information from a depth sensor to update its local exploration map (represented by an exploration graph data structure), thereby expanding its local exploration graph, and then deciding the next exploration goal depending on the state of the exploration graph as explained in further detail below. The objective of a collaborative exploration task is to efficiently explore the environment through sharing the workload among the exploring mobile devices.

[0036] Operating in a decentralized manner, mobile devices can only share information between each other when they are within communication range. The information shared between mobile devices may include the exploration graph, indication of whose nodes (vertices) correspond to poses in the environment and store a metric of information gain ("information gain metric") which is the amount of new information about the environment that can be acquired from that pose. The edges of the graph indicate that the mobile devicecan navigate between the connected nodes (vertices), and its weight represents the distance between them.

[0037] To enable coordination among the mobile devices while exploring, nodes with high information gain (a value above a certain threshold), also called high-gain nodes or exploration goals, can be enhanced with information indicating which mobile device is planning (assigned) to explore it.

[0038] According to some embodiments, operations enable mobile devices to coordinate through communicative sharing of at least exploration graph data structures or, in some embodiments, only sharing exploration graph data structures. These operations can involve enhancing the exploration graphs with indications of which mobile device is expected (assigned) to explore which nodes. By doing so, the observed mobile device (second mobile device in Figure 2) can infer which regions have been explored and where the observer mobile device (first mobile device in Figure 2) plans to travel and explore. This related exploration can avoid duplication of effort by two or more mobile devices exploring overlapping areas, can reduce communication resources which are used to perform coordinated exploration, and can speed up the exploration.

[0039] Referring to Figure 2, the first mobile device gets 200 sensor data (depth information) from its transported depth sensor. When the first mobile device determines 202 from the sensor data that another (second) mobile device is not visible, it proceeds with updating 204 its map (represented by a first exploration graph data structure), calculates 206 a next exploration goal, and navigates 208 towards that goal to further explore the exploration space. In contrast, when the first mobile device determines 202 that a second mobile device is visible, it determines 210 relative pose information indicating pose of the first mobile device relative to the second mobile device, creates 212 a graph message including the relative pose information and the first exploration graph data structure containing an indication of a first group of assigned nodes which are assigned to (intended for use by) the first mobile device for exploration, and sends 214 the graph message to the second mobile device.

[0040] The graph message can be created 212 to include the first exploration graph data structure and may further include a coordinate transform between the first and second mobile devices. To compose the graph message, the first (observer) mobile device uses the transform on its first exploration graph data structure so that its vertices and edges are in the second (observed) mobile device's reference frame.

[0041] In a further embodiment, the operations transform the first exploration graph data structure from a first coordinate system used by the first (observer) mobile device for determining locations of object features sensed by the depth sensor, to a second coordinate system used by the second (observed) mobile device for determining locations of object features sensed by a depth sensor of the second mobile device, using the relative pose information. Operations then generate the graph message using the first exploration graph data structure after the transform to the second coordinate system is completed.

[0042] Figures 3a and 3b illustrate assigned nodes in an exploration graph data structure and corresponding indications in a graph message for communication in accordance with some embodiments. Figure 3a corresponds to no use of anchor nodes while Figure 3b corresponds to use of two anchor nodes 7 and 8. This example graph message contains the following information: a node list which lists the nodes with identifiers, an edge list with the node ids that are connected by edges, poses of the nodes, indications of claimed nodes and indications of which mobile devices are claiming which of the claim nodes for exploration. The lists of poses for 2D motion, lists an x coordinate value, a y coordinate value, and a directional (e.g., yaw) angle.

[0043] In the illustrative example of Figure 3a, node list shows that nodes 0 through 6 are involved in the coordinated exploration that can contribute to the exploration graph data structure. The edge list indicates that node 0 is connected to nodes 1 and 2, indicates that node 1 is connected to nodes 3, 4, and 4, indicates that node 4 is also connected to node 5, and indicates that node 5 is connected to node 6. The poses are indicated as follows: node 0 at location 0,2 (x,y) with direction 0, node 1 at location 3,3 (x,y) with direction 0, node 2 at location 2,0 (x,y) with direction 0.5, node 3 at location 1,6 (x,y) with direction 0, node 4 at location 4,5 (x,y) with direction 0, node 5 at location 4,3 (x,y) with direction 1.5, and node 6 at location 5,1 (x,y) with direction 2. The example lists of claimed nodes and claiming mobile devices (indicated by "R" in Figures 3a-3b) together indicate that node 3 is claimed by each of mobile devices R0, Rl, R3 for exploration, node 4 is claimed by mobile device R2 for exploration, and node 0 is claimed by each of mobile devices R0, Rl, R2 for exploration.

[0044] In some embodiments, to ensure that the second (observed) mobile device will be able to connect the first exploration graph data structure from the first (observer) mobile device to its own second exploration graph data structure, anchor nodes are used and included in the graph message, e.g., as shown in Figure 3b. Figure 3b differs from Figure 3a by adding two anchor nodes, node 7 and node 8 shown in dashed lines. Anchor nodes in this context are nodes sent along with the graph message, and only exist to connect the two graphs (first andsecond exploration graph data structures). The creation of anchor nodes assumes the first (observer) mobile device has a sufficiently clear line-of-sight towards the second (observed) mobile device. This assumption may be required since only the exploration graph data structure is shared, so the first (observer) mobile device and second (observed) mobile device have no other way to connect their exploration graph data structures while ensuring that the created edges will only cross free space. The creation of anchor nodes may be performed according to the following logic: 1) create a node at the location of the second (observed) mobile device; 2) if an edge may be created from this node to the second exploration graph data structure, an edge is inserted, and the creation of anchor nodes has succeeded; 3) in contrast if no edge can be created from this node, a new node is placed as far from the previous as the maximum edge length allows, in the direction of the first (observer) mobile device; and 4) this process repeats until a connection to the first (observer) mobile devices' first exploration graph data structure is made.

[0045] The second mobile device similarly gets 201 sensor data (depth information) from its transported depth sensor. When the second mobile device determines 203 from the sensor data that another (third) mobile device is not visible, it proceeds with updating 205 its map (represented by a second exploration graph data structure), calculates 207 a next exploration goal, and navigates 209 towards that goal to further explore the exploration space. In contrast, when the second mobile device determines 203 that a third mobile device is visible, it determines 211 relative pose information indicating pose of the second mobile device relative to the third mobile device, creates 211 a graph message including the relative pose information and the second exploration graph data structure containing an indication of a second group of assigned nodes which are assigned to (intended for use by) the second mobile device for exploration, and sends 211 the graph message to the third mobile device.

[0046] The second mobile device receives 214 the graph message and generates a merged exploration graph data structure by merging 216 the first exploration graph data structure with the second exploration graph data structure defining a second group of assigned nodes assigned to the second mobile device for exploration. The second mobile device updates 218 which groups of nodes are assigned (intended) for exploration by which of the mobile devices. More particularly, the second mobile device assigns a third group of assigned nodes in the merged exploration graph data structure to the first mobile device for exploration, and a fourth group of assigned nodes in the merged exploration graph data structure to the second mobile device for exploration.

[0047] The second mobile device determines (e.g., selects) 220 a next exploration goal based on prioritizing individual nodes in the fourth group of assigned nodes for exploration by the second mobile device based on relative distance from nodes of the third group of assigned nodes. The second mobile device creates 222 a response graph message including the merged exploration graph data structure and indicating the third and fourth groups of assigned nodes, and sends 224 the response graph message to the first mobile device. The second mobile device controls its movement based on the next exploration goal to explore the expiration space. The second mobile device may remain stationary while performing various of operations, such as while sending the response graph message to the first mobile device to avoid potential loss of communications which can arise if moving away from the first mobile device, and can then resume exploration once communications are completed.

[0048] The first mobile device receives the response graph message and merges 226 the merged exploration graph data structure with the first exploration graph data structure to generate a further merged exploration graph data structure. The first mobile device may update 228 which nodes of the further merged exploration graph data structure are assigned to (intended for) exploration by which of the mobile devices, and determine 230 (e.g., selects) a next exploration goal prioritizing individual nodes in the third group of assigned nodes for exploration by the first mobile device based on relative distance from nodes of the fourth group of assigned nodes.

[0049] The first mobile device can then control 232 its movement based on the next exploration goal to explore the exploration space, which may include resuming exploration when the first mobile device has stopped movement while performing various of operations, e.g., coordinating with the second mobile device.

[0050] These and other operations are now described more generally with reference to Figures 5 and 6, before describing in more detail various ways the operations by an observer mobile device e.g., first mobile device, and an observed mobile device, e.g., second mobile device, may be performed.

[0051] Figure 5 illustrates a flowchart of operations by a first mobile device functioning as an observer in accordance with some embodiments. Figure 6 illustrates a flowchart of operations by a second mobile device functioning as an observed in accordance with some embodiments.

[0052] Referring now to Figure 5, the first mobile device obtains 500 the depth information from the depth sensor, and updates 502 a first exploration graph data structure used by the first mobile device to explore the exploration space, based on the depth information. Thefirst exploration graph data structure has nodes representing locations in the exploration space and edges representing paths that the first mobile device can be moved between the nodes. For each of the nodes an information gain metric is assigned to the node based on an amount of new information about the exploration space which can be acquired by the depth sensor of the first mobile device exploring at the location represented by the node. For each of the edges a weight is assigned to the edge based on the distance the first mobile device would move on the path between the nodes.

[0053] The operation to update 502 the first exploration graph data structure may include to determine the information gain metric which is to be assigned to individual nodes in the first exploration graph data structure based on whether the location represented by the node has been explored by the depth sensor of the first mobile device and based on the weights assigned to the edges of the node. A relatively lower value may be assigned to any of the nodes in the first exploration graph data structure which correspond to locations that have been explored by the depth sensor of the first mobile device.

[0054] The first mobile device determines 504 relative pose information indicating pose of the first mobile device relative to the second mobile device. The first mobile device sends 506 through a wireless transceiver to the second mobile device, a graph message including the relative pose information and the first exploration graph data structure containing an indication of a first group of assigned nodes which are assigned to the first mobile device for exploration.

[0055] The first mobile device receives 508 through the wireless transceiver from the second mobile device, a response graph message including a merged exploration graph data structure containing an indication of a third group of assigned nodes which are assigned to the first mobile device for exploration and an indication of a fourth group of assigned nodes which are assigned to the second mobile device for exploration. The first mobile device merges 510 the merged exploration graph data structure with the first exploration graph data structure to generate a further merged exploration graph data structure. The first mobile device determines a next exploration goal prioritizing individual nodes in the third group of assigned nodes for exploration by the first mobile device based on relative distance from nodes of the fourth group of assigned nodes, and controls 514 its movement based on the next exploration goal to explore the exploration space.

[0056] Referring now to Figure 6, the second mobile device obtain 600 depth information from its depth sensor, and updates 602 a second exploration graph data structure used by the second mobile device to explore the exploration space, based on the depth information. Thesecond exploration graph data structure has nodes representing locations in the exploration space and edges representing paths that the second mobile device can be moved between the nodes. For each of the nodes an information gain metric is assigned to the node based on an amount of new information about the exploration space which can be acquired by the depth sensor of the second mobile device exploring at the location represented by the node. For each of the edges a weight is assigned to the edge based on the distance the second mobile device would move on the path between the nodes.

[0057] The operation to update 602 the second exploration graph data structure used by the second mobile device to explore the exploration space, can include to determine the information gain metric to be assigned to individual nodes in the second exploration graph data structure based on whether the location represented by the node has been explored by the depth sensor of the second mobile device and based on the weights assigned to the edges of the node. A relatively lower value is assigned to any of the nodes in the second exploration graph data structure which correspond to locations that have been explored by the depth sensor of the second mobile device.

[0058] The second mobile device receives 604 through a wireless transceiver, a graph message including relative pose information indicating pose of the first mobile device relative to pose of the second mobile device, and comprising a first exploration graph data structure containing an indication of a first group of assigned nodes which are assigned to the first mobile device for exploration.

[0059] The second mobile device merges 606 the first exploration graph data structure with the second exploration graph data structure defining a second group of assigned nodes assigned to the second mobile device for exploration, to generate a merged exploration graph data structure.

[0060] The second mobile device assigns 608 a third group of assigned nodes in the merged exploration graph data structure to the first mobile device for exploration, and a fourth group of assigned nodes in the merged exploration graph data structure to the second mobile device for exploration.

[0061] The operation to assign 608 third and fourth groups of assigned nodes can include to select high-gain nodes from among the nodes in the merged exploration graph data structure based on determining the high-gain nodes have assigned information gain metrics that satisfy a threshold value, and assign the high-gain nodes to the fourth group of assigned nodes in the merged exploration graph data structure for exploration by the second mobile device. Furthermore, the operation to assign the high-gain nodes to the fourth group of assignednodes in the merged exploration graph data structure for exploration by the second mobile device, can include to exclude from selection as one of the high-gain nodes any node in the second group that overlaps in location with any of node in the first group and has a lower information gain metric than the overlapped node in the first group.

[0062] The operation to assign 608 third and fourth groups of assigned nodes can include to split a group of nodes in the merged second exploration graph data structure into the third group of assigned nodes which are assigned to the first mobile device for exploration and into the fourth group of assigned nodes which are assigned to the second mobile device for exploration.

[0063] The second mobile device sends 610 to the first mobile device a response graph message including the merged exploration graph data structure and indicating the third and fourth groups of assigned nodes. The operation to split may include to add to the third group the nodes in the merged exploration graph data structure that satisfy a closeness rule relative to the first mobile device, and add to the fourth group the nodes in the merged exploration graph data structure that satisfy the closeness rule relative to the second mobile device.

[0064] The response graph message sent by the second mobile device 610 may include a list of the nodes in the merged exploration graph data structure, a list of the edges connecting the nodes in the merged exploration graph data structure, and an indication of which of the nodes in the merged exploration graph data structure are assigned to the first mobile device for exploration and which are assigned to the second mobile device for exploration. The response graph message may further include, for each additional mobile device other than the first and second mobile devices which has a communication connection to at least one of the first and second mobile devices, a list mapping the additional mobile device to individual ones of the nodes in the merged second exploration graph data structure that the additional mobile device is assigned to explore.

[0065] The second mobile device 610 may transform the merged exploration graph data structure from a second coordinate system used by the second mobile device for determining locations of object features sensed by the depth sensor, to a first coordinate system used by the first mobile device for determining locations of object features sensed by a depth sensor of the first mobile device, using the relative pose information. The second mobile device 610 may then generate the response graph message using the merged exploration graph data structure after the transform to the first coordinate system is completed.

[0066] The second mobile device determines 612 a next exploration goal prioritizing individual nodes in the fourth group of assigned nodes for exploration by the second mobiledevice based on relative distance from nodes of the third group of assigned nodes, and controls 614 its movement based on the next exploration goal to explore the exploration space.

[0067] In some embodiments, the first and / or second mobile device operates to determine which of the first and second mobile devices is to operate in an observer role responsible for sending the graph message and which is to operate in an observed role responsible for receiving and processing the graph message. In one illustrative embodiment, the determination corresponds to selecting the first mobile device to operate in an observer role based on at least one of: the first mobile device initiating establishment of the communication connection between the first and second mobile devices, sensing the second mobile device before being sensed by the second mobile device, having a depth sensor that is sensing location of the second mobile device, and operating to determine a coordinate transform to the second mobile device.

[0068] In order to prevent mobile devices from attempting to establish connection too many times within a short period of time, the operations to establish a connection can define a timeout of teseconds between each connection attempt, where the timeout is a time duration during which the mobile devices will not exchange messages if they have previously done so within the last teseconds.

[0069] Further optional operational embodiments are directed to ways to merge the first and second exploration graph data structures, splitting groups (clusters) of nodes between the first and second mobile devices, and updating the exploration plans.

[0070] The second (observed) mobile device may merge 606 the first and second exploration graph data structures to generate a merged exploration graph data structure, through operations that are based on one or more of: determinations of absence of an exploration graph generally indicates unknown space; an explored area (or low-gain value area) of an exploration graph indicates that area is free-space; an unexplored area (or high-gain value area) of an exploration graph indicates the presence of unknown space which is possible to visit; and when two graphs overlap, the graph with the lower information gain metric should be used as that is the graph created by the mobile device with the best map for that area.

[0071] The second (observed) mobile device may determine 612 its next exploration goal based on the merged exploration graph data structure. The second (observed) mobile device may modify the merged exploration graph data structure to split up groups (clusters) of nodes between the first and second mobile devices. For example, referring to Figure 6, the operation to assign 608 the third group of assigned nodes in the merged exploration graph data structureto the first (observer) mobile device for exploration, and the fourth group of assigned nodes in the merged exploration graph data structure to the second (observed) mobile device for exploration, can include to split a group of nodes in the merged second exploration graph data structure into the third group of assigned nodes which are assigned to the first mobile device for exploration and into the fourth group of assigned nodes which are assigned to the second mobile device for exploration. The operation to split can include to add to the third group the nodes in the merged exploration graph data structure that satisfy a closeness rule relative to the first mobile device, and add to the fourth group the nodes in the merged exploration graph data structure that satisfy the closeness rule relative to the second mobile device.

[0072] Figures 4a and 4b illustrate new grouping (splitting) of assigned nodes in an exploration graph data structure into two groups relative to the illustrated separating line 420, in accordance with some embodiments. The new grouping (splitting) operation can include the following:1. If there are no groups (clusters) left, no split occurs.2. If there is only one group (cluster) remaining it may be claimed by both the first and second mobile devices to allow joint exploration thereof.3. Gather groups (clusters (e.g., all clusters)) claimed by one of the mobile devices in the exchange, creating a separating plane between the groups (clusters). In the illustrated example of Figure 4a before groups of nodes are split, the separating plane 420 is defined to separate the groups 400, 402, and 404 from the groups 406, 408, and 410. In Figure 4b also before groups of nodes are split, nodes indicated filled-in black belong to a different mobile device than the nodes indicated with diagonal hashing. Pose of the separating plane can be defined by identifying two groups (clusters) of nodes furthest from each other (e.g., 400 and 410 in Figure 4a), and defining the separating plane using a vector between centers of the two groups (clusters) as a plane-norm and positions to be about (e.g., exactly) in-between the two groups (clusters).4. Groups (clusters) may then be split up by having all groups (clusters) on one side of the separating plane be claimed only by one mobile device, and the clusters on the other side of the separating plane be claimed by the other mobile device.5. The second (observed) mobile device takes the group (cluster) furthest from the first (observer) mobile device's groups (clusters) and sets that group) cluster center as its next exploration goal.

[0073] Some operational embodiments for updating assignment of which nodes are assigned to which mobile devices for exploration are further explained.

[0074] Exploration goals that are near each other are assumed to refer to the same area of the environment. Also, if a mobile device plans to explore one specific exploration goal, it can be assumed to eventually explore all exploration goals in that area of the environment, since the next goal tends to be the closest to the mobile device's position.

[0075] Given the previous assumptions, some embodiments group (cluster) exploration goals together. The process may start by removing previous assignment of a groups of nodes to mobile devices, and iterating through each node in the set of high-gain nodes. If the node is not assigned in a group (cluster) of assigned nodes, it is used to create a new group of assigned nodes. The group (cluster) is then placed in a set of groups depending on which mobile devices are estimated to explore the group (cluster).

[0076] In a further embodiment, operations to generate new groups can include using a modified depth-first search algorithm over the high-gain nodes. In the algorithm, depth can be defined as the number of edges traversed over without visiting a high-gain node. The generation of clusters can then follow the following steps: a. The next node in the queue is evaluated, if the node is a high-gain node, it is added to the current group (cluster) and the list of mobile devices expected (assigned) to explore it is saved. b. The neighbors of the current node are iterated through, skipping neighbors already in the queue. c. Any neighbor which is a high-gain node and not in the queue is recorded to be at zero depth and added to the queue. d. If the neighbor is a low-gain node, three checks are made. i. If its depth is above a threshold, it is skipped, ii. If the neighbor is already in the queue, operations check its depth and update it if it is lower than before, iii. If the neighbor is not in the cluster, it is added to the group (cluster). e. When the queue is empty the generation finishes, and the center of the group (cluster) is calculated as the average position of the group's nodes.

[0077] Groups are stored in one of two sets, the first is referred to as the set of own groups (assigned to that mobile device for exploration) while the other set is the set of other mobile device's groups for exploration.

[0078] Some operational embodiments for planning are further explained. If no contact has been made with any other mobile device, the operation sets the closest high-gain node as the next exploration goal. If the mobile device has sensed another mobile device, the exploration graph will contain nodes which it has not created itself. The nodes are then referred to as local and non-local. In this case, when the mobile device acquires new data, after communication finishes, the planning may follow the following steps:1. Update graph a. Non-local nodes are only removed from the graph if the area they cover is occupied, allowing nodes in unknown space to be kept. b. Only edges between local nodes are removed. A motivation for this operation can be that creation of edges when a non-local node is involved, may require creating an edge through unknown areas of the map. As unknown space may contain occupied space, creating edges through it is difficult to do safely.2. Adding new nodes to exploration graph. In one embodiment operations a. Check if another mobile device has explored the area by using a parameter rt, representing the range of influence of a non-local node. All nodes within range rtare evaluated. If any of the evaluated nodes are both non-local and explored, the area is to be considered explored by another mobile device, and the new node is not added to the graph. b. Check if the area will be explored by other mobile devices. Any new high-gain node that has an edge to another high-gain node, inherits which mobile devices have claimed the node. Meaning that all new high-gain nodes adjacent to a high-gain node claimed by a mobile device, are also considered to be claimed by that mobile device. If the nodes are claimed by the mobile device, they are added to the exploration goals, if not they are added to a set of unprocessed nodes. Unprocessed nodes are not considered as candidates for exploration until the next time the groups are updated. c. Any new high-gain node which is not claimed by any mobile device is added to the set of high-gain nodes used as exploration goals.3. Selecting the closest exploration goal. a. While the set of exploration goals is non-empty select the closest one to explore b. If it is empty, exploration is suspended, and the mobile devices return to its starting position. This behavior is cancelled if the mobile device beginsexploring upon an unexplored area or another mobile device and doing so results in it creating new candidate nodes. This operation allows the mobile device to meet up with other mobile devices which also have received commands to return to their starting position, assuming all mobile devices start off in the same area. If the mobile device manages to receive new candidate nodes, either through contact with other mobile devices, or finding new unexplored areas, exploration continues.4. Exploration finished a. If a mobile device is at its starting position, and it has no high-gain nodes in its exploration graph, including nodes claimed by other mobile devices, exploration is finished. b. Exploration may also be terminated if the mobile device is at its starting position and observes another mobile device which has finished its exploration.

[0079] Figure 7 illustrates components of mobile devices and components of a computing node which can operate in accordance with some embodiments.

[0080] Referring to Figure 7, operations of a first mobile device as described herein may be performed by components of the illustrated first mobile device 800a alone or in combination with a computing node 840. Similarly, operations of a second mobile device as described herein may be performed by components of the illustrated second mobile device 800b alone or in combination with a computing node 840. The second mobile device 800b and any further mobile devices may include components that correspond to those illustrated for the first mobile device 800a.

[0081] The first mobile device 800a includes at least one depth sensor 812 ("depth sensor), at least one processor 810 ("processor"), at least one memory 820 ("memory") storing program code executable by the processor 810, and a wireless transceiver 814 to communicate with a radio access network 830. The depth sensor 812 may include, without limitation, Lidar sensor, stereo cameras, monocular camera, radar, ultrasonic sensor, or other sensor that senses distance to surfaces. Memory 820 may include a map processing module 820 having instructions executable by the processor 810 to perform some or all of the operations of the map processing device according to one or more embodiments disclosed herein. Memory 820 may include an exploration map 824 that is updated as explained above and / or an exploration module 826 that controls movement of the mobile device 800a to explore anexploration space represented by the exploration map 824. The wireless transceiver 814 can communicate with the second mobile device 800b directly (e.g., cellular sidelink communications, WiFi direct, Bluetooth, etc.) or via the computing node 840 through the radio access network 830 and one or more networks 832, e.g., private or public (Internet) networks.

[0082] The computing node 840 includes at least one processor 842 ("processor"), at least one memory 850 ("memory") storing program code executable by the processor 842, and a network interface 860 to communicate with the first mobile device 800a and / or the second mobile device 800b. Memory 850 may include a map processing module 852 having instructions executable by the processor 842 to perform some or all of the operation of the map processing device according to one or more embodiments disclosed herein. Memory 850 may include an exploration map 854 that is updated as explained above and / or an exploration module 856 that controls movement of the first mobile device 800a and / or the second mobile device 800b to explore an exploration space represented by the exploration map 854.

[0083] Further definitions and embodiments are now explained below.

[0084] In the above description of various embodiments of present inventive concepts, it is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of present inventive concepts. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which present inventive concepts belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense expressly so defined herein.

[0085] When an element is referred to as being "connected", "coupled", "responsive", or variants thereof to another element, it can be directly connected, coupled, or responsive to the other element or intervening elements may be present. In contrast, when an element is referred to as being "directly connected", "directly coupled", "directly responsive", or variants thereof to another element, there are no intervening elements present. Like numbers refer to like elements throughout. Furthermore, "coupled", "connected", "responsive", or variants thereof as used herein may include wirelessly coupled, connected, or responsive. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Well-known functions or constructions maynot be described in detail for brevity and / or clarity. The term "and / or" includes any and all combinations of one or more of the associated listed items.

[0086] It will be understood that although the terms first, second, third, etc. may be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another element / operation. Thus, a first element / operation in some embodiments could be termed a second element / operation in other embodiments without departing from the teachings of present inventive concepts. The same reference numerals or the same reference designators denote the same or similar elements throughout the specification.

[0087] As used herein, the terms "comprise", "comprising", "comprises", "include", "including", "includes", "have", "has", "having", or variants thereof are open-ended, and include one or more stated features, integers, elements, steps, components or functions but does not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions or groups thereof. Furthermore, as used herein, the common abbreviation "e.g.", which derives from the Latin phrase "exempli gratia," may be used to introduce or specify a general example or examples of a previously mentioned item and is not intended to be limiting of such item. The common abbreviation "i.e.", which derives from the Latin phrase "id Est," may be used to specify a particular item from a more general recitation.

[0088] Example embodiments are described herein with reference to block diagrams and / or flowchart illustrations of computer-implemented methods, apparatus (systems and / or devices) and / or computer program products. It is understood that a block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by computer program instructions that are performed by one or more computer circuits. These computer program instructions may be provided to a processor circuit of a general purpose computer circuit, special purpose computer circuit, and / or other programmable data processing circuit to produce a machine, such that the instructions, which execute via the processor of the computer and / or other programmable data processing apparatus, transform and control transistors, values stored in memory locations, and other hardware components within such circuitry to implement the functions / acts specified in the block diagrams and / or flowchart block or blocks, and thereby create means (functionality) and / or structure for implementing the functions / acts specified in the block diagrams and / or flowchart block(s).

[0089] These computer program instructions may also be stored in a tangible computer- readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions which implement the functions / acts specified in the block diagrams and / or flowchart block or blocks. Accordingly, embodiments of present inventive concepts may be embodied in hardware and / or in software (including firmware, resident software, micro-code, etc.) that runs on a processor such as a digital signal processor, which may collectively be referred to as "circuitry," "a module" or variants thereof.

[0090] It should also be noted that in some alternate implementations, the functions / acts noted in the blocks may occur out of the order noted in the flowcharts. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality / acts involved. Moreover, the functionality of a given block of the flowcharts and / or block diagrams may be separated into multiple blocks and / or the functionality of two or more blocks of the flowcharts and / or block diagrams may be at least partially integrated. Finally, other blocks may be added / inserted between the blocks that are illustrated, and / or blocks / operations may be omitted without departing from the scope of inventive concepts. Moreover, although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that communication may occur in the opposite direction to the depicted arrows.

[0091] Many variations and modifications can be made to the embodiments without substantially departing from the principles of the present inventive concepts. All such variations and modifications are intended to be included herein within the scope of present inventive concepts. Accordingly, the above disclosed subject matter is to be considered illustrative, and not restrictive, and the appended examples of embodiments are intended to cover all such modifications, enhancements, and other embodiments, which fall within the spirit and scope of present inventive concepts. Thus, to the maximum extent allowed by law, the scope of present inventive concepts is to be determined by the broadest permissible interpretation of the present disclosure including the following examples of embodiments and their equivalents and shall not be restricted or limited by the foregoing detailed description.

Claims

CLAIMS:

1. A first mobile device comprising: a wireless transceiver operative to provide a communication connection with a second mobile device; a depth sensor configured to provide depth information comprising data points indicating locations of object features sensed by the depth sensor within an exploration space; and at least one processor operative to: obtain the depth information from the depth sensor, update a first exploration graph data structure used by the first mobile device to explore the exploration space, based on the depth information, wherein the first exploration graph data structure has nodes representing locations in the exploration space and edges representing paths that the first mobile device can be moved between the nodes, wherein for each of the nodes an information gain metric is assigned to the node based on an amount of new information about the exploration space which can be acquired by the depth sensor of the first mobile device exploring at the location represented by the node, and wherein for each of the edges a weight is assigned to the edge based on a distance the first mobile device would move on the path between the nodes, determine relative pose information indicating pose of the first mobile device relative to the second mobile device, send through the wireless transceiver to the second mobile device, a graph message comprising the relative pose information and the first exploration graph data structure containing an indication of a first group of assigned nodes which are assigned to the first mobile device for exploration, receive through the wireless transceiver from the second mobile device, a response graph message including a merged exploration graph data structure containing an indication of a third group of assigned nodes which are assigned to the first mobile device for exploration and an indication of a fourth group of assigned nodes which are assigned to the second mobile device for exploration,merge the merged exploration graph data structure with the first exploration graph data structure to generate a further merged exploration graph data structure, determine a next exploration goal prioritizing individual nodes in the third group of assigned nodes for exploration by the first mobile device based on relative distance from nodes of the fourth group of assigned nodes, and control movement of the first mobile device based on the next exploration goal to explore the exploration space.

2. The first mobile device of Claim 1, wherein the operation to update the first exploration graph data structure used by the first mobile device to explore the exploration space, comprises to: determine the information gain metric to be assigned to individual nodes in the first exploration graph data structure based on whether the location represented by the node has been explored by the depth sensor of the first mobile device and based on the weights assigned to the edges of the node, wherein a relatively lower value is assigned to any of the nodes in the first exploration graph data structure which correspond to locations that have been explored by the depth sensor of the first mobile device.

3. The first mobile device of any of Claims 1 and 2, wherein the at least one processor is further operative to: determine which of the first and second mobile devices is to operate in an observer role responsible for sending the graph message and which is to operate in an observed role responsible for receiving and processing the graph message, wherein the sending of the graph message is performed based on determining the first mobile device is to operate in the observer role and the second mobile device is to operate in the observed role.

4. The first mobile device of Claim 3, wherein the operation to determine which of the first and second mobile devices is to operate in an observer role responsible for sending the graph message and which is to operate in an observed role responsible for receiving and processing the graph message, comprises to: determine the first mobile device is to operate in the observer role and the second mobile device is to operate in the observed role, based on at least one of: the firstmobile device initiating establishment of the communication connection between the first and second mobile devices, sensing the second mobile device before being sensed by the second mobile device, having a depth sensor that is sensing location of the second mobile device, and operating to determine a coordinate transform to the second mobile device.

5. The first mobile device of any of Claims 1 and 4, wherein the graph message sent to the second mobile device comprises: a list of the nodes; a list of the edges connecting the nodes; the relative pose information indicating a pose of the first mobile device relative to the second mobile device; and an indication of the first group of assigned nodes in the first exploration graph data structure which is assigned to the first mobile device for exploration.

6. The first mobile device of Claim 5, wherein the graph message sent to the second mobile device further comprises: a further indication of a second group of assigned nodes in the first exploration graph data structure which is assigned to the second mobile device for exploration, wherein the modified second group of assigned nodes assigned to the second mobile device for exploration, is based on the second group of assigned nodes.

7. The first mobile device of any of Claims 5 to 6, wherein the graph message sent to the second mobile device further comprises: for each additional mobile device other than the first and second mobile devices which has a communication connection to at least one of the first and second mobile devices, a list mapping the additional mobile device to individual ones of the nodes in the first exploration graph data structure that the additional mobile device is assigned to explore.

8. The first mobile device of any of Claims 1 and 7, wherein the at least one processor is further operative to: transform the first exploration graph data structure from a first coordinate system used by the first mobile device for determining locations of object features sensed bythe depth sensor, to a second coordinate system used by the second mobile device for determining locations of object features sensed by a depth sensor of the second mobile device, using the relative pose information; and generate the graph message using the first exploration graph data structure after the transform to the second coordinate system is completed.

9. A second mobile device comprising: a wireless transceiver operative to provide a communication connection with a first mobile device; a depth sensor configured to provide depth information comprising data points indicating locations of object features sensed by the depth sensor within an exploration space; and at least one processor operative to: obtain the depth information from the depth sensor, update a second exploration graph data structure used by the second mobile device to explore the exploration space, based on the depth information, wherein the second exploration graph data structure has nodes representing locations in the exploration space and edges representing paths that the second mobile device can be moved between the nodes, wherein for each of the nodes an information gain metric is assigned to the node based on an amount of new information about the exploration space which can be acquired by the depth sensor of the second mobile device exploring at the location represented by the node, and wherein for each of the edges a weight is assigned to the edge based on a distance the second mobile device would move on the path between the nodes, receive through the wireless transceiver, a graph message comprising relative pose information indicating pose of the first mobile device relative to pose of the second mobile device, and comprising a first exploration graph data structure containing an indication of a first group of assigned nodes which are assigned to the first mobile device for exploration, merge the first exploration graph data structure with the second exploration graph data structure defining a second group of assigned nodes assigned to the second mobile device for exploration, to generate a merged exploration graph data structure,assign a third group of assigned nodes in the merged exploration graph data structure to the first mobile device for exploration, and a fourth group of assigned nodes in the merged exploration graph data structure to the second mobile device for exploration, send to the first mobile device a response graph message including the merged exploration graph data structure and indicating the third and fourth groups of assigned nodes, determine a next exploration goal prioritizing individual nodes in the fourth group of assigned nodes for exploration by the second mobile device based on relative distance from nodes of the third group of assigned nodes, and control movement of the second mobile device based on the next exploration goal to explore the exploration space.

10. The second mobile device of Claim 9, wherein the operation to update the second exploration graph data structure used by the second mobile device to explore the exploration space, comprises to: determine the information gain metric to be assigned to individual nodes in the second exploration graph data structure based on whether the location represented by the node has been explored by the depth sensor of the second mobile device and based on the weights assigned to the edges of the node, wherein a relatively lower value is assigned to any of the nodes in the second exploration graph data structure which correspond to locations that have been explored by the depth sensor of the second mobile device.

11. The second mobile device of any of Claims 9 to 10, wherein the operation to assign a third group of assigned nodes in the merged exploration graph data structure to the first mobile device for exploration, and a fourth group of assigned nodes in the merged exploration graph data structure to the second mobile device for exploration, comprises to: select high-gain nodes from among the nodes in the merged exploration graph data structure based on determining the high-gain nodes have assigned information gain metrics that satisfy a threshold value; and assign the high-gain nodes to the fourth group of assigned nodes in the merged exploration graph data structure for exploration by the second mobile device.

12. The second mobile device of Claim 11, wherein the operation to assign the high-gain nodes to the fourth group of assigned nodes in the merged exploration graph data structure for exploration by the second mobile device, further comprises to: exclude from selection as one of the high-gain nodes any node in the second group that overlaps in location with any of node in the first group and has a lower information gain metric than the overlapped node in the first group.

13. The second mobile device of any of Claims 9 and 12, wherein the at least one processor is further operative to: determine which of the first and second mobile devices is to operate in an observer role responsible for sending the graph message and which is to operate in an observed role responsible for receiving and processing the graph message, wherein the receiving of the graph message is performed based on determining the first mobile device is to operate in the observer role and the second mobile device is to operate in the observed role.

14. The second mobile device of Claim 13, wherein the operation to determine which of the first and second mobile devices is to operate in an observer mobile device responsible for sending the graph message and which is to operate in an observed mobile device responsible for receiving and processing the graph message, comprises to: select the observer mobile device based on it performing at least one of: initiating establishment of the communication connection between the first and second mobile devices, sensing the other mobile device before being sensed by the other mobile device, having a depth sensor that is sensing location of the other mobile device, and operating to determine a coordinate transform to the other mobile device.

15. The second mobile device of any of Claims 9 and 14, wherein the response graph message sent by the second mobile device comprises: a list of the nodes in the merged exploration graph data structure; a list of the edges connecting the nodes in the merged exploration graph data structure;an indication of which of the nodes in the merged exploration graph data structure are assigned to the first mobile device for exploration and which are assigned to the second mobile device for exploration.

16. The second mobile device of Claim 15, wherein the response graph message sent to the first mobile device, further comprises: for each additional mobile device other than the first and second mobile devices which has a communication connection to at least one of the first and second mobile devices, a list mapping the additional mobile device to individual ones of the nodes in the merged second exploration graph data structure that the additional mobile device is assigned to explore.

17. The second mobile device of any of Claims 9 and 16, wherein the at least one processor is further operative to: transform the merged exploration graph data structure from a second coordinate system used by the second mobile device for determining locations of object features sensed by the depth sensor, to a first coordinate system used by the first mobile device for determining locations of object features sensed by a depth sensor of the first mobile device, using the relative pose information; and generate the response graph message using the merged exploration graph data structure after the transform to the first coordinate system is completed.

18. The second mobile device of any of Claims 9 and 17, wherein operation to assign the third group of assigned nodes in the merged exploration graph data structure to the first mobile device for exploration, and the fourth group of assigned nodes in the merged exploration graph data structure to the second mobile device for exploration, comprises to: split a group of nodes in the merged second exploration graph data structure into the third group of assigned nodes which are assigned to the first mobile device for exploration and into the fourth group of assigned nodes which are assigned to the second mobile device for exploration, wherein the operation to split comprises to add to the third group the nodes in the merged exploration graph data structure that satisfy a closeness rule relative to the first mobile device, and add to the fourth group the nodes in the mergedexploration graph data structure that satisfy the closeness rule relative to the second mobile device.

19. A method by a first mobile device comprising: obtaining (500) from a depth sensor, depth information comprising data points indicating locations of object features sensed by the depth sensor within an exploration space, updating (502) a first exploration graph data structure used by the first mobile device to explore the exploration space, based on the depth information, wherein the first exploration graph data structure has nodes representing locations in the exploration space and edges representing paths that the first mobile device can be moved between the nodes, wherein for each of the nodes an information gain metric is assigned to the node based on an amount of new information about the exploration space which can be acquired by the depth sensor of the first mobile device exploring at the location represented by the node, and wherein for each of the edges a weight is assigned to the edge based on a distance the first mobile device would move on the path between the nodes, determining (504) relative pose information indicating pose of the first mobile device relative to a second mobile device, sending (506) through the wireless transceiver to the second mobile device, a graph message comprising the relative pose information and the first exploration graph data structure containing an indication of a first group of assigned nodes which are assigned to the first mobile device for exploration, receiving (508) through the wireless transceiver from the second mobile device, a response graph message including a merged exploration graph data structure containing an indication of a third group of assigned nodes which are assigned to the first mobile device for exploration and an indication of a fourth group of assigned nodes which are assigned to the second mobile device for exploration, merging (510) the merged exploration graph data structure with the first exploration graph data structure to generate a further merged exploration graph data structure, determining (512) a next exploration goal prioritizing individual nodes in the third group of assigned nodes for exploration by the first mobile device based on relative distance from nodes of the fourth group of assigned nodes, andcontrolling (514) movement of the first mobile device based on the next exploration goal to explore the exploration space.

20. The method of Claim 19, further comprising performing the operations of any of Claims 2 through 8.

22. A computer program product comprising a non-transitory computer readable medium storing instructions executable by at least one processor of a first mobile device to perform the operations of any of Claims 2 through 8.

21. A method by a second mobile device comprising: obtaining (600) from a depth sensor, depth information comprising data points indicating locations of object features sensed by the depth sensor within an exploration space; updating (602) a second exploration graph data structure used by the second mobile device to explore the exploration space, based on the depth information, wherein the second exploration graph data structure has nodes representing locations in the exploration space and edges representing paths that the second mobile device can be moved between the nodes, wherein for each of the nodes an information gain metric is assigned to the node based on an amount of new information about the exploration space which can be acquired by the depth sensor of the second mobile device exploring at the location represented by the node, and wherein for each of the edges a weight is assigned to the edge based on a distance the second mobile device would move on the path between the nodes, receiving (604) through the wireless transceiver, a graph message comprising relative pose information indicating pose of a first mobile device relative to pose of the second mobile device, and comprising a first exploration graph data structure containing an indication of a first group of assigned nodes which are assigned to the first mobile device for exploration, merging (606) the first exploration graph data structure with the second exploration graph data structure defining a second group of assigned nodes assigned to the second mobile device for exploration, to generate a merged exploration graph data structure,assigning (608) a third group of assigned nodes in the merged exploration graph data structure to the first mobile device for exploration, and a fourth group of assigned nodes in the merged exploration graph data structure to the second mobile device for exploration, sending (610) to the first mobile device a response graph message including the merged exploration graph data structure and indicating the third and fourth groups of assigned nodes, determining (612) a next exploration goal prioritizing individual nodes in the fourth group of assigned nodes for exploration by the second mobile device based on relative distance from nodes of the third group of assigned nodes, and controlling (614) movement of the second mobile device based on the next exploration goal to explore the exploration space.

22. The method of Claim 21, further comprising performing the operations of any of Claims 10 through 18.

23. A computer program product comprising a non-transitory computer readable medium storing instructions executable by at least one processor of a second mobile device to perform the operations of any of Claims 10 through 18.

Images