Positioning using sensors that can be transported with the equipment

Abstract

A device is configured to perform positioning using a set of sensors that can be transported with the device. The device includes at least one processor operatively connected to the set of sensors and at least one memory storing program code. The program code configures the at least one processor to determine a first set of device postures (PA) in which a first sensor satisfies a positioning performance rule, and to determine a second set of device postures (PB) in which a second sensor satisfies the positioning performance rule. The at least one processor is further configured to activate a second sensor based on a transition of the posture of the device from not being within a second set of device postures (PB) to being within the second set of device postures (PB) when the first sensor is active.

Description

Technical Field

[0001] The present disclosure relates to a device configured to perform positioning using a set of sensors that can be transported with the device, a method of performing positioning by the device using a set of sensors that can be transported with the device, and a corresponding computer program product. Background Art

[0002] Localization and mapping algorithms, such as simultaneous localization and mapping (SLAM) algorithms, are technologies that allow a device to localize itself in an environment while relying on onboard sensors such as cameras, range sensors, inertial sensors, etc. This is critical for robots to navigate and understand the environment to perform tasks, as well as being able to display real and persistent content in mixed reality devices.

[0003] For example, current mixed reality (MR) headsets and state-of-the-art smartphones contain RGB cameras, depth / 3D cameras (e.g., passive or active stereo, LIDAR, etc.), and inertial sensors (IMU), and this is also true for indoor and outdoor robotics (e.g., drones and autonomous vehicles). For example, Intel and Microvision recently introduced small “consumer” LIDAR products. However, other depth sensors or 3D sensors are very common in headsets and other devices such as robots.

[0004] Several SLAM algorithms have been proposed that rely on RGB and IMU sensors, depth sensors, or a combination of all of these sensors. One reason for using a combination of different types of sensors is to take advantage of the unique operating advantages of different sensors and improve their respective limitations. For example, RGB cameras perform poorly in dark or too bright environments because it is difficult or impossible to acquire visual features in such environments, while depth cameras (such as LIDAR or active stereo cameras) will perform well in such scenes. In addition, by measuring depth directly, positioning and mapping can be performed with higher accuracy, and a larger amount of environmental information can be captured (for example, building dense maps instead of sparse maps), in addition to other benefits. However, depth cameras generally have greater energy consumption and processing requirements, and may perform poorly under certain conditions. For example, depth cameras have a limited measurement range and may perform poorly in low-texture environments (passive stereo cameras) and areas with direct sunlight or IR interference (active stereo cameras and LIDAR), rain conditions (LIDAR), etc. Therefore, it is desirable to schedule the use of all available sensors to achieve accurate and robust positioning and mapping, while also reducing the energy resources of the device.

[0005] Current solutions assume that all sensors are always on (such as Microsoft HoloLens or any other AR headset with these multiple sensors). Summary of the invention

[0006] Some embodiments disclosed herein relate to a device configured to perform positioning using a set of sensors that can be transported with the device. The device includes: at least one processor that is operatively connected to the set of sensors; and at least one memory that stores program code. The program code configures the at least one processor to: determine a first set of device postures (PA) in which a first sensor satisfies a positioning performance rule; determine a second set of device postures (PB) in which a second sensor satisfies the positioning performance rule. The at least one processor is also configured to: when the first sensor is active, based on the posture of the device transitioning from not within the second set of device postures (PB) to within the second set of device postures (PB), activate the second sensor.

[0007] Some other related embodiments disclosed herein relate to a method for performing positioning by a device using a set of sensors transported with the device. The method includes: determining a first set of device postures (PA) in which a first sensor satisfies a positioning performance rule; and determining a second set of device postures (PB) in which a second sensor satisfies the positioning performance rule. The method activates the second sensor based on the posture of the device transitioning from not within the second set of device postures (PB) to within the second set of device postures (PB) when the first sensor is active.

[0008] Some related other embodiments disclosed herein relate to a computer program product for performing positioning of a device using a set of sensors transported with the device. The computer program product includes a non-transitory computer-readable medium storing program code that can be executed by at least one processor of the device. The program code configures the at least one processor to: determine a first set of device postures (PA) in which a first sensor satisfies a positioning performance rule; and determine a second set of device postures (PB) in which a second sensor satisfies the positioning performance rule. The program code also configures the at least one processor to: when the first sensor is active, based on the posture of the device transitioning from not within the second set of device postures (PB) to within the second set of device postures (PB), activate the second sensor.

[0009] A potential advantage that may be provided by these and other embodiments is that activation of a second sensor enables the device to deactivate the first sensor and then perform positioning using data from the second sensor rather than from the first sensor. This can result in a reduction in system power consumption and enable more optimized resource usage while continuing to meet sustained positioning performance requirements. Positioning performance rules can be configured based on the performance / robustness of positioning when sensors are activated / deactivated, which is a very relevant solution because sensors perform differently depending on environmental characteristics. Embodiments can support graceful switching between sensors to optimize resource usage while avoiding degradation of positioning operations.

[0010] After reading the following drawings and detailed description, other devices, methods and computer program products according to the embodiments will be apparent or become apparent to those skilled in the art. All of these devices, methods and computer program products are intended to be included in this specification, within the scope of the present disclosure and protected by the appended claims. In addition, all embodiments disclosed herein are intended to be implemented individually or combined in any manner and / or combination. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Various aspects of the present disclosure are illustrated by way of example and not limitation in the accompanying drawings. In the drawings:

[0012] Figure 1 shows a top view of a map showing movement of a device between different areas and associated sensor activation / deactivation operations according to some embodiments of the present disclosure;

[0013] Figure 2 A system diagram showing device components used to activate and deactivate sensors when performing positioning or performing positioning and mapping operations such as SLAM according to some embodiments of the present disclosure;

[0014] Figure 3 shows a top view of a map showing a vehicle moving through an example environment showing a transportation device and a set of sensors in accordance with some embodiments of the present disclosure;

[0015] Figure 4 A SLAM system is shown, comprising a pair of devices each connected to a respective set of sensors, and comprising a SLAM support node operating in accordance with some embodiments of the present disclosure;

[0016] Figure 5 A mixed reality (MR) system is shown that includes a MR head mounted device holding a mobile electronic device that may include or be operatively connected to a set of sensors and is configured to operate according to some embodiments of the present disclosure; and

[0017] Figures 6 to 16 A flow chart illustrating operations for controlling deactivation and activation of sensors used by a device for positioning, according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] The concept of the present invention will now be described more fully below with reference to the accompanying drawings, in which examples of embodiments of the concept of the present invention are shown. However, the concept of the present invention can be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided so that this disclosure will be thorough and complete, and the scope of various current inventive concepts will be fully communicated to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. Components from one embodiment may be assumed by default to be present in / used in another embodiment.

[0019] Figure 1 A top view of a map showing movement of a display device between different areas and associated sensor activation / deactivation operations according to some embodiments of the present disclosure is shown. The device is configured to perform positioning operations using a set of sensors, and may also perform combined positioning and map building operations, such as SLAM operations. The set of sensors includes a first sensor (sensor A) that is assumed to be active and a second sensor (sensor B) that is assumed to be inactive. In the illustrated embodiment, these operations determine the posture (e.g., position and / or orientation) of the device and identify the timing at which sensor B is to be activated, and also determine another posture (e.g., position and / or orientation) of the device and the time point at which sensor A is to be deactivated.

[0020] As used herein, the term "posture" refers to the position and / or orientation of a device (e.g., a mobile electronic device) relative to a defined coordinate system or relative to another device (e.g., a head-mounted device). Therefore, a posture can be defined based only on the multi-dimensional position of a device relative to another device or a defined coordinate system, based only on the multi-dimensional orientation of the device relative to another device or a defined coordinate system, or based on a combination of multi-dimensional position and multi-dimensional orientation. Therefore, the term "posture" is used to refer to position, orientation, or a combination thereof.

[0021] The device can be configured to determine a set of device poses (positions and / or orientations) in which corresponding sensors of sensors A and B can be used to satisfy the positioning performance rules during a transition from using sensor A to using sensor B. When the pose is determined, the pose can be used to generate guidance that can be provided to the user and / or the device to guide the movement of the device to obtain a pose that satisfies the positioning performance rules so as to allow a transition between sensors. Thus, sensor A is considered to satisfy the positioning performance rules when the data of sensor A processed by the positioning operation results in the device positioning accuracy satisfying the positioning performance rules, or when an estimate of the device positioning accuracy achieved using the data from sensor A will satisfy the positioning performance rules. Similarly, sensor B is considered to satisfy the positioning performance rules when the data of sensor B processed by the positioning operation results in the device positioning accuracy satisfying the positioning performance rules, or when an estimate of the device positioning accuracy achieved using the data from sensor B will satisfy the positioning performance rules.

[0022] In some embodiments, activation and / or deactivation of individual sensors is performed at a suitable device posture, at which posture there are sufficient available environmental features that the sensor can acquire to enable positioning operations to be performed on data from the sensor with sufficient accuracy to meet positioning performance rules. Performing sensor activation and / or deactivation in this manner enables positioning operations and / or combined positioning and mapping operations (e.g., SLAM) to be performed at reduced power consumption while providing a sufficiently high level of performance and robustness. The need to switch between sensors can be stimulated by a current or predicted decrease in positioning performance and / or combined positioning and mapping performance (e.g., SLAM performance). As will be further described below, the decision to switch from using sensor A to activating and using sensor B can be triggered based on a determination that sensor B will perform better than sensor A, a determination that sensor A consumes more operating power than sensor B, etc.

[0023] In some embodiments, the operation for activating the sensor may include triggering the sensor to transition to a high power state or triggering power on of the sensor. Similarly, the operation for deactivating the sensor may include triggering the sensor to transition to a low power state or triggering power off of the sensor. In some other embodiments, the operation for deactivating the sensor may include setting the data sampling rate of the sensor to a reduced sampling rate or zero, while the operation for activating the sensor may include setting the data sampling rate of the sensor to an increased sampling rate (sufficient for performing positioning operations). In some other embodiments, the operation for deactivating the sensor may include setting the resolution of the sensor to a reduced resolution or zero, while the operation for activating the sensor may include setting the resolution of the sensor to an increased resolution (sufficient for performing positioning operations).

[0024] Although various embodiments are described in the context of activating and deactivating individual sensors (e.g., sensor A and sensor B), these embodiments may be used to activate and deactivate a group of sensors. Thus, for example, sensor A may correspond to a group of sensors A, and similarly sensor B may correspond to a group of sensors B. Activation of sensor B and deactivation of sensor A are selectively performed when the device is located in a suitable position where a particular sensor characteristic is determined to be achievable, for example because when a switch between sensors occurs, sufficient available environmental features may be obtained from both sensors so that positioning can be performed with sufficient accuracy to satisfy positioning performance rules. By doing so, sensor A performs well until the moment sensor B is activated, and sensor B is able to perform well from that moment on, which enables a smooth transition from using two sensors for positioning to using only one sensor for positioning, in a manner that does not degrade or can degrade to an acceptable amount the performance and robustness of the positioning operation processing data from the sensors. By performing the transition at such a device location, energy consumption used by the sensors may be reduced because the amount of time that the two sensors must be active at the same time is reduced. Using this solution, it is possible that in case a switch from using sensor A to using sensor B would occur in a position where sensor B performs poorly, no time is wasted in trying to find a position where sensor B will start performing well immediately after being activated, which will also save energy.

[0025] Figure 1 A top view of a map showing movement of a device between different areas and associated sensor activation / deactivation operations is shown in accordance with some embodiments of the present disclosure.

[0026] refer to Figure 1 An illustrative example of , showing a geographic area PA where sensor A satisfies the positioning performance rules, showing another geographic area PB where sensor B satisfies the positioning performance rules, and showing an overlapping area PX where both sensors A and B satisfy the positioning performance rules.

[0027] When a device that uses data from sensors A and B to perform positioning operations is within area PA (where only sensor A meets the positioning performance rules), outside area PX, sensor A actively feeds data to the positioning algorithm or a combined positioning and mapping algorithm (e.g., a SLAM algorithm), and sensor B is deactivated. When the device moves from area PA to area PX (where both sensors A and B meet the positioning performance rules) along path segment PF_B, sensor B may be activated and sensor A may be deactivated based on the relative power consumption, positioning performance, combined positioning and mapping performance (e.g., SLAM performance), resource utilization, etc. of sensors A and B. In area PX, positioning may be performed using data from one or both of sensors A and B. When the device moves along path segment PF_A outside area PB and area PX, sensor B actively feeds data to the positioning algorithm or a combined positioning and mapping algorithm (e.g., a SLAM algorithm), and sensor A is deactivated.

[0028] As used herein, a positioning performance rule may include a set of conditions that may be the same or different for each sensor. For example, the conditions used to determine whether one sensor satisfies a positioning performance rule may be the same or different than the conditions evaluated to determine whether another sensor satisfies a positioning performance rule. The conditions may depend on the operating characteristics of the corresponding sensor, such as the operating characteristics of a visible light (RGB) camera, the operating characteristics of an infrared (IR) camera, the operating characteristics of a LIDAR sensor, etc.

[0029] Figure 3 A top view of a map showing the movement of a vehicle 300 transporting a device 400 is shown, and the device 400 uses a set of sensors 301 and 303 that can be transported with the device 400 to perform positioning. The device and sensors can be transported in other ways, such as by people, robots, aircraft (e.g., drones), ships, etc. The device 400 can be, but is not limited to, a component of any one of a smart phone, a wearable computer, an augmented reality headset, a virtual reality headset, a mixed reality headset, a semi-autonomous or autonomous vehicle, a drone, an aircraft, a robot, etc. Although only two sensors are shown, it will be understood that the device 400 is configured to perform positioning operations or combined positioning and mapping operations (e.g., SLAM) using any number of multiple sensors for feeding data of the appropriate type. Example types of sensors include, but are not limited to, RGB visible light cameras, infrared cameras, stereo cameras, inertial measurement units (IMUs), radar sensors, light detection and ranging (LIDAR) sensors, acoustic ranging sensors, proximity sensors, GPS, and RF transceivers (e.g., 5G radios).

[0030] refer to Figure 3, as the vehicle 300 travels through the environment along a path 309 (e.g., a road), the device 400 uses one or more of the sensors 301 and 303 to perform a positioning operation or a combined positioning and mapping operation (e.g., SLAM) to sense environmental features in the adjacent area. For example, in the first area 305, the first sensor 301 is active and the second sensor 303 is deactivated. The device 400 determines that the first sensor 301 satisfies the positioning performance rule, which results in the device keeping the first sensor 301 activated and the second sensor 303 deactivated while the vehicle 300 is still in the first area 305. When the data of the sensor 301 processed by the positioning algorithm results in the device positioning accuracy satisfying the positioning performance rule, the sensor 301 can be considered to satisfy the positioning performance rule. The vehicle 300 then travels into the second area 307, where the device 400 determines that the second sensor 303 can satisfy the positioning performance rule. When the estimate of the device positioning accuracy achieved using the data from the sensor 303 will satisfy the positioning performance rule, the sensor 303 can be considered to satisfy the positioning performance rule. Thus, device 400 can recognize a gesture, such as position and / or orientation, within region 307 (where positioning processing can be switched from using sensor 301 to using sensor 303). The gesture can correspond to a set of gesture sets corresponding to any position of device 400 within the overlapping portion of regions 305 and 307. Sensor 303 can be activated before switching, and sensor 301 can be deactivated after switching to avoid degradation or failure of positioning of device 400. The operation for switching positioning operation from using data from sensor 301 to using data from sensor 303 instead can include a period of time when positioning operation uses data from both sensors 301 and 303.

[0031] Because sensors inherently have specific operating constraints (e.g., light sensitivity), different types of sensors can provide different levels of positioning performance when affected by different geographic area characteristics. Device 400 is configured to switch between these sensors based on the corresponding positioning performance of different types of sensors as a function of different characteristics of the geographic area through which device 400 moves. Device 400 can also be configured to control switching based on how much energy different sensors consume and / or how many other system resources (e.g., processing, memory, communication bandwidth, etc.) are used to receive, store, and / or process data from sensors through algorithms. A decision to activate and deactivate sensors and switch from one sensor to another for positioning operations can be performed based on a combination of positioning performance achievable using data from insight (perspective), and based on how much energy or other resources the corresponding sensors use. In addition, the switch from using one sensor to using another sensor can be delayed until the device reaches such a posture: wherein the positioning operation can be performed without reducing the accuracy of the positioning operation or the accuracy of the positioning operation does not appear unacceptably reduced, such as to prevent the positioning operation from failing during switching between sensors and causing the tracking loss of the device posture.

[0032] Reference again Figure 1 and 2 In some embodiments, the device is configured to perform the following operations:

[0033] 1. Detect the need to deactivate sensor A and activate sensor B;

[0034] 2. Detect the current device posture P (when only sensor A is active);

[0035] 3. Determine the device pose PA with the desired SLAM performance for sensor A;

[0036] 4. Determine the device pose PB with the desired SLAM performance for sensor B;

[0037] 5. Determine device pose PF_B (which is the pose at which sensor B should be activated) and device pose PF_A (which is the pose at which sensor A should be deactivated), wherein when processing data from both sensors A and B, both poses PF_B and PF_A are expected to provide the desired SLAM performance; and

[0038] 6. Guiding the device to reach the determined area / pose (e.g., displaying information to guide the user, or controlling the movement of the device, such as when the device is transported by an autonomous or semi-autonomous vehicle).

[0039] With reference to the third to fifth elements of the above-mentioned operations, in some embodiments, other operations are defined for controlling the activation and deactivation of sensors transported with the device performing the positioning operation to perform combined positioning and mapping operations (eg, SLAM operations).

[0040] refer to Figure 6 In an operational embodiment of the present invention, a device is configured to perform operations including determining 601 a first set of device postures (PA) in which a first sensor satisfies a positioning performance rule. The operations also determine 603 a second set of device postures (PB) in which a second sensor satisfies a positioning performance rule. The operations responsively activate 605 the second sensor based on a transition of a posture of the second sensor device from not within the second set of device postures (PB) to within the second set of device postures (PB).

[0041] In other optional embodiments, these operations deactivate 607 the first sensor based on the posture of the first sensor device transitioning from within the first set of device postures (PA) to not within the first set of device postures (PA) while the posture of the device remains within the second set of device postures (PB).

[0042] refer to Figure 7 Other operation embodiments of the present invention may guide the movement of the device 701 so that the posture of the second sensor is changed from not within the second set of device postures (PB) to within the second set of device postures (PB). The operation for guiding the movement of the device 701 may include: providing information (such as displaying information, generating audible information, generating tactile feedback, etc.) through the user interface of the device to guide the user on how to move the device so that the posture of the device is changed from not within the second set of device postures (PB) to within the second set of device postures (PB). Alternatively or additionally, the operation for guiding the movement of the device 701 may include: providing guidance information for controlling the movement of the device and sensors (such as an autonomous vehicle or a semi-autonomous vehicle) to the navigation system of the device so that the posture of the device is changed from not within the second set of device postures (PB) to within the second set of device postures (PB).

[0043] Figure 2A system diagram of a device component used to activate and deactivate a sensor when performing positioning or performing positioning and map building operations such as SLAM according to some embodiments of the present disclosure is shown. Based on the previous performance (e.g., positioning performance, combined positioning and map building performance, SLAM performance) and the associated device posture obtained from component 210, the performance of both sensors A and B is analyzed (component 207), and the analyzed performance and the associated device posture are stored in the map (component 211). The analyzed performance is then used to determine (component 215) the desired postures PF_A and PF_B in which sensors A and B should be deactivated and activated, respectively. This information is provided to a sensor selector block (component 209), which determines which sensor should be selected to provide sensor information to a SLAM algorithm (component 213). Instructions or actuations (component 217) are provided to a user or device as guidance for how to move the device to obtain postures PF_A and PF_B.

[0044] A variety of different operations can be used to determine the positioning performance or positioning and mapping (e.g., SLAM) performance of each sensor A and B. This information can be determined online (e.g., to perform real-time tracking when sensors perform poorly). In some embodiments, the device determines a first set of device poses in which sensor A meets positioning performance rules, determines a second set of device poses in which sensor B meets positioning performance rules, and activates sensor B based on the device's pose transitioning from not within the second set of device poses to within the second set of device poses when sensor A is active. In this way, the device can control the activation of sensor B and the timing of when the device switches from using data from sensor A to using data from sensor B to perform positioning based on the device pose being achieved, which will provide acceptable positioning performance.

[0045] In some embodiments, localization performance or localization and mapping (e.g., SLAM) performance can be quantified by one of many factors, including, but not limited to, the number of false relocalizations, the number and quality of features in the area, localization error (position and orientation of the device), position / orientation drift, and other metrics. A false relocalization can correspond to an error associated with the device locating itself in the wrong part of the map (which is not where the device is). A localization error can correspond to a difference between an estimated pose of the device and an actual pose of the device.

[0046] False relocalization and localization errors can typically be characterized a posteriori with greater certainty and / or accuracy after a SLAM session is completed (e.g., when a map is further optimized and multiple devices / agents have contributed information to improve the map). At this point, sensor data from previous passes in the area can be fed again to the SLAM algorithm with the optimized map, comparisons can be made between the new and old trajectories, and errors can be calculated. Additionally, if known features (landmarks) are available in the area with known ground truth poses (e.g., as fiducial markers, specific objects, or environmental features, such as doors, windows, etc.), localization errors can be calculated against these landmarks.

[0047] refer to Figure 6 In an operational embodiment of the present invention, determining 601 a first set of device poses (PA) in which a first sensor satisfies a positioning performance rule may include determining a positioning performance score of the first sensor at one or more poses based on at least one of: a determined number of erroneous repositionings determined based on data from the first sensor, a number and quality of feature measurements determined in an area measured by data from the first sensor, a positioning error determined based on data from the first sensor, and an amount of uncertainty in measurements based on data from the first sensor. Alternatively or additionally, determining 601 may include determining whether the positioning performance score of the first sensor at one or more poses satisfies the positioning performance rule. Similarly, determining 603 a second set of device poses (PB) in which a second sensor satisfies a positioning performance rule may include determining a positioning performance score of the second sensor at one or more poses based on at least one of: a determined number of erroneous repositionings determined based on data from the second sensor, a number and quality of features determined in an area measured by data from the second sensor, and a positioning error determined based on data from the second sensor. Determining 603 whether the second sensor satisfies the positioning performance rule may include determining whether the positioning performance score of the second sensor at one or more poses satisfies the positioning performance rule.

[0048] In some embodiments, information collected to quantify positioning performance or positioning and mapping (e.g., SLAM) performance is collected and stored in a positioning map, which indicates which performance information is obtained using various sensors for the postures or areas defined in the map. This information can be collected by the current device when the sensors are respectively active, and can be collected from other devices that have previously been at these locations and have the same type of sensors or other types of sensors that are actively used for positioning. The map can store information identifying the posture associated with the performance data of different sensors. Performance data can be obtained from the map for a specific sensor identifier (unique to each sensor) based on the type of sensor (e.g., monocular camera, stereo camera, LIDAR, etc.) and / or based on one or more operating characteristics of the sensor (e.g., camera resolution, camera sensitivity, etc.). Therefore, for example, a device can query the map using a specific sensor identifier, using the sensor type, or using one or more operating characteristics of the sensor to determine from the information in the map a device posture or a set of device postures that the sensor will meet the positioning performance rules (i.e., the output data processed by the positioning algorithm will provide a device positioning that meets the positioning performance rules).

[0049] refer to Fig. 9 In an operational embodiment of the present invention, determining 601 a first set of device postures (PA) in which a first sensor satisfies a positioning performance rule includes determining 901 the positioning of the first sensor relative to a map defined by a data structure identifying previously sensed features of a local environment (i.e., environmental features previously sensed by the first sensor, another sensor operationally similar to the first sensor, and / or another type of sensor). Determining 601 also includes determining 903 a positioning performance score of the first sensor when the device is in one or more postures based on whether the first sensor senses the previously sensed features identified by the data structure. Determining 601 also includes determining 905 whether the positioning performance score of the first sensor satisfies the positioning performance rule when the device is in the one or more postures.

[0050] refer to Fig.10 In an operational embodiment of the invention, determining 603 a second set of device postures (PB) in which the second sensor satisfies the positioning performance rule includes determining 1001 a positioning performance score of the second sensor when the device is in one or more postures based on whether the second sensor senses a previously sensed feature identified by the data structure. Determining 603 also includes determining 1003 whether the positioning performance score of the second sensor satisfies the positioning performance rule when the device is in one or more postures based on whether the second sensor senses a previously sensed feature identified by the data structure.

[0051] refer to Fig.11In an operational embodiment of the present invention, determining 601 a first set of device postures (PA) in which a first sensor satisfies a positioning performance rule includes determining 1101 a positioning of the first sensor relative to a map defined by a data structure identifying locations of features of a local environment (which may have been defined by a user in the data structure, previously sensed and added to the data structure, or input to the data structure by any other process). Determining 601 also includes determining 1103 a positioning performance score of the first sensor when the device is in one or more postures based on a comparison of the locations where the first sensor identifies the features to the locations where the data structure identifies the features. Determining 601 also includes determining 1105 whether the positioning performance score of the first sensor satisfies the positioning performance rule when the device is in the one or more postures.

[0052] refer to Fig.12 In an operational embodiment of the invention, determining 603 a second set of device postures (PB) in which the second sensor satisfies the positioning performance rule includes determining 1201 a positioning performance score of the second sensor when the device is in one or more postures based on a comparison of the location where the second sensor identifies the feature and the location where the data structure identifies the feature. Determining 603 also includes determining 1203 whether the positioning performance score of the second sensor satisfies the positioning performance rule when the device is in the one or more postures.

[0053] refer to Fig.13 In an operational embodiment of the present invention, determining 601 a first set of device postures (PA) in which a first sensor satisfies a positioning performance rule includes determining 1301 to locate the device at locations within a map defined by a data structure, the data structure identifying positioning performance scores of the device at these locations. Determining 601 also includes obtaining 1303 a positioning performance score of the first sensor from the data structure based on locating the device at these locations. Determining 601 also includes determining 1305 whether the positioning performance score of the first sensor satisfies the positioning performance rule.

[0054] refer to Fig.14 In an operational embodiment of the present invention, determining 603 a second set of device postures (PB) in which the second sensor satisfies the positioning performance rule includes obtaining 1401 a positioning performance score of the second sensor from the data structure based on positioning the device to these positions. Determining 603 also includes determining 1403 whether the positioning performance score of the second sensor satisfies the positioning performance rule.

[0055] In some embodiments, based on accessing a file stored in a networked data repository (e.g. Figure 4 The data structure of the map in the map repository 431 in the SLAM support node 427 is used to determine the positioning performance scores of the first and second sensors.

[0056] In some embodiments, the positioning performance scores of the first sensor and the second sensor are determined based on accessing a data structure of the map repository 402 stored in the memory 404 of the device 400 .

[0057] Some embodiments involve detecting the need to activate one sensor and deactivate another sensor, which may be motivated by one or both of the following embodiments.

[0058] In a first embodiment, a current or predicted degradation in localization performance when using sensor A is determined, where it is believed that sensor B will perform better. For example, if sensor A is a monocular camera and sensor B is a LIDAR, the current or predicted area may contain few visual features that will be detected by sensor A, while sensor B may track the structure of the environment well. As another example, based on the recorded information, it may be known that the lighting conditions or visual texture of the environment in different parts of a building are different, which seriously affects the localization performance on previous occasions. This information can be extracted from a map constructed using the localization performance based on sensors A and B.

[0059] refer to Fig.15 In an operational embodiment, these operations may also include initiating 1501 activation of a second sensor (sensor B) based on determining at least one of a currently measured decrease in positioning performance of the first sensor (sensor A) and a predicted decrease in positioning performance of the first sensor (sensor A).

[0060] In the second embodiment, sensor A consumes more power when operating than sensor B, and therefore the device operating using sensor A is less energy efficient. Fig.16 In an embodiment of the operation of the present invention, the operations may also include initiating 1601 activation of a second sensor (sensor B) based on determining that the first sensor (sensor A) uses more than a threshold amount of defined hardware resources. The defined hardware resources may include at least one of power consumption, processor utilization, memory utilization, and network communication utilization.

[0061] Some embodiments involve determining device postures PA and PB, which may include the operations discussed below.

[0062] In one embodiment, given the currently available map (which contains information about the positioning performance of sensor A and sensor B for each device pose as described above), a device pose Z is searched for which the positioning performance using sensor X is above a certain threshold. The pose set Z may contain all possible poses of the device, or may be optimized for the most suitable pose (e.g., not considering poses where the device is pointing to the floor or ceiling, etc.).

[0063] In another embodiment, for a given pose set Z, a search in the map is performed within an area of ​​a maximum of Y meters from the current position of the device. The value of Y can be defined based on parameters such as available energy, a deadline to perform sensor transitions, the current and predicted trajectory of the device, etc.

[0064] In some embodiments, Figure 4 , 7 As shown in FIG. 8 , the method for performing positioning using a set of sensors 401, 403, and 405 that can be transported with the device 400 further includes searching 801 among the second set of device poses (PB) to identify a subset of at least one device pose of the second sensor (sensor B) in which the positioning performance of the second sensor (sensor B) satisfies the positioning performance rule. Directing 701 the movement of the device to guide the movement of the second sensor (sensor B) toward at least one device pose in the subset.

[0065] In some embodiments, searching 801 among the second set of device poses (PB) can be constrained to identify only poses within the second set of device poses (PB) that are within a threshold distance from the current position of the device. The threshold distance can be determined based on at least one of the following: remaining battery power in the device, remaining time before a deadline to perform a sensor transition occurs, and an amount of deflection from the current movement trajectory of the device that would be required for the device to be moved to a device pose in the second set of device poses (PB).

[0066] In some embodiments, the operation for searching 801 among the second set of device poses (PB) may be constrained to identify only poses among the second set of device poses (PB) that are within a threshold proximity to the current pose of the second sensor.

[0067] In some embodiments, an operation for searching 801 among the second set of device poses (PB) may be performed to identify a pose among the second set of device poses (PB) where both the first sensor and the second sensor will have SLAM performance that satisfies the SLAM performance rule.

[0068] Further references Figure 7 In some embodiments, the operation for guiding the movement of the device 701 so that the device's posture transitions from not within the second set of device postures (PB) to within the second set of device postures (PB) may include identifying a device posture sequence from the second set of device postures (PB), and when performing the operation for activating the second sensor and deactivating the first sensor, guiding the movement of the device so that the device transitions through the device posture sequence.

[0069] Some other embodiments involve determining device posture PF_B (which is a posture for which sensor B should be activated) and determining device posture PF_A (which is a posture for which sensor A should be deactivated).

[0070] In one embodiment, based on the posture sets PA and PB, a posture set PX included in the sets PA and PB is determined, which is the intersection of the two sets, for example Figure 3 The overlapping part of regions 305 and 307 in . Then, the final posture PF can be determined in various ways, for example, PF can be the posture closest to the current device posture P, or the posture that maximizes the positioning performance of both sensors A and B, or a combination of the two.

[0071] In another embodiment, the posture PF may also be a continuous set of postures (i.e., trajectory) for the device to follow, which may include a posture set PF_A (where sensor A performs well), followed by a posture set PF_AB (where the positioning performance of both sensors is above a certain threshold), followed by a posture set PF_B (where only sensor B performs well).

[0072] Based on this solution, when there is a transition between using sensor A and sensor B, the performance will drop smoothly and the situation where a forced switch between one sensor and another has to be performed (where it will be unknown whether the new sensor will perform well) will be avoided. Sensor transitions can then be performed in a planned manner to ensure that a certain positioning performance is expected to occur.

[0073] In one embodiment, the current map is enhanced in the desired areas / poses so that the positioning performance (e.g., SLAM performance) in these desired areas is above a defined threshold. For example, considering that sensor A is RGB+IMU and sensor B is a depth sensor or 3D sensor (e.g., LiDAR or stereo camera), this enhancement can be to densify the RGB+IMU data to obtain better depth features in a specific area so that the depth sensor or 3D sensor can perform better positioning in such an area when it is turned on. Conversely, the enhancement can be to download additional keyframes to the device so that the RGB+IMU sensor can perform better when it is turned on.

[0074] In one embodiment, information retrieved from a map to calculate the performance of a given sensor in a given area is weighted based on the temporal nature of the information. For example, more trust is placed on data that was captured recently. Additionally, more trust may be placed on data captured at the same time of day so that lighting conditions are likely to be the same.

[0075] In one embodiment, if both sensors are active during the transition, information fusion between the two sensors is performed to enable higher positioning performance.

[0076] Figure 4 A system 450 is shown that includes a pair of devices 400 and 410, each of which is connected to a corresponding set of sensors and operates according to some embodiments of the present disclosure. The example system 450 may also include a SLAM support node 427. Although the context of activating and deactivating sensors during SLAM processing is described, Figure 4 Systems, however, these embodiments are not limited to use with SLAM, but may more generally be used to activate and deactivate sensors during positioning operations or during combined positioning and mapping operations (which may or may not be based on SLAM).

[0077] The first device 400 may include or be operatively connected to a first set of sensors 401, 403, and 405 that can be transported with the first device 400. The second device 410 may include or be operatively connected to a second set of sensors 411, 413, and 415 that can be transported with the second device 410. In some embodiments, the first device 400 determines 601 ( Figure 6 ) wherein a first sensor (eg, sensor 401) satisfies a first set of device postures (PA) of positioning performance rules, and determining 603 ( Figure 6 ) wherein a second sensor (eg, sensor 403) satisfies a second set of device postures (PB) of the positioning performance rules.

[0078] In some embodiments, the first device 400 may be assisted in performing determinations 601 and / or 603 based on information obtained from the second device 410 and / or from the SLAM support node 427. For example, when the first device 400 is in the vicinity of the second device 410, the first device 400 may operationally assume that it can successfully use sensors of the same type or sufficiently similar types as the sensors that the second device 410 is actively using to perform positioning for positioning. Thus, the first device 400 may be configured to query the second device 410 to obtain a list of sensors (e.g., one or more sensors) that the second device 410 is actively using to perform positioning, and then determine from the list which sensors (if any) in its first group 401, 403, and 405 can be used to perform positioning while satisfying the positioning performance rules.

[0079] In one illustrative other example, assume that first device 400 is located near second device 410, assume that first set of sensors 401, 403, and 405 are respectively identical to (or respectively satisfy a threshold level of operational similarity with) second set of sensors 411, 413, and 415, and assume that second device 410 reports that it is actively using sensor 411 and is not using sensors 413 and 415 for positioning. First device 400 can then accordingly determine that it is currently in a posture where sensor 401 will satisfy the positioning performance rule (e.g., based on devices 400 and 410 being located adjacently and sensor 411 being actively used by second device 410), and also determine that sensors 403 and 405 will not satisfy the positioning performance rule.

[0080] In some other embodiments, data from one or more sensors in the second group of sensors 411, 413, and 415 transported with the second device 410 can be used to assist the first device 400 in performing positioning. For example, when the first device 400 is located near the second device 410 and is also communicatively connected to the second device 410 via a communication channel of sufficient bandwidth, the first device 400 can query the second device 410 to obtain a list of sensors (e.g., one or more sensors) that the second device 410 is actively using to perform positioning, and then determine from the list which sensors (if any) in the second group 411, 413, and 415 can be used by the first device 400 to perform positioning while satisfying the positioning performance rules. The first device 410 can use the received list to decide to switch from using one or more sensors in the first group 401, 403, and 405 to using data from one or more sensors in the second group 411, 413, and 415 for positioning of the first device 400. In other embodiments, the first device 400 may request the second device 410 to activate a sensor in the second group 411, 413 and 415, and also provide data from the activated sensor to the first device 400 based on the first device 400 determining that, for example, none of the sensors in the first group 401, 403 and 405 meet the positioning performance rules for the current posture of the first device 400.

[0081] The first device 400 and the second device 410 may be, but are not limited to, components of any of a smartphone, a wearable computer, an augmented reality headset, a virtual reality headset, a mixed reality headset, a semi-autonomous or autonomous vehicle, an aircraft, a robot, a ship, etc. The first device 400 and the second device 410 and their connected sensors may be transported in any manner, such as by a person, a vehicle, a drone, an aircraft, a ship, a robot, etc. Example types of sensors include, but are not limited to, an RGB visible light camera, an infrared camera, an inertial measurement unit (IMU), a radar sensor, a stereo camera (visible light and / or infrared), a light detection and ranging (LIDAR) sensor, an acoustic ranging sensor, a proximity sensor, a GPS, and an RF transceiver (e.g., a 5G radio).

[0082] The first device 400 includes a SLAM (or positioning) processor 409, a memory 404, and a wireless transceiver 407, which can communicate with a second device 410 and / or a SLAM (or positioning) support node 427 via a radio access node (RAN) 421 or 423. The processor 409 is operatively connected to the first set of sensors 401, 403, 405. The memory 404 stores program code executed by the processor 409 to perform operations, and can store a map repository 402. The operations performed by the SLAM processor 409 may include some or all of the operations described herein involving controlling sensor activation and / or deactivation, and more specifically, including the operations described herein in Figure 6-16 The operations described in any one or more of the contexts.

[0083] The second device 410 can be configured similarly to the first device 400. The second device 410 includes a set of sensors 411, 413, and 415, a SLAM (or positioning) processor 419, a wireless transceiver 417, and a memory 414 that stores program code executed by the processor 419 to perform operations and can also store a map repository 412. The processor is operatively connected to the set of sensors 411, 413, and 415.

[0084] The first device 400 and / or the second device 410 may be configured to include program code and associated circuits required to perform operations to control sensor activation and deactivation and to perform positioning operations, and may also perform map building operations (e.g., SLAM algorithm processing). Some positioning and / or map building operations of the first device 400 may be performed by a networked node (e.g., the SLAM support node 427 shown) and / or by the second device 410. Similarly, some positioning and / or map building operations of the second device 410 may be performed by a networked node (e.g., the SLAM support node 427 shown) and / or by the first device 400.

[0085] The devices 400 and 410 may be configured to communicate with a SLAM support node 427 via one or more of the radio access networks (RANs) 421 and 423 and the network 425. The SLAM support node 427 includes a processor 433, which may be configured to perform some or all operations for activating and deactivating sensors transported with the first device 400 and / or the second device 410 according to one or more embodiments disclosed herein. The SLAM support node 427 may store a map repository 431, which may be used to determine which sensors to activate or deactivate and when to activate or deactivate sensors based on the posture of the first device 400 and / or the second device 410.

[0086] Figure 5 An MR system is shown that includes a mixed reality (MR) head mounted device 500 holding a mobile electronic device 520, which may include a (e.g., built-in) camera 522 and is operatively connected to a set of sensors 533, 535, etc. The mobile electronic device 520 may include a processor and a memory storing program code, which configures the processor to activate and deactivate individual sensors of the sensors 522, 533, 535, while using active sensors of the sensors 522, 533, 535 to perform positioning. The MR head mounted device 500 includes a lens 510, through which a user wearing the MR head mounted device can view real world features. The MR head mounted device 500 also includes a bracket 521, which is configured to releasably hold the mobile electronic device 520 in a defined orientation relative to the lens 510.

[0087] The mobile electronic device 520 includes a display device and a processor. The camera 522 is configured to output digital pictures (e.g., still pictures and / or videos) and is arranged by the bracket 521 to view at least a portion of the lens 510 of the MR head-mounted device 500. The display device is arranged to display information projected on the lens 510 for direct or indirect reflection toward the user's eyes (i.e., when the MR head-mounted device 500 is being worn) and the camera 522 of the mobile electronic device 520. Although not shown, the head-mounted device may include an intervening mirror located between the lens 510 and the user's eyes and / or the camera 522, and thus light may be reflected directly or indirectly toward the user's eyes and / or the camera 522.

[0088] Mobile electronic device 520 may include, but is not limited to, a smart phone, a PDA, a tablet computer, a gaming device, or other computing device. For the sake of brevity, a "mobile electronic device" is also referred to herein as a "mobile device" and a "device".

[0089] Other definitions and examples:

[0090] In the above description of various embodiments of the present invention, it should be understood that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those commonly understood by those of ordinary skill in the art to which the present invention belongs. It will also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of this specification and related fields, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0091] When an element is referred to as "connected", "coupled" to another element or "responsive to" (or its variant) another element, it can be directly connected, coupled to another element or responsive to another element, or there can be an intermediate element. In contrast, when an element is referred to as "directly connected", "directly coupled" to another element or "directly responsive to" (or its variant) another element, there is no intermediate element. The same number always refers to the same element. In addition, "coupling", "connection", "responsive to" or its variants used herein may include wireless coupling, connection or response. As used herein, the singular "one", "an" and "the" are intended to also include plural forms, unless the context clearly states otherwise. For brevity and / or clarity, known functions or structures may not be described in detail. The term "and / or" includes any and all combinations of one or more associated listed items.

[0092] It should be understood that although the terms first, second, third, etc. can be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another element / operation. Therefore, without departing from the teaching of the present invention, the first element / operation in some embodiments may be referred to as the second element / operation in other embodiments. Throughout the specification, the same reference numerals or the same figure indicators represent the same or similar elements.

[0093] As used herein, the terms "comprises," "including," "having," or variations thereof, are open ended and include one or more stated features, integers, elements, steps, components, or functions, but do not exclude the presence or addition of one or more other features, integers, elements, steps, components, functions, or combinations thereof. In addition, as used herein, the general abbreviation "eg," derived from the Latin phrase "exempli gratia," may be used to introduce or specify one or more general examples of previously mentioned items, and is not intended to limit such items. The general abbreviation "ie," derived from the Latin phrase "id est," may be used to specify a specific item from a more general narrative.

[0094] Example embodiments are described herein with reference to block diagrams and / or flowchart illustrations of computer-implemented methods, devices (systems and / or equipment), and / or computer program products. It should be understood that a block diagram and / or flowchart illustration of a frame and a combination of frames in the block diagram and / or flowchart illustration can be implemented by computer program instructions executed by one or more computer circuits. These computer program instructions can be provided to a processor circuit of a general-purpose computer circuit, a special-purpose computer circuit, and / or other programmable data processing circuit to produce a machine so that instructions executed by a processor of a computer and / or other programmable data processing device transform and control transistors, values ​​stored in storage cells, and other hardware components within such circuits to implement the functions / actions specified in the block diagram and / or one or more flowchart frames, thereby creating a device (function) and / or structure for implementing the functions / actions specified in the block diagram and / or one or more flowchart frames.

[0095] These computer program instructions may also be stored in a tangible computer-readable medium that can direct a computer or other programmable data processing device to work in a specific manner so that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions / actions specified in the block diagram and / or one or more flowchart blocks. Therefore, embodiments of the present inventive concept may be embodied in hardware and / or software (including firmware, resident software, microcode, etc.) running on a processor such as a digital signal processor, which may be collectively referred to as "circuits", "modules" or variations thereof.

[0096] It should also be noted that in some alternative implementations, the function / action mentioned in the frame may not occur in the order mentioned in the flow chart. For example, two frames shown in succession can actually be performed substantially simultaneously, or these frames can sometimes be performed in reverse order, depending on the function / action involved. In addition, the function of a given frame of a flow chart and / or block diagram can be divided into a plurality of frames and / or the function of two or more frames of a flow chart and / or block diagram can be integrated at least in part. Finally, other frames can be added / inserted between the illustrated frames, and / or frames / operations can be omitted without departing from the scope of the inventive concept. In addition, although some figures include arrows on the communication path to illustrate the main direction of communication, it should be understood that communication can occur in the direction opposite to the arrows shown.

[0097] Without departing substantially from the principles of the inventive concept, many changes and modifications may be made to the embodiments. All of these changes and modifications are intended to be included within the scope of the inventive concept. Therefore, the subject matter disclosed above should be considered to be illustrative rather than restrictive, and the attached examples of the embodiments are intended to cover all such modifications, enhancements, and other embodiments that fall within the spirit and scope of the inventive concept. Therefore, to the maximum extent permitted by law, the scope of the inventive concept will be determined by the broadest permitted interpretation of the present disclosure including the following examples of the embodiments and their equivalents, and should not be bound or limited by the aforementioned detailed description.

Claims

1. A device (400) configured to perform positioning using a set of sensors (401, 403, 405) transportable with the device (400), the set of sensors comprising a first sensor and a second sensor, the device (400) comprising: at least one processor (409) operatively connected to the set of sensors (401, 403, 405); at least one memory (404) storing program code, the program code configuring the at least one processor to: determining (601) a first set of device postures (PA) in which the first sensor satisfies positioning performance rules for the first sensor; determining (603) a second set of device poses (PB) in which the second sensor satisfies positioning performance rules for the second sensor; directing (701) movement of the device to transition a posture of the device from being not within the second set of device postures (PB) to being within the second set of device postures (PB); as well as The second sensor is activated (605) based on the posture of the device transitioning from not within the second set of device postures (PB) to within the second set of device postures (PB) while the first sensor is active.

2. The device according to claim 1, wherein The program code also configures the at least one processor to deactivate (607) the first sensor based on the posture of the device transitioning from within the first set of device postures (PA) to not within the first set of device postures (PA) while the posture of the device remains within the second set of device postures (PB).

3. The device according to any one of claims 1 to 2, wherein: Directing (701) movement of the device includes providing, through a user interface of the device, information guiding movement of the device to transition a posture of the device from not within the second set of device postures (PB) to within the second set of device postures (PB).

4. The device according to any one of claims 1 to 2, wherein: Directing (701) movement of the device includes providing guidance information to a navigation system of the device that controls movement of the device to transition a posture of the device from not within the second set of device postures (PB) to within the second set of device postures (PB).

5. The device according to any one of claims 1 to 2, further comprising: searching (801) among the second set of device poses (PB) to identify a subset of at least one device pose in which the positioning performance of the second sensor satisfies the positioning performance rule for the second sensor, Therein, directing (701) movement of the device is performed to guide movement of the second sensor toward the at least one device posture in the subset.

6. The device according to claim 5, wherein: The subset of searching (801) among the second set of device postures (PB) to identify at least one device posture in which the positioning performance of the second sensor satisfies the positioning performance rule for the second sensor is constrained to identify only postures within the second set of device postures (PB) that are within a threshold distance from a current position of the device.

7. The device according to claim 6, wherein: The threshold distance is determined based on at least one of: remaining battery power within the device, remaining time before a deadline for performing a sensor transition, and an amount of deflection from a current movement trajectory of the device that would be required for the device to be moved to one of the second set of device postures (PB).

8. The device according to claim 5, wherein: A search (801) is performed among the second set of device postures (PB) to identify the subset of device postures in which the positioning performance of the second sensor satisfies the positioning performance rule for the second sensor, to identify among the second set of device postures (PB) a posture in which both the first sensor and the second sensor will have positioning performance that satisfies the positioning performance rule for the first sensor and the positioning performance rule for the second sensor, respectively.

9. The device according to any one of claims 1 to 2, wherein: Directing (701) movement of the device to transition the device's posture from not within the second set of device postures (PB) to within the second set of device postures (PB) comprises: identifying a sequence of device poses from among said second set of device poses (PB); and While the operations for activating the second sensor and deactivating the first sensor are performed, movement of the device is directed to transition the device through the sequence of device gestures.

10. The apparatus according to any one of claims 1 to 2, wherein: Determining (601) the first set of device poses (PA) in which the first sensor satisfies the positioning performance rule for the first sensor comprises: determining a positioning performance score for the first sensor at one or more poses based on at least one of: a determined number of erroneous relocalizations determined based on data from the first sensor, a number and quality of feature measurements determined in an area measured by data from the first sensor, a positioning error determined based on data from the first sensor, and an amount of uncertainty in measurements based on data from the first sensor, and determining whether the positioning performance score of the first sensor at the one or more poses satisfies the positioning performance rule for the first sensor; and Determining (603) the second set of device poses (PB) in which the second sensor satisfies the positioning performance rule for the second sensor comprises: determining a localization performance score for the second sensor at one or more poses based on at least one of: a determined number of erroneous relocalizations determined based on data from the second sensor, a number and quality of features determined in an area measured by data from the second sensor, and a localization error determined based on data from the second sensor, and A determination is made as to whether the positioning performance score of the second sensor at the one or more poses satisfies the positioning performance rule for the second sensor.

11. The apparatus of claim 1, wherein: Determining (601) the first set of device poses (PA) in which the first sensor satisfies the positioning performance rule for the first sensor comprises: determining (901) a location of the device relative to a map defined by a data structure identifying previously sensed features of a local environment, determining (903) a positioning performance score for the first sensor when the device is in one or more postures based on whether the first sensor senses the previously sensed feature identified by the data structure, and determining (905) whether the positioning performance score of the first sensor satisfies the positioning performance rule for the first sensor when the device is in the one or more postures; and Determining (603) the second set of device poses (PB) in which the second sensor satisfies the positioning performance rule for the second sensor comprises: determining (1001) a positioning performance score for the second sensor when the device is in one or more postures based on whether the second sensor senses the previously sensed feature identified by the data structure, and Based on whether the second sensor senses the previously sensed feature identified by the data structure, it is determined (1003) whether the positioning performance score of the second sensor satisfies the positioning performance rule for the second sensor when the device is in the one or more postures.

12. The apparatus of claim 1, wherein: Determining (601) the first set of device poses (PA) in which the first sensor satisfies the positioning performance rule for the first sensor comprises: determining (1101) a location of the device relative to a map defined by a data structure identifying locations of features of a local environment, determining (1103) a positioning performance score for the first sensor when the device is in one or more postures based on a comparison of where the first sensor identifies the location of the feature and where the data structure identifies the location of the feature, and determining (1105) whether the positioning performance score of the first sensor satisfies the positioning performance rule for the first sensor when the device is in the one or more postures; and Determining (603) the second set of device poses (PB) in which the second sensor satisfies the positioning performance rule for the second sensor comprises: determining (1201) a positioning performance score for the second sensor when the device is in one or more postures based on a comparison of where the second sensor identifies the location of the feature and where the data structure identifies the location of the feature, and A determination is made (1203) as to whether the positioning performance score of the second sensor satisfies the positioning performance rule for the second sensor when the device is in the one or more postures.

13. The apparatus of claim 1, wherein: Determining (601) the first set of device poses (PA) in which the first sensor satisfies the positioning performance rule for the first sensor comprises: determining (1301) to locate the device at a location within a map defined by a data structure, the data structure identifying a positioning performance score for the device at the location; Based on positioning the device at the location, obtaining (1303) a positioning performance score of the first sensor from the data structure; and determining (1305) whether the positioning performance score of the first sensor satisfies the positioning performance rule for the first sensor; and Determining (603) the second set of device poses (PB) in which the second sensor satisfies the positioning performance rule for the second sensor comprises: Based on positioning the device at the location, obtaining (1401) a positioning performance score of the second sensor from the data structure; and A determination is made (1403) as to whether the positioning performance score of the second sensor satisfies the positioning performance rule for the second sensor.

14. The apparatus according to any one of claims 11 to 13, wherein: The positioning performance scores of the first sensor and the second sensor are determined based on accessing the data structure of the map stored in a networked data repository.

15. The apparatus according to any one of claims 11 to 13, wherein: The positioning performance scores of the first sensor and the second sensor are determined based on accessing the data structure of the map stored in a memory of the device.

16. The apparatus according to any one of claims 1 to 2, wherein: Activating (605) the second sensor includes one of: triggering the second sensor to transition to a high power state, and triggering power-on of the second sensor; and Deactivating (607) the first sensor includes one of: triggering the first sensor to transition to a low power state, and triggering power off of the first sensor.

17. The apparatus according to any one of claims 1 to 2, wherein: Deactivating (607) the first sensor includes setting a data sampling rate of the first sensor to one of a reduced sampling rate and zero, or setting a resolution of the first sensor to a reduced resolution or zero.

18. The apparatus according to any one of claims 1 to 2, wherein: Determining (601) the first set of device postures (PA) in which the first sensor satisfies the positioning performance rule for the first sensor and determining (603) the second set of device postures (PB) in which the second sensor satisfies the positioning performance rule for the second sensor is determined based on obtaining a list of sensors that are currently active for use by other devices positioned nearby.

19. The apparatus according to any one of claims 1 to 2, further comprising: Based on determining at least one of a currently measured decrease in positioning performance of the first sensor and a predicted decrease in positioning performance of the first sensor, activation of the second sensor is initiated (1501).

20. The apparatus according to any one of claims 1 to 2, further comprising: Based on determining that the first sensor uses more than a threshold amount of defined hardware resources, activation of the second sensor is initiated (1601).

21. The apparatus according to claim 20, wherein: The defining hardware resources includes at least one of power consumption, processor utilization, memory utilization, and network communication utilization.

22. A method of performing positioning by a device using a set of sensors transported with the device, the set of sensors comprising a first sensor and a second sensor, the method comprising performing the operations of the device according to any one of claims 1 to 21.

23. A computer program product for performing positioning of a device using a set of sensors transported with the device, the set of sensors comprising a first sensor and a second sensor, the computer program product comprising: A non-transitory computer readable medium storing program code executable by at least one processor of the device to configure the at least one processor to perform the operation of the device according to any one of claims 1 to 21.

Images