Posture estimation

Abstract

Provides posture estimation. [Solution] The subject is a method comprising: using a first motion model of the motion of a mobile device to determine the current state of the mobile device, referred to as a first state estimate of the mobile device, using a primary pose estimate of the mobile device; determining at least one of a secondary pose estimate of the mobile device using first sensor measurements, or a first reference state estimate using a reference motion model and a primary pose estimate; determining a first estimation error of the first state estimate using at least one of the secondary pose estimate or the first reference state estimate; and triggering an update of the primary pose estimate to use the updated primary pose estimate for the next run of the method, based on the first estimation error.

Description

[Technical Field]

[0001] Various exemplary embodiments relate to positioning, and more specifically, to systems for updating attitude estimates in a localization system. [Background technology]

[0002] Positioning and motion control in autonomous systems may involve estimating the current state of a mobile device and generating movement commands to efficiently and accurately reach a desired target. This process may rely on sensor data to determine the device's attitude and correct deviations from the motion plan. The motion planner may calculate a viable path based on the latest attitude estimate, taking into account environmental and kinematic constraints. The motion plan may be updated as new positioning data becomes available, ensuring smooth and adaptive navigation. However, this process can be computationally intensive and transmission-intensive. [Overview of the project]

[0003] An exemplary embodiment provides a system comprising a device referred to as a first device, the first device including at least one processor and at least one memory that, when executed by the at least one processor, stores instructions causing the first device to perform a first positioning operation, the first positioning operation including using a first motion model of the motion of a mobile device to determine the current state of the mobile device, referred to as a first state estimate of the mobile device, using a primary pose estimate of the mobile device; determining at least one of a secondary pose estimate of the mobile device using first sensor measurements relating to the mobile device, or a first reference state estimate of the mobile device using a reference motion model and the primary pose estimate; determining a first estimation error of the first state estimate using at least one of the secondary pose estimate or the first reference state estimate; and triggering an update of the primary pose estimate to use the updated primary pose estimate for the next execution of the first positioning operation based on the first estimation error.

[0004] An exemplary embodiment provides a method comprising: using a primary pose estimate of a mobile device to determine a first motion model of the mobile device's motion to determine a current state of the mobile device, referred to as a first state estimate of the mobile device; determining at least one of a secondary pose estimate of the mobile device using first sensor measurements, or a first reference state estimate using a reference motion model and the primary pose estimate; using at least one of the secondary pose estimate or the first reference state estimate to determine a first estimation error of the first state estimate; and triggering an update of the primary pose estimate to use the updated primary pose estimate for the next run of the method based on the first estimation error.

[0005] An exemplary embodiment provides a computer program product that includes processor-executable instructions for causing a device to perform at least one method.

[0006] An exemplary embodiment provides a non-temporary computer-readable medium that, when executed by the device, includes program instructions causing the device to perform at least one method.

[0007] As used herein, terms such as “first,” “second,” etc., are used as labels for the preceding nouns and do not imply any type of order (e.g., spatial, temporal, logical) unless explicitly defined.

[0008] The attached drawings are included and incorporated herein, and constitute part thereof, to provide a further understanding of the examples. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram of a system for location determination, based on an example from this subject. [Figure 2] This is a process flowchart illustrating a method for updating the primary pose estimate, using an example from this subject. [Figure 3] This is a process flowchart illustrating a method for updating the primary pose estimate, using an example from this subject. [Figure 4] This is a process flowchart illustrating a method for updating the primary pose estimate, using an example from this subject. [Figure 5A] This paper presents a cloud-based location system as an example of this subject. [Figure 5B] Figure 5A shows a workflow diagram detailing how the cloud-based localization system in this subject processes attitude estimation updates. [Figure 6A] This paper presents a cloud-based location system as an example of this subject. [Figure 6B] Figure 6A shows a workflow diagram detailing how the cloud-based localization system in this subject processes attitude estimation updates. [Figure 7A] This paper presents a cloud-based location system as an example of this subject. [Figure 7B]According to an example of the present subject matter, there is shown a workflow diagram detailing the method by which the cloud-based positioning system of FIG. 7A processes pose estimation updates. [Figure 7C] According to an example of the present subject matter, there is shown a workflow diagram detailing the method by which the cloud-based positioning system of FIG. 7A processes pose estimation updates. [Figure 8A] According to an example of the present subject matter, there is shown a cloud-based positioning system. [Figure 8B] According to an example of the present subject matter, there is shown a workflow diagram detailing the method by which the cloud-based positioning system of FIG. 8A processes pose estimation updates. [Figure 9] According to an example of the present subject matter, there is shown a workflow diagram detailing the method by which a cloud-based positioning system provides pose estimation drift using a data-driven model. [Figure 10] It is a block diagram showing an exemplary apparatus according to the present subject matter. Mode for Carrying Out the Invention

[0010] In the following description, for purposes of explanation rather than limitation, specific details such as specific architectures, interfaces, and techniques are set forth to provide a thorough understanding of the examples. However, it will be apparent to those skilled in the art that the subject matter of the present disclosure may be practiced in other illustrative examples that depart from these specific details. In some instances, detailed descriptions of well-known devices and / or methods are omitted so as not to obscure the description with unnecessary detail.

[0011] This subject can reduce traffic transmission while maintaining accurate localization by dynamically updating the primary attitude estimate only when necessary, based on a controlled error determination. By using predefined error conditions, the system optimizes energy consumption and processing load to ensure that localization updates occur efficiently and only when needed. This approach enhances scalability and adaptability and is well-suited for real-time autonomous navigation, industrial automation, and IoT-based mobile tracking applications where low latency and high-precision positioning are essential while minimizing unnecessary data transmission and computational overhead.

[0012] The system may be provided to enable accurate positioning of mobile devices while maintaining efficiency. The system may support the implementation of a localization system or be integrated into an existing localization system to enhance localization estimation, motion tracking, and state determination. Specifically, this subject may enable accurate and controlled updating or refinement of the primary pose estimation of a mobile device. Primary pose estimation refers to the pose of the mobile device and may represent its position and orientation in a given reference frame. The reference frame may be a global coordinate system (e.g., world coordinate system, geographic coordinate system, or inertial coordinate system) or a local coordinate system (e.g., mobile device-centered frame, map-relative frame, or environment-specific reference). The term “estimation” as used herein does not imply inaccuracy, but reflects the fact that positioning relies on sensor data, motion models, or computational techniques, which may include approximations or corrections based on available measurements. Thus, pose estimation refers to the pose representing the position and orientation of the mobile device, while state estimation may refer to the state, which may include additional motion parameters such as velocity and acceleration. The term primary attitude estimation may be used to indicate its role as a starting point for motion planning of a mobile device. That is, a mobile device may use primary attitude estimation, provided, for example, by a localization system, to perform motion control and motion planning for the mobile device. Primary attitude estimation may be provided by the system or received from another system that performs localization or state estimation. In the latter case, the system acts as a controller that determines the time instance for updating or redetermining the primary attitude estimation, ensuring accurate and timely adjustments to maintain reliable positioning. Primary attitude estimation may be determined using sensor data that may be provided by a source referred to as a primary sensor source and using an estimation algorithm that may be referred to as a primary algorithm.

[0013] The system may be provided with a first device and inputs, which may not have been used in determining the primary pose estimate but may provide complementary or corrective information to enhance the accuracy and robustness of localization. The inputs may be provided by additional sensor data sources and / or motion models. This may allow for an independent decision process to determine the time instance in which the primary pose estimate is updated. In contrast to using the same inputs as in determining the primary pose estimate, this approach may enable more adaptive and reliable updates by incorporating diverse data sources to detect deviations or uncertainties that may not have been considered in the original estimate. For example, the system may include a first device.

[0014] The first device may be configured to perform an action referred to as a first positioning action. The first positioning action may use a primary pose estimate as an input referred to as a pose input. The first positioning action may include using a first motion model of the mobile device's motion to determine or estimate the current state of the mobile device, referred to as a first state estimate, using the primary pose estimate of the mobile device. The first device may be configured to use a first motion model to estimate the first state estimate using the primary pose estimate of the mobile device. The mobile device may be a system capable of moving within an environment. The mobile device may be equipped with sensors, processing units, and communication interfaces to support tasks such as navigation, localization, and control. The mobile device may include robots such as wheeled mobile robots (WMRs) and autonomous vehicles. The first motion model may be a motion model. The motion model may be, for example, a mathematical representation of how the mobile device moves over time based on starting conditions such as control inputs and a primary pose estimate. The motion model may predict the state of the mobile device (e.g., a future state) using equations that describe its kinematics or dynamics. The state of the device may include, for example, at least one of the following: the device's position, orientation, velocity, or acceleration.

[0015] The first positioning operation may include determining at least one of the following: a secondary pose estimate of the mobile device using first sensor measurements of the mobile device, or a first reference state estimate of the mobile device using a reference motion model and a primary pose estimate. The first device may be configured to determine the secondary pose estimate of the mobile device using the first sensor measurements. Furthermore, or alternatively, the first device may be configured to determine the first reference state estimate of the mobile device using a reference motion model and a primary pose estimate. The first sensor measurements may refer to sensor data obtained from sources other than at least some of the primary sensor sources used to determine the primary pose estimate, or from sources different from at least some of the primary sensor sources, thereby enabling complementary or independent verification of the mobile device's position and motion. In another example, the first sensor measurements may include sensor data obtained from at least some of the primary sensor sources, but the algorithm used to determine the secondary pose estimate from the sensor data may differ from the primary algorithm. The reference motion model may also be called a ground truth motion model, as it may serve as a more accurate or verified representation of the mobile device's motion, for example, depending on externally corrected or post-processed estimations.

[0016] The first motion model may be, for example, a kinematic model that describes the motion of a mobile device based on position, velocity, and acceleration without considering external forces, while the reference motion model may be a kinematic model of a mobile device that incorporates corrections for disturbances, delays, or environmental interventions.

[0017] Therefore, in this subject, once the primary pose estimate is determined, it may be supplemented in each instance of performing the first positioning operation by incorporating a first state estimate, a secondary pose estimate, and / or a first reference state estimate, to provide a more comprehensive and robust representation of the mobile device's position and motion. The first state estimate, the secondary pose estimate, and the first reference state estimate may be determined, for example, for the same time and for the same reference frame. Alternatively, these estimates may be determined for different times and / or different reference frames. In such cases, time alignment mechanisms (e.g., interpolation, extrapolation, or delay compensation techniques) and frame transformations (e.g., coordinate frame transformation, pose re-referencing, or sensor fusion adjustment) may be used to minimize errors and maintain accuracy of comparison.

[0018] The first positioning operation may include determining an estimation error, referred to as the first estimation error of the first state estimate, using at least one of a secondary attitude estimation or a first reference state estimation. The first device may be configured to determine the first estimation error of the first state estimate using a secondary attitude estimation. Alternatively, the first device may be configured to determine the first estimation error of the first state estimate using a first reference state estimation. Alternatively, the first device may be configured to determine the first estimation error of the first state estimate using both a secondary attitude estimation and a first reference state estimation.

[0019] For example, the first device may determine the first estimation error of the first state estimate by comparing it with at least one of a secondary pose estimate or a first reference state estimate. The secondary pose estimate may provide an independent measurement of the mobile device's position derived from an additional sensor data source, and the first reference state estimate, calculated using a different motion model, may present an alternative prediction of the mobile device's state. By detecting discrepancies between the first state estimate and these independently obtained estimates, the system can quantify localization errors, detect drift, and trigger corrective updates as needed. This method can ensure greater reliability and robustness by cross-validating state estimates against multiple sources.

[0020] The first positioning operation may include triggering an update of the primary attitude estimate based on a first estimation error. This may result in an updated primary attitude estimate. The updated primary attitude estimate may be provided, for example, as an attitude input to the next execution of the first positioning operation, thereby ensuring continuous refinement of the estimated state. The first device may be configured to trigger an update of the primary attitude estimate based on a first estimation error, meaning that the first device may dynamically evaluate the discrepancy between the first state estimate and an alternative estimate, such as a secondary attitude estimate or a first reference state estimate, in order to determine when an update of the primary attitude estimate is necessary. For example, the update of the primary attitude estimate may be conditionally triggered depending on whether the first estimation error satisfies a first trigger condition. The first trigger condition may, for example, require that the first estimation error exceeds a predefined threshold. If the first estimation error does not exceed the threshold, an update of the primary attitude estimate may not be necessary, and the system may avoid wasting computational and communication resources. Conversely, if the first estimation error exceeds a threshold, an update of the primary pose estimate is triggered to improve the accuracy of the first positioning operation. Triggering an update may include providing the mobile device's current sensor data to perform the primary pose estimate update and providing the updated primary pose estimate to the first device. Updating the primary pose estimate may include redetermining the primary pose estimate using the provided sensor data. For example, the primary pose estimate may be redetermined using the provided sensor data and primary algorithm. The updated primary pose estimate may be the current primary pose estimate. The first device may be configured to maintain only the most recent (current) primary pose estimate provided to or by the first device, which may mean that for each instance of performing the first positioning operation, the maintained primary pose estimate is used as input to perform the first positioning operation. Furthermore, if the first device is responsible for providing a primary pose estimate to another entity, this maintained primary pose estimate may also be used for that provision.

[0021] The updated primary attitude estimate can be used as an attitude input for the next execution of the first positioning action.

[0022] For example, the first device may be configured to repeatedly perform a first positioning operation. The iterations may be performed at predefined time intervals, in response to specific events, or dynamically based on system conditions, until a termination criterion is met. Such termination criteria may include reaching a target location, achieving a predefined accuracy of localization, exceeding a maximum number of iterations, or detecting system constraints such as limited computing resources or communication failures. The first device may be configured to repeatedly perform the first positioning operation according to a first frequency. This iterative process can ensure that the system maintains accurate and reliable state estimations while adapting to environmental and operational constraints.

[0023] This subject may further improve the update control of primary attitude estimation by using another device referred to as a second device. The system may include a second device. The first motion model may be implemented as an identical copy in both the first and second devices. The second device is configured to perform a second positioning operation. The second positioning operation may differ from the first positioning operation in terms of the type of sensor data used, attitude input, or update criteria employed. The second positioning operation may implement different update thresholds, enabling more adaptive and accurate correction of the primary attitude estimation. The second positioning operation may receive the primary attitude estimation of a mobile device as input.

[0024] The second positioning operation includes using the first motion model to determine or estimate the current state, referred to as a second state estimate, using the primary pose estimate of the mobile device; determining at least one of the following: another secondary pose estimate of the mobile device using second sensor measurements of the mobile device (referred to as another secondary pose estimate); or a second reference state estimate of the mobile device using a reference motion model and the primary pose estimate; determining a second estimation error for the second state estimate using at least one of the other secondary pose estimate or the second reference state estimate; and triggering an update of the primary pose estimate to use the updated primary pose estimate for the next execution of the first or second positioning operation, based on the second estimation error.

[0025] A second device may be configured to use the first motion model to estimate a current second state estimate using a primary pose estimate of the mobile device. The second device may be configured to determine other secondary pose estimates of the mobile device using second sensor measurements of the mobile device. Alternatively, or further, the second device may be configured to determine a second reference state estimate of the mobile device using a reference motion model and a primary pose estimate. The second sensor measurements may refer to sensor data obtained from sources other than at least some of the primary sensor sources used to determine the primary pose estimate, or from sources different from at least some of the primary sensor sources, thereby allowing complementary or independent verification of the mobile device's position and motion. In another example, the second sensor measurements may include sensor data obtained from at least some of the primary sensor sources, but the algorithm used to determine other secondary pose estimates from the sensor data may differ from the primary algorithm and form the algorithm used to determine the secondary pose estimate. The second device may be configured to determine a second estimation error for the second state estimate using other secondary pose estimates. Alternatively, the second device may be configured to determine a second estimation error for the second state estimate using a second reference state estimate. Alternatively, the second device may be configured to determine a second estimation error for the second state estimate using other secondary pose estimates and a second reference state estimate. The second device may be configured to trigger an update of the primary pose estimate based on the second estimation error. This may result in an updated primary pose estimate. The updated primary pose estimate may be provided, for example, as a pose input to the next execution of the first positioning operation and / or the second positioning operation, thereby ensuring continuous refinement of the estimated state. The second device may be configured to trigger an update of the primary pose estimate based on the second estimation error, which means that the second device may dynamically evaluate the discrepancy between the second state estimate and alternative estimates, such as other secondary pose estimates or a second reference state estimate, to determine when an update of the primary pose estimate is necessary.For example, updating the primary attitude estimate is conditionally triggered depending on whether the second estimation error satisfies a second trigger condition. The second trigger condition may or may not be the same as the first trigger condition. The second trigger condition may, for example, require that the second estimation error exceeds a predefined threshold. If the second estimation error does not exceed the threshold, updating the primary attitude estimate may not be necessary, and the system can avoid wasting computational and communication resources. On the other hand, if the second estimation error exceeds the threshold, updating the primary attitude estimate is triggered to improve positioning accuracy. Triggering an update may include providing current sensor data from the mobile device to perform the primary attitude estimate update and providing the updated primary attitude estimate to the first device. The updated primary attitude estimate may be the current primary attitude estimate. The first device may be configured to decide when and whether to transfer the current primary attitude estimate to the second device.

[0026] The second device may be configured, for example, to maintain only the most recent primary pose estimate provided to it, which may mean that for each instance of performing the second positioning operation, the maintained primary pose estimate is used as input for performing the second positioning operation. Furthermore, if the second device is responsible for providing a primary pose estimate to another entity, this maintained primary pose estimate may also be used for that provision.

[0027] The second state estimate, other secondary pose estimates, and second first reference state estimates may be determined, for example, for the same time and for the same reference frame. Alternatively, these estimates may be determined for different times and / or different reference frames. In such cases, time alignment mechanisms (e.g., interpolation, extrapolation, or delay compensation techniques) and frame transformations (e.g., coordinate frame transformation, pose re-referencing, or sensor fusion adjustment) may be used to minimize errors and maintain accuracy of the comparison.

[0028] For example, the second device may be configured to repeatedly perform a second positioning operation. The iterations may be performed at predefined time intervals, in response to specific events, or dynamically based on system conditions, until a stop criterion is met. For example, the second positioning operation may be performed automatically or repeated in response to receiving attitude input from the first device, including a (current) primary attitude estimate. Such stop criteria may include reaching a target position, achieving a predefined accuracy of positioning, exceeding a maximum number of iterations, or detecting system constraints such as limited computing resources or a communication failure. The second device may be configured to repeatedly perform the second positioning operation according to a second frequency, which may or may not be different from the first frequency. For example, the second frequency may be lower than the first frequency, thereby ensuring that the second positioning operation is performed at a lower frequency than the first positioning operation. By performing the second positioning operation at a lower frequency, the system can reduce processing overhead and minimize unnecessary data transfer while maintaining accurate localization and state estimation through controlled updates based on predefined conditions such as error thresholds or environmental changes.

[0029] Therefore, as defined above, the second positioning operation may differ from the first positioning operation in at least the sensor measurements, the algorithms used to determine the secondary attitude estimation, and their execution rates. This may be advantageous as it enables a modular, distributed processing approach where each operation is performed on a different device, allowing for parallel execution and efficient workload balancing. By having distinct execution rates, the system can prioritize real-time localization updates through the first positioning operation, while the second positioning operation may be performed at different frequencies, focusing on error correction, model refinement, or sensor fusion with an external data source.

[0030] For example, for each execution instance of the first positioning operation, the following may occur: If the primary attitude estimate is updated, the first device may use the updated primary attitude estimate for the next execution of the first positioning operation. However, the updated primary attitude estimate for the execution instance may or may not be transmitted by the first device to the mobile device for motion execution. This difference may be particularly advantageous because the error analysis performed by the first positioning operation may generally require a higher accuracy than what is required for motion execution on the mobile device. By separating attitude estimation for positioning from attitude utilization for motion control, the system can enhance the accuracy of state estimation while avoiding unnecessary processing overhead or delay in real-time motion execution.

[0031] In one example, the updated primary attitude estimate for an execution instance may or may not be transmitted by the first device to the second device. In one example, the first device may be configured to transmit the updated primary attitude estimate to the second device for each execution instance of the first positioning operation if an update occurs. This may allow the second device to use the updated primary attitude estimate as input for executing the second positioning operation. Alternatively, the first device may be configured to transmit the updated primary attitude estimate to the second device under certain conditions, such as when a predefined error threshold is exceeded, when external sensor corrections are integrated, or at regular intervals based on system requirements.

[0032] For example, the first device may be configured to determine whether the primary attitude estimate matches the first state estimate, and, in response to the determination that the primary attitude estimate does not match the first state estimate, to provide the current primary attitude estimate to the second device. For example, this process may be part of the first positioning operation. The primary attitude estimate provided to the second device may be the most recent primary attitude estimate acquired by the first device. Transmitting the primary attitude estimate to the second device may include transmitting the primary attitude estimate to a mobile device.

[0033] The agreement of a primary attitude estimate with a first state estimate may mean that the discrepancy between them is within an acceptable threshold range, which may be defined based on the difference in position, orientation, or velocity. This threshold may account for measurement noise, model uncertainty, and minor deviations that do not significantly affect the accuracy of localization. If the difference between the primary attitude estimate and the first state estimate is within this threshold range, it may not be necessary to provide the current attitude estimate. However, if the discrepancy exceeds the threshold, the primary attitude estimate may be provided to the second device. For example, the difference between the position and / or orientation of the first state estimate and the position and / or orientation of the primary attitude estimate may be evaluated against the threshold. If the position deviation is within a certain distance threshold range and the orientation difference does not exceed a specified angular tolerance, the estimates may be considered consistent.

[0034] For example, the first device may be configured to use a data-driven model to determine whether a primary pose estimate matches other secondary pose estimates, and in response to the determination that the primary pose estimate does not match other secondary pose estimates, to provide the second device with the drift between the primary pose estimate and the other secondary pose estimates provided by the data-driven model, thereby triggering the second device to update the primary pose estimate using the drift.

[0035] In one example, the data-driven model may receive a primary attitude estimate from a localization system or a first device (e.g., providing at least an initial primary attitude estimate) and other secondary attitude estimates from a second device. The first device may be configured to request attitude estimate drift from the second device from the data-driven model, which may receive it from the data-driven model as an output. The second device may use the received attitude estimate drift to update other secondary attitude estimates. Thus, the data-driven model may be used to predict and contribute to reducing the cumulative error of the attitude estimation of an onboard odometry sensor over time.

[0036] In fact, to further reduce the need for communication between the localization system and the mobile device, data-driven models can be employed to learn and predict the drift between the mobile device's secondary pose estimate, which may inherently be less accurate, and the localization system's more precise pose estimate. This drift can be influenced by a variety of factors, including the type of sensor used (e.g., motor encoder, inertial measurement unit (IMU)), the estimation algorithm implemented, the mobile device's speed, and environmental conditions such as the location and surface properties encountered during navigation. To achieve this objective, in this example, a pre-trained and optimized data-driven model, such as a model based on polynomial regression, may be deployed on the localization system side. While hosting the data-driven model directly on the mobile device may be feasible, a localization system deployment may be preferred due to the superior computational power available for continuous training. The data-driven model may be trained using timestamped pose estimate data from both the mobile device and the localization system, and may be able to be continuously trained against real-time pose estimation. The output of the data-driven model may be an estimate of the drift between the pose estimate on the mobile device side and the corresponding pose estimate on the localization system side. When transmitting a more accurate pose from the first device to the second device, the data-driven model may also predict the drift between the current estimate and subsequent estimates, and transmit this data along with the primary pose estimate. The mobile device can then improve its internal model and refine its local pose estimate using additional information that considers its current position, velocity, and other parameters. The internal model refers to the first motion model.

[0037] For example, the primary pose estimate is obtained by correcting the initial primary pose estimate determined from sensor data, and the correction is performed using the speed and latency of the mobile device.

[0038] This process may include, for example, a correction mechanism that adjusts the primary pose estimate by integrating the speed and latency of the mobile device. Speed-based corrections may take into account the fact that the mobile device may continue to move during the latency period, which means that the original pose estimate may no longer accurately reflect its true position.

[0039] When the primary attitude estimate is used in a first positioning operation, the delay may include the delay in the transfer of sensor data to the system (e.g., including the first device) where the primary attitude estimate is determined, taking into account the delay from the time the sensor data was acquired. In this case, the correction may also be referred to as the first correction to the primary attitude estimate. When the primary attitude estimate is used in a second positioning operation or motion control of a mobile device, the delay may include both the delay in the transfer of sensor data to the second device and the additional transmission delay of the primary attitude estimate. This may take into account the time required to transmit the corrected attitude estimate for further processing or use in the second device. In this case, the correction may also be referred to as the second correction to the primary attitude estimate.

[0040] In one example, the attitude input for the first positioning operation may already have a first correction applied, meaning that the primary attitude estimate has been pre-adjusted using the mobile device's speed and transmission delay before being used in the positioning operation. Alternatively, the first device receives the initially determined primary attitude estimate and then performs a first correction to ensure that the resulting primary attitude estimate accurately accounts for the transmission delay. This corrected estimate may then be used as the attitude input for the first positioning operation to improve the accuracy of the state estimation. In one example, the transmission of the updated primary attitude estimate by the first device to the second device may include the first device performing a second correction, which further accounts for an additional transmission delay before transmitting the resulting primary attitude estimate to the second device. This ensures that the second device receives a time-adjusted primary attitude estimate, compensating for any latency that occurs during transmission, thereby improving synchronization, motion control accuracy, and positioning reliability. Alternatively, the first device may receive a primary attitude estimate that may include two separate primary attitude estimates, one of which has already been corrected to a first correction and is used as an attitude input for a first positioning operation, and the other of which has already been corrected to a second correction and is transmitted by the first device to the second device for use in a second positioning operation.

[0041] This correction may be applied when the first and second devices are remotely connected and the mobile device is moving while the attitude estimation is still being processed and / or transmitted. When the first and second devices are co-located, the latency is minimal due to the absence of network-induced delay, so the initial primary attitude estimation may already be sufficiently synchronized. In such scenarios, the first and second corrections may not need to be applied.

[0042] For example, a first motion model is adapted to use a first control input of a mobile device for estimating a first state estimate, and a reference motion model is adapted to use a second control input of a mobile device for estimating a first reference state estimate, where the first control input is different from the second control input. The term control input refers to a set of parameters that affect the motion of the mobile device, such as velocity, acceleration, or steering commands. The first control input may have a different (e.g., worse) accuracy than the second control input. For example, the first control input may be less accurate than the second control input, meaning it may be affected by execution uncertainty, actuator limitations, or disturbances. In contrast, the second control input may represent a refined or corrected version that may take into account compensation mechanisms, model adjustments, or externally optimized control signals, thereby providing a more accurate basis for estimating the first reference state estimate. By utilizing different control inputs for state estimation, the system can detect inconsistencies and improve its robustness to errors in applied motion commands. In one example, the first control input may include the linear and angular velocity of the mobile device, and the second control input may include the linear velocity, angular velocity, and additional disturbance parameters such as external forces or model corrections. By including disturbances in the second control input, the reference motion model can compensate for real-world uncertainties, potentially providing a more accurate reference state estimate compared to a first state estimate based solely on the first control input.

[0043] For example, the first sensor measurement differs from the second sensor measurement in that the first sensor measurement includes external sensor measurements, while the second sensor measurement includes odometry measurements. In fact, the first sensor measurement differs from the second sensor measurement in that the first sensor measurement includes external sensor measurements such as data obtained from the Global Positioning System (GPS), Light Detection and Ranging (LIDAR), a camera, or other external localization sources, while the second sensor measurement includes odometry measurements, including data derived from a wheel encoder, inertial sensor, or motor feedback that estimates motion based on the internal movement of the mobile device.

[0044] The primary sensor source may include, for example, sensors used initially to determine the primary attitude estimate of a mobile device, combining onboard and external sensors to establish a baseline for localization. For example, in an autonomous vehicle, the primary sensor source may include GPS for absolute positioning, an IMU for detecting movement and orientation, and wheel speed sensors for speed estimation. These primary sensor sources may differ from the sources of the first and second sensor measurements, and may introduce additional or alternative data such as an external tracking system (e.g., GPS, fixed camera) or odometry-based correction (e.g., motor feedback, additional IMU data). This difference may enable error detection and correction, ensuring a more robust and adaptive localization system.

[0045] For example, the first device is configured to be remotely connected to the second device. Alternatively, for example, the first device is configured to be collateralized with the second device.

[0046] The first and second devices may be either remotely connected or co-located. In a remote connection scenario, the first device may be part of a processing system remote from the second device, and the second device may be part of a mobile device. The processing system may include, for example, cloud-based computing resources, edge computing infrastructure, or a dedicated remote server that can be configured to perform high-precision localization, motion modeling, and data fusion. This configuration may allow computationally intensive tasks to be offloaded from the mobile device, reducing power consumption and reducing onboard processing requirements. This approach may also improve scalability and enable integration with external localization infrastructure (e.g., GPS networks, real-time mapping systems). However, it may depend on network stability and low-latency communication for real-time updates. In contrast, when the first and second devices are co-located (e.g., both operate within the same physical system, such as the onboard computing unit of a mobile device), data exchange may occur without transmission delay, ensuring faster and more reliable positioning updates. This configuration can be resilient to connectivity issues and may provide real-time adaptability, especially in environments where remote communications are unreliable.

[0047] For example, the system further includes a sensing system for acquiring sensor data from a mobile device, and a positioning system configured to use the sensor data to determine a primary pose estimate, provide the primary pose estimate to a first device, optionally to a second device, or to the mobile device for planning and controlling the mobile device's motion, wherein the first device is included in the positioning system, and the second device is included in the sensing system.

[0048] In practice, the system includes a sensing system and a localization system, each playing a separate yet interconnected role, for example, in determining and refining a primary pose estimate of a mobile device. The sensing system may be responsible for acquiring sensor data from the mobile device using various onboard and external sensors such as an IMU, camera, LiDAR, GPS, or wheel encoder. The localization system may process this data to determine a primary pose estimate, ensuring an accurate representation of the mobile device's position and orientation. Once the primary pose estimate is calculated, the localization system may provide it to at least a first device in accordance with this subject, thereby enabling continuous refinement of the state estimate. Structurally, the first device may be part of the localization system, meaning it can handle state estimation, sensor fusion, and motion modeling, while the second device may be part of the sensing system. The sensing system may further include a motion planner that can control and plan the motion of the mobile device based on the primary pose estimate received by the mobile device. An advantage of this configuration may be that it can enable dynamic updates to the primary pose estimate based on newly acquired sensor data. This structure can also support distributed or hierarchical localization methods, with the localization system residing in the cloud or an external processing unit, while the sensing system remains onboard.

[0049] For example, the transfer delay of sensor data may be between the sensing system and the positioning system. The positioning system may be configured to provide an initial primary attitude estimate, or to perform a first correction and / or a second correction and provide the resulting primary attitude estimate to the first device.

[0050] For example, triggering an update to the primary pose estimate involves controlling the sensing system to provide current sensor data to the localization system.

[0051] This process may play a role in adaptive localization, requiring real-time adjustments to maintain an accurate representation of the mobile device's location and orientation. While sensing systems can continuously collect raw sensor data, they don't always transmit it immediately. Instead, the system can control when sensor data is transferred to the localization system based on predefined triggers, such as a first and second trigger condition.

[0052] For example, an embodiment of the first motion model includes using a state adjustment period that takes into account the change in orientation of the mobile device between time steps, where the time step is a discrete interval in the numerical integral of the first motion model.

[0053] This adjustment can ensure that the estimated motion trajectory accurately reflects the real-world dynamics of the mobile device, especially in systems where rotational motion can significantly impact positioning accuracy. By integrating azimuth corrections such as intermediate step angular velocity correction or higher-order numerical methods (e.g., Runge-Kutta), the system can more accurately predict the future state of the mobile device while minimizing drift and positioning errors. Furthermore, the use of numerical integration ensures that the system can handle variable time steps, enabling more adaptive calculations.

[0054] For example, the orientation adjustment period includes the angular displacement contribution during half a time step.

[0055] Rather than assuming that the orientation remains constant throughout the entire time step, this adjustment can account for gradual changes in orientation. By incorporating angular displacement at the midpoint of the time step, the system can effectively reduce integral errors arising from assuming instantaneous changes in orientation at the beginning or end of the step. This approach may be particularly advantageous in motion models that rely on numerical integration, such as the Runge-Kutta method.

[0056] For example, the first device is configured to determine a first estimation error for a first state estimation by comparing the first state estimation with at least one of a secondary pose estimation or a first reference state estimation.

[0057] This comparison may be performed with respect to position by evaluating the difference in position coordinates (e.g., x, y, z), and / or with respect to orientation by assessing deviations in azimuth angles (e.g., yaw, pitch, roll). When comparing a first state estimate with a secondary attitude estimate, the first device can determine whether the estimated position and orientation match the orientation derived from sensor measurements. Alternatively, when comparing a first state estimate with a first reference state estimate, the first device can evaluate whether the two state-estimated positions and orientations are in a consistent state. Alternatively, the first estimation error may be a combination of the two alternative comparisons, meaning that both the difference between the first state estimate and the secondary attitude estimate, and the difference between the first state estimate and the first reference state estimate, are considered. This combination of error metrics may provide a more comprehensive measure of localization accuracy by allowing the integration of deviations in the orientation derived from the sensor and inconsistencies in the state estimates.

[0058] For example, the comparison between the first state estimate and the first reference state estimate is performed by calculating the difference between the first state estimate and the first reference state estimate at a specific time.

[0059] This comparison may be performed with respect to position by evaluating deviations in positional coordinates (e.g., x, y, z), and / or with respect to orientation by assessing differences in azimuth angles (e.g., yaw, pitch, roll). By ensuring that the comparison is performed at the same time, this can prevent inaccuracies due to latency or asynchronous updates, taking temporal consistency into account.

[0060] Furthermore, the comparison between the first state estimate and the secondary pose estimate can be performed, for example, at the time corresponding to when the secondary pose estimate is obtained.

[0061] Similarly, the comparison features described in relation to the first device for determining the first estimation error can also be applied to a second device, which may be configured to determine a second estimation error for a second state estimate using at least one of other secondary attitude estimates or a second reference state estimate. For example, the second device may perform positional comparisons that evaluate differences in position coordinates (e.g., x, y, z) and / or orientational comparisons that evaluate differences in azimuth angles (e.g., yaw, pitch, roll), similar to the comparisons performed by the first device.

[0062] In one exemplary embodiment of this subject, state and orientation estimation may include two-dimensional (2D) motion, where the position and orientation of the mobile device are determined within a 2D reference frame. The orientation of the mobile device may be presented by its position and orientation, represented in a global reference frame. In this example of 2D orientation (for example, for a mobile device moving in a 2D space such as WMR), the orientation can be expressed as X=(x,y,θ), where the x and y coordinates represent its 2D position and the y yaw (or direction of motion). The inputs to the mobile device are represented by linear velocity and angular velocity, denoted by u1 and u2, respectively. The former moves the mobile device forward, and the latter rotates the mobile device to change its direction of motion. The kinematics of a device moving in 2D space may be represented by the following model, which may provide an example of a reference motion model.

[0063]

number

[0064]

Formula

[0065] The posture estimation at the current time stamp t is provided using sensor data previously collected at time stamp (t-h u ), so the posture estimation that provides an example of a primary posture estimation obtainable from on-board sensor data is Y c1 (t)=(x(t-h u ),y(t-h u ),θ(t-h u )) T +v c1 (t) given in (3), where h u is a communication or processing delay assumed to be constant.

[0066]

Formula

[0067]

number

[0068] An internal model that can provide an example of a first motion model uses a velocity input given to the mobile device to estimate the mobile device's posture. An example of an internal model is:

[0069]

number

[0070]

number

[0071] Equation (5) represents the general state progression in continuous form and is typically expressed as a differential equation defining how the state changes over time. This subject may use numerical integration of the first motion model to enable an efficient embodiment of the first motion model based on estimated errors. Specifically, numerical integration methods may be applied to discretize the continuous-time representation of state dynamics in order to transition from equation (5) to a numerical embodiment such as equation (6). For example, in the discrete-time embodiment, an embodiment of the internal model described in equation (5) may be given by the following equation based on the Runge-Kutta approximation method.

[0072]

number

[0073]

number

[0074] In the context of a cloud-based localization system, two internal models are maintained on both the mobile device and the cloud, and their states are synchronized. The subscripts r and c are used to distinguish between the internal model implemented on the mobile device (e.g., a robot) and the internal model implemented on the cloud where the localization system resides. To quantify the estimation error of the internal models, the estimation error variable is first

[0075]

number

[0076]

number

[0077]

number

[0078]

number

[0079]

number

[0080]

number

[0081] Considering the relationship in equation (9), after mathematical simplification, the following inequality is obtained.

[0082]

number

[0083] Finally, by integrating the derivative of the Lyapunov function derived in equation (12), the function denoted by π(t) can be defined as a solution to the linear differential inequality (12), which can be used to assess the level of estimation error in the presence of model disturbances, measurement noise, and uncertainty. Lyapunov analysis is one example of a method that can be used to assess the level of estimation error. However, other methods may also be used.

[0084] This subject could enable the transmission of onboard sensor data to a localization system according to three different cases that can be mathematically expressed as follows: Case 1:

[0085]

number

[0086]

number

[0087]

number

[0088]

number

[0089]

number

[0090]

number

[0091]

number

[0092]

number

[0093] After an uplink communication instance is triggered, attitude estimation is provided by a third-party localization algorithm, which is then used to predict the state estimation at the current timestamp (since sensor data is collected before processing). The state estimation update of the attitude measurement can be expressed by the following formula:

[0094]

number

[0095] These requirements are triggered by the following conditions corresponding to Case 4.

[0096]

number

[0097]

number

[0098] Figure 1 is a block diagram of a system for localization, as illustrated in this subject. System 100 includes a mobile device 101 and a processing unit 103, which may be remotely connected via a communication link 102 or co-located within the same system architecture.

[0099] The mobile device 101 includes a sensing system 105, which acquires sensor data from onboard sources such as an IMU, camera, LiDAR, GPS, or wheel encoder. Furthermore, the mobile device 101 includes a second device 107, which may process the sensor data locally for state estimation or motion tracking.

[0100] The processing unit 103 includes a positioning system 109 responsible for determining, refining, or correcting the attitude estimation of the mobile device 101. The first device 111, which is also part of the processing unit 103, can perform higher-level decision-making, state estimation refinement, model updating, or communication tasks.

[0101] Communication link 102 facilitates data exchange between the mobile device 101 and the processing unit 103, enabling flexible localization, navigation, and motion estimation strategies. In a remote configuration, the processing unit 103 may leverage cloud computing resources to offload computationally intensive tasks, for example, while in a colocation configuration, all processing may occur onboard the mobile device 101 for real-time responsiveness and minimal network dependency. Communication between components of system 100 may utilize publish / subscribe messaging protocols using wired or wireless links. For example, communication link 102 may be a wired or wireless link.

[0102] By integrating sensor data, motion models, and estimation algorithms, system 100 can support adaptable localization methods to ensure robust, scalable, and efficient state estimation across different operating environments.

[0103] Figure 2 is a process flowchart illustrating an example method of the subject matter. For illustrative purposes, the method described with reference to Figure 2 may be carried out in devices such as the first device 111 or the second device 107 illustrated and described with reference to Figure 1, or the device illustrated and described with reference to Figure 10, but is not limited to this embodiment. The method includes a first positioning operation comprising blocks 201-207.

[0104] In block 201, a first motion model of the mobile device's motion may be used to determine the current state of the mobile device, referred to as a first state estimate, using a primary pose estimate of the mobile device. In block 203, at least one of the following may be determined: a secondary pose estimate of the mobile device using a first sensor measurement of the mobile device, or a first reference state estimate of the mobile device using a reference motion model and a primary pose estimate. In block 205, a first estimation error of the first state estimate may be determined using at least one of the secondary pose estimate or the first reference state estimate. In block 207, based on the first estimation error, an update of the primary pose estimate may be triggered to use the updated primary pose estimate for the next execution of the first positioning operation.

[0105] Figure 3 is a process flowchart illustrating an example method of the subject matter. For illustrative purposes, the method described with reference to Figure 3 may be carried out in devices such as the first device 111 or the second device 107 illustrated and described with reference to Figure 1, or the device illustrated and described with reference to Figure 10, but is not limited to this embodiment. The method includes a second positioning operation comprising blocks 301-307.

[0106] In block 301, the first motion model may be used to determine a second state estimate using a primary pose estimate of the mobile device. In block 303, at least one of the following may be determined: another secondary pose estimate of the mobile device using a second sensor measurement of the mobile device, or a second reference state estimate of the mobile device using a reference motion model and a primary pose estimate. In block 305, a second estimation error for the second state estimate may be determined using at least one of the other secondary pose estimates or a second reference state estimate. In block 307, based on the second estimation error, an update of the primary pose estimate may be triggered to use the updated primary pose estimate for the next execution of the first or second positioning operation.

[0107] Figure 4 is a process flowchart illustrating an example method of the subject matter. For illustrative purposes, the method described with reference to Figure 4 may be carried out in two devices, such as the first device 111 and the second device 107 illustrated and described with reference to Figure 1, or the device illustrated and described with reference to Figure 10, but is not limited to this embodiment. The method includes performing a first positioning operation in block 401 and a second positioning operation in block 403. Block 401 may include the execution of blocks 201-207. Block 403 may include the execution of blocks 301-307.

[0108] In one exemplary embodiment of the method shown in Figure 4, in block 401, a first positioning operation may be performed repeatedly according to a first frequency. In block 403, a second positioning operation may be performed repeatedly according to a second frequency. After each execution instance of the first positioning operation, the method may include providing the resulting updated primary attitude estimate to the first device for the next / future execution of the first positioning operation using the updated primary attitude estimate, and optionally, if an update occurs, transmitting the resulting updated primary attitude estimate to the second device for the next / future execution of the second positioning operation using the updated primary attitude estimate. In one example, the two loops may be synchronized, meaning that if an update occurs during one of the positioning operations, the updated primary attitude estimate will always be available before the execution of the other positioning operation, ensuring that both processes operate with the most up-to-date information. Alternatively, the loop may operate asynchronously, and the second positioning operation may proceed using the most recent available primary attitude estimate, even if an update from the first positioning operation has not yet been received.

[0109] In one example, a computer program product may be provided that includes processor-executable instructions for causing the device to perform a first positioning operation or a second positioning operation (as described, for example, with reference to Figure 4).

[0110] Figure 5A is a block diagram illustrating a cloud-based location system as an example of this subject.

[0111] The cloud-based location system 500 includes a mobile device 501 and a cloud processing unit 503, the cloud processing unit 503 may be hosted on a cloud computing infrastructure or another remote processing system. The mobile device 501 and the cloud processing unit 503 may be connected via a communication link to enable data exchange between the sensing component and the processing component of the system. The mobile device 501 includes a sensing system 505, the sensing system 505 acquires sensor data and time instance t u,k The sensor data is transmitted to the location system 509 in the cloud processing unit 503. The mobile device 501 includes a second device 507 (C2 unit), which may be responsible for handling local processing and state estimation tasks. The cloud processing unit 503 hosts the location system 509, which processes the sensor data and records it over time in instance t. d,k The orientation (primary orientation estimation) of the mobile device 501 is estimated. The first device 511 (C1 unit) is included in the cloud processing unit 503 and can facilitate computational tasks related to orientation estimation and synchronization with the mobile device 501 and the C2 unit. Furthermore, external sensors provide environmental data that can supplement the accuracy of localization. Each of the C1 and C2 units contains the same copy of the internal model that can provide an exemplary embodiment of the first motion model.

[0112] Therefore, in the case of a cloud-based positioning system, the C1 unit may reside on a remote server within the cloud processing unit 503, and the C2 unit may reside alongside the onboard sensors and be implemented on the mobile device 501. However, if the positioning system 500 is implemented on the mobile device 501, the C1 unit may also reside on the mobile device 501. The C1 unit maintains an internal model of the mobile device 501, for example, to estimate the state of the mobile device over time when precise measurements are not available. The internal model of the mobile device is used to reduce reliance on sensor data for more accurate pose estimation. Initially, the C1 unit receives a primary pose estimation (e.g., initial primary pose estimation) provided by the positioning system 509 using the mobile device's internal sensors and available external sensors. Subsequent instances of updating the accurate primary pose estimation may occur under specific conditions. An internal model of the mobile device representing its dynamic behavior is maintained in the C2 unit, which initially receives an accurate primary pose estimation provided by the positioning system, and then updates to the accurate primary pose estimation occur under specific conditions. A C2 unit located near the internal sensors may periodically estimate the attitude of the mobile device using odometry if available. This may rely on sensors such as motor encoders or onboard IMUs, which may provide less accurate attitude estimations but may require minimal computational resources. First-order attitude estimation is referred to as accurate attitude estimation or accurate first-order attitude estimation to indicate that it was determined by a positioning system using, for example, a first-order algorithm and sensor data from a first-order sensor source.

[0113] By integrating on-device sensing with cloud-based processing, the cloud-based positioning system 500 balances local real-time computation with high-precision remote positioning, enabling adaptive positioning. Specifically, Figure 5A shows the state of the cloud-based positioning system for a use case that includes a step. The step represents the transmission of sensor data to enable updating the primary attitude estimate of the mobile device. This update occurs through the sensor client of the sensing system 505 and is triggered based on the fulfillment of specific trigger conditions, ensuring that attitude updates are performed only when necessary to optimize resource efficiency and positioning accuracy. In this use case, the C2 unit may detect, based on odometry measurements, that the current position of the mobile device is too far from the estimate of the internal model, and thus trigger the execution of the step. Figure 5B provides further details of this use case.

[0114] Figure 5B shows a workflow diagram detailing how the positioning system in Figure 5A processes attitude estimation updates, using an example from this subject. The workflow diagram may include components of the positioning system 500, such as a sensor client, a positioning client, and C1 and C2 units.

[0115] An initial stage 520 may be provided. In the initial stage, the localization client includes an algorithm (521) that provides an attitude estimate about the mobile device based on various sensor sources such as onboard sensors, an external tracking system, and environmental features. Unit C1 includes an internal model (522) of the motion of the mobile device and maintains an accurate primary attitude estimate (523) received from the localization system to determine the state of the mobile device, such as a first state estimate. Similarly, Unit C2 maintains an internal model (524) and an accurate primary attitude estimate (525) to determine the state of the mobile device, such as a first state estimate, and further has periodic attitude estimates (other secondary attitude estimates) using odometry (526). In step 530, the validation process may include checking whether the error of the internal model exceeds a defined threshold based on odometry measurements. This validation may include, for example, a comparison between the first state estimate and other secondary attitude estimates. If the error exceeds a threshold, the C2 unit may control the sensor client (531) to send the current sensor data (532) to the localization client. This sensor data is then processed by the localization system (533) to obtain a new primary pose estimate of the mobile device, leading to a pose estimate update stage (540). The state estimate update stage 540 includes steps 541 and 542. After this update, the new primary pose estimate is sent to the C1 unit (541). The C1 unit may update (542) the state estimate of the mobile device (e.g., a first state estimate) based on the received primary pose estimate. Figure 5B may provide an exemplary embodiment according to Case 1.

[0116] Figure 6A is a block diagram illustrating a cloud-based localization system as an example of this subject. The cloud-based localization system in Figure 6A is identical to the system shown in Figure 5A, differing in that it illustrates a different use case. In this scenario, the C1 unit detects, based on external sensor measurements, that the primary pose estimate of the mobile device deviates too much from the estimate of the internal model (e.g., the first state estimate). This triggers a localization estimate correction, as described in Figure 6B.

[0117] Figure 6B shows a workflow diagram detailing how the positioning system in Figure 6A processes attitude estimation updates, using an example from this subject. The workflow diagram may include components of the positioning system 500, such as the sensor client, positioning client, C1 unit, and C2 unit.

[0118] An initial stage 520 may be provided, as illustrated with reference to Figure 5B. The C1 unit may verify (630) whether the error of the internal model exceeds a defined threshold based on external sensor measurements (e.g., secondary pose estimation). This verification may include, for example, a comparison between a first state estimation and a secondary pose estimation obtained by the C1 unit. If the error of the internal model exceeds a defined threshold, the C1 unit may control (631) the sensor client to send the current sensor data (632) to the localization client. This sensor data is then processed (633) by the localization system to obtain a new primary pose estimation of the mobile device, leading to a pose estimation update stage 540, as illustrated with reference to Figure 5B. Figure 6B may provide an exemplary embodiment according to Case 2.

[0119] Figure 7A is a block diagram illustrating a cloud-based localization system as an example of this subject. The cloud-based localization system in Figure 7A is identical to the system shown in Figure 5A, differing in that it illustrates a different use case. In this scenario, either the C1 or C2 unit detects that the error in the internal model has become too large due to measurement noise and disturbances. Figures 7B and 7C provide further details of this use case.

[0120] Figure 7B shows a workflow diagram detailing how the positioning system in Figure 7A processes attitude estimation updates, using an example from this subject. The workflow diagram may include components of the positioning system 500, such as the sensor client, positioning client, C1 unit, and C2 unit.

[0121] An initial stage 520 may be provided, as described with reference to Figure 5B. The C1 unit may detect an excess error in the internal model due to measurement noise and disturbances, as well as uncertainty (730), and the C1 unit may control the sensor client to transmit the current sensor data (732) to the localization client (731). This process may include, for example, a comparison between a first state estimate and a first reference state estimate obtained using a reference model and a first pose estimate.

[0122] Alternatively, as shown in Figure 7C, the C2 unit may detect excess errors in the internal model due to measurement noise and disturbances, as well as uncertainty (735), and the C2 unit may control the sensor client to send updated sensor data (732) to the localization client (736). This process may include, for example, a comparison between a first state estimate and a second reference state estimate obtained using a reference model and a primary pose estimate.

[0123] This sensor data is then processed by a localization system (733) to obtain a new primary pose estimate of the mobile device, leading to a pose estimate update step 540, which is described with reference to Figure 5B. Figure 7B or Figure 7C may provide an exemplary embodiment according to Case 3.

[0124] Figure 8A is a block diagram illustrating a cloud-based localization system as an example of this subject. The cloud-based localization system in Figure 8A is identical to the system shown in Figure 5A, differing in that it illustrates a different use case. In this scenario, unit C1 detects that the primary pose estimate from the localization system is too far from the internal model (e.g., from the first state estimate). At this point, the internal models of units C1 and C2 may be updated. Figure 8B provides further details of this use case. Updating the internal models means providing the internal models with updated inputs, for example, by updating the primary pose estimate, as used herein, which can serve as a starting point for the internal models to perform state estimation of the mobile device.

[0125] Figure 8B shows a workflow diagram detailing how the localization system in Figure 8A processes attitude estimation updates, using an example from this subject. The workflow diagram may include components of the localization system 500, such as the sensor client, localization client, C1 unit, and C2 unit.

[0126] An initial stage 520 may be provided, as described with reference to Figure 5B. The local client may send a new primary pose estimate of the mobile device to the C1 unit (830). The C1 unit may verify (831) whether the error of the internal model exceeds a defined threshold based on the mobile device state estimate (e.g., a first state estimate). This process may include, for example, a comparison between the first state estimate and the primary pose estimate. If the error of the internal model exceeds a defined threshold, the C1 unit may update the internal model (832). The C1 unit may synchronize (833) the updated internal model with the C2 unit. The C2 unit may update its internal model (834). Figure 8B may provide an exemplary embodiment according to case 4.

[0127] Figure 9 shows a workflow diagram detailing how a cloud-based localization system provides pose estimation drift using a data-driven model, as an example in this subject. The data-driven model may receive a primary pose estimation from the localization system (901) and other secondary pose estimations determined by the C2 unit from the C2 unit (902). The C1 unit may request pose estimation drift on a mobile device from the data-driven model (903) and receive it from the data-driven model as output (904). Triggering an update of the first motion model may include the C1 unit sending the pose estimation drift to the C2 unit (905). The C2 unit may update the first motion model using the received pose estimation drift (906).

[0128] Figure 10 shows a block diagram illustrating the configuration of device 1070, which is configured to perform at least part of the subject matter. Note that device 1070 shown in Figure 10 may include several additional elements or functions in addition to those described below, which are not essential for understanding and are therefore omitted here for simplification. Furthermore, the device may also be another device having similar functions, such as a chipset, chip, or module, which may be part of the device or mounted as a separate element to device 1070. Device 1070 may include a processor 1071, such as a processing function or a central processing unit (CPU), which executes instructions given by a program related to a flow control mechanism, etc. The processor 1071 may include one or more processing parts dedicated to specific processing described later, or the processing may be performed by a single processor. The part for performing such specific processing may also be provided as a separate element or in one or more additional processors or processing parts, for example, in one physical processor such as a CPU or in several physical entities. Reference numeral 1072 indicates a transceiver or input / output (I / O) unit (interface) connected to the processor 1071. The I / O unit 1072 may be used to communicate with one or more other network elements, entities, terminals, etc. The I / O unit 1072 may be a coupling unit containing communication equipment for several network elements, or it may include a distributed structure having multiple different interfaces for different network elements. Reference numeral 1073 indicates memory that can be used, for example, to store data and programs executed by the processor 1071 and / or as working storage for the processor 1071.

[0129] The processor 1071 is configured to perform processing related to the subject matter described throughout this disclosure. In particular, the device 1070 may be configured to perform methods such as those described with reference to Figure 2 or Figure 3.

[0130] For example, the processor 1071 is configured to use a first motion model of the mobile device's motion to determine the current state of the mobile device, referred to as a first state estimate of the mobile device, using a primary pose estimate of the mobile device; to determine at least one of a secondary pose estimate of the mobile device using first sensor measurements of the mobile device, or a first reference state estimate of the mobile device using a reference motion model and a primary pose estimate; to determine a first estimation error of the first state estimate using at least one of the secondary pose estimate or the first reference state estimate; and to trigger an update of the primary pose estimate to use the updated primary pose estimate for the next execution of a first positioning operation based on the first estimation error.

[0131] As will be understood by those skilled in the art, aspects of the present invention may be embodied as apparatus, methods, computer programs, or computer program products. Accordingly, aspects of the present invention may take the form of complete hardware embodiments, complete software embodiments (including firmware, resident software, microcode, etc.), or embodiments combining software and hardware embodiments, all of which may be generally referred to herein as “circuits,” “modules,” or “systems.” Furthermore, aspects of the present invention may take the form of computer program products embodied in one or more computer-readable media having computer executable code embodied thereon. A computer program includes computer executable code or “program instructions.”

[0132] Any combination of one or more computer-readable media may be used. The computer-readable media may also be computer-readable storage media. As used herein, “computer-readable storage media” encompasses any tangible storage media capable of storing instructions executable by the processor of a computing device. Computer-readable storage media may also be referred to as computer-readable non-temporary storage media. Computer-readable storage media may also be referred to as tangible computer-readable media. In some embodiments, the computer-readable storage media may also be capable of storing data accessible by the processor of a computing device.

[0133] "Computer memory" or "memory" is an example of a computer-readable storage medium. Computer memory is any memory that is directly accessible to the processor. "Computer storage" or "storage" is a further example of a computer-readable storage medium. Computer storage is any non-volatile computer-readable storage medium. In some embodiments, computer storage may also be computer memory, and vice versa.

[0134] As used herein, “processor” encompasses an electronic component capable of executing a program or computer executable instruction or computer executable code. References to computing devices containing a “processor” should be interpreted as likely containing one or more processors or processing cores. A processor may, for example, be a multi-core processor. A processor may also refer to a collection of processors within a single computer system or distributed across multiple computer systems. The term “computing device” should also be interpreted as likely referring to a collection or network of computing devices, each containing one or more processors. Computer executable code may be executed by multiple processors, which may be within the same computing device or distributed across multiple computing devices.

[0135] Computer executable code may include computer executable instructions or programs that cause a processor to perform aspects of the present invention. Computer executable code that performs operations for aspects of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the C programming language or similar languages, and may be compiled into computer executable instructions. In some cases, computer executable code may be in the form of a high-level language or in a pre-compiled form and may be used in conjunction with an interpreter that generates computer executable instructions on the fly.

[0136] Generally, program instructions can be executed on one or more processors. If there are multiple processors, they can be distributed across several different entities. Each processor can execute a portion of the instructions that target that entity. Therefore, when referring to a system or process involving multiple entities, it is understood that a computer program or program instruction is adapted to be executed by the processor associated with or related to each entity. [Explanation of symbols]

[0137] 100 Systems 101 Mobile Devices 102 Communication Link 103 Processing Unit 105 Sensing System 107 Second apparatus 109 Location tracking system 111 First apparatus 500 Location Tracking Systems 501 Mobile Devices 503 Cloud Processing Unit 505 Sensing System 507 Second device 509 Location tracking system 511 First apparatus 1070 equipment 1071 Processor 1072 I / O units 1073 memory

Claims

1. A system comprising a device referred to as a first device, the first device including at least one processor and at least one memory which, when executed by the at least one processor, stores instructions causing the first device to perform a first positioning operation, the first positioning operation is Using a primary pose estimation of the mobile device, a first motion model of the mobile device's motion is used to determine the current state of the mobile device, which is referred to as a first state estimation of the mobile device. To determine at least one of the following: a secondary pose estimation of the mobile device using a first sensor measurement value relating to the mobile device, or a first reference state estimation of the mobile device using a reference motion model and the primary pose estimation; The first estimation error of the first state estimation is determined using at least one of the secondary posture estimation or the first reference state estimation, Based on the first estimation error, trigger an update of the primary attitude estimate in order to use the updated primary attitude estimate for the next execution of the first positioning operation. A system that includes this.

2. The system according to claim 1, wherein when the instruction is executed by the at least one processor, it causes the first device to repeatedly perform the first positioning operation.

3. The system according to claim 1 or 2, wherein the first motion model is adapted to use a first control input of the mobile device for determining the first state estimation, and the reference motion model is adapted to use a second control input of the mobile device for determining the first reference state estimation, the first control input being different from the second control input.

4. The second device further comprises at least one processor and at least one memory which, when executed by the at least one processor of the second device, stores instructions causing the second device to perform a second positioning operation, the second positioning operation is The first motion model is used to determine a second state estimate using the primary pose estimate of the mobile device, To determine at least one of the following: another secondary pose estimation of the mobile device using a second sensor measurement value relating to the mobile device, or a second reference state estimation of the mobile device using the reference motion model and the primary pose estimation; The second estimation error for the second state estimation is determined using at least one of the aforementioned alternative secondary pose estimation or the aforementioned second reference state estimation, Based on the second estimation error, trigger an update of the primary attitude estimate in order to use the updated primary attitude estimate for the next execution of the first or second positioning operation, A system according to any one of claims 1 to 3, including the system described in any one of claims 1 to 3.

5. The system according to claim 4, wherein when the instruction is executed by the at least one processor, it causes the second device to repeatedly perform the second positioning operation.

6. The system according to any one of claims 4 to 5, wherein when the instruction is executed by the at least one processor, the first device further causes the first device to determine whether the primary attitude estimate matches the first state estimate, and in response to the determination that the primary attitude estimate does not match the first state estimate, causes the first device to provide the primary attitude estimate.

7. The system according to any one of claims 4 to 6, wherein, when the instruction is executed by the at least one processor, the first device further causes using a data-driven model to determine whether the primary pose estimate matches the other secondary pose estimate, and in response to the determination that the primary pose estimate does not match the other secondary pose estimate, causes the second device to provide the drift between the primary pose estimate and the other secondary pose estimate provided by the data-driven model, and uses the drift to trigger an update of the other secondary pose estimate.

8. The system according to any one of claims 4 to 7, wherein the first sensor measurement value differs from the second sensor measurement value, the first sensor measurement value includes an external sensor measurement value, and the second sensor measurement value includes an odometry measurement value.

9. The system according to any one of claims 4 to 8, wherein the first device is configured to be remotely connected to the second device, or the first device is configured to be collateral with the second device.

10. The system according to any one of claims 4 to 9, further comprising: a sensing system for acquiring sensor data from the mobile device; and a positioning system configured to determine the primary attitude estimation using the sensor data, provide the primary attitude estimation to the first device, optionally to the second device, or to the mobile device for planning and controlling the motion of the mobile device, wherein the first device is included in the positioning system and the second device is included in the sensing system.

11. The system according to claim 10, wherein the primary attitude estimation is determined by collecting an initial primary attitude estimation estimated using the sensor data, the collection is performed using the speed of the mobile device and at least the transfer delay of the sensor data between the sensing system and the localization system.

12. The system according to claim 10 or 11, wherein triggering the update of the primary pose estimation includes controlling the sensing system to provide current sensor data to the localization system.

13. The system according to any one of claims 4 to 12, wherein when the instruction is executed by the at least one processor of the second device, the second device further synchronizes the updated primary attitude estimate with the first device, or when the instruction is executed by the at least one processor, the first device further synchronizes the updated primary attitude estimate with the second device.

14. It is a method, Using a primary pose estimation of the mobile device, a first motion model of the mobile device's motion is used to determine the current state of the mobile device, which is referred to as a first state estimation of the mobile device. Determining at least one of the following: a secondary pose estimation of the mobile device using first sensor measurements, or a first reference state estimation using a reference motion model and the primary pose estimation; The first estimation error of the first state estimation is determined using at least one of the secondary posture estimation or the first reference state estimation, Based on the first estimation error, trigger an update of the primary pose estimate in order to use the updated primary pose estimate for the next run of the method, Methods that include...

15. A computer program product comprising processor-executable instructions for causing a device to perform the method described in claim 14.

Images