Dead Reckoning for Planar Robots with Optical Flow, Wheel Encoders, and Inertial Measurement Units

Abstract

A system and method are provided for estimating a robot trajectory by fusing multiple robot velocity measurements from multiple robot sensors located within the robot to generate a fused robot velocity based on the accuracy of these robot velocity measurements, and applying Kalman filtering to the fused robot velocity to calculate a current robot location.

Description

[Technical Field]

[0001] Related Applications This application is related to and claims priority to U.S. Provisional Patent Application No. 63 / 073,311, filed September 1, 2020, entitled "PLANAR ROBOTS DEAD-RECKONING WITH OPTICAL FLOW, WHEEL ENCODER AND INERTIAL MEASUREMENT UNIT," the disclosure of which is incorporated herein by reference.

[0002] FIELD OF THE INVENTION Embodiments of the subject matter disclosed herein relate to navigation in planar robots. [Background technology]

[0003] Planar robots move on flat surfaces and are used in applications such as robotic vacuum cleaners. Dead reckoning navigation and simultaneous localization and mapping (SLAM) in planar robots utilize an inertial measurement unit (IMU), optical flow (OF) sensors, and wheel encoders (WE). The IMU typically contains an accelerometer and gyroscope and provides the relative orientation and linear acceleration of the planar robot. The OF sensors and WE provide the speed of the planar robot. Dead reckoning is used to calculate the trajectory of the planar robot by updating the current position estimate with the current speed and previous position of the planar robot. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] https: / / electronics.howstuffworks.com / gadgets / home / robotic-vacuum1.htm [Non-patent document 2] Ronald PM Chan et al., "Characterization of Low-cost Optical Flow Sensors," Proceedings of Australasian Conference on Robotics and Automation, Brisbane, Australia (2010) [Non-patent document 3] "Outdoor downward-facing optical flow odometry with commodity sensors.", In: Howard A, Iagnemma K, Kelly A (eds) Field and service robots, Springer tracts in advanced robotics, vol 62, Springer, Berlin, pp. 183-193. [Non-patent document 4] "Introduction to Autonomous Mobile Robots," R. Siegwart and I. Nourbakhsh, MIT Press (2004) Summary of the Invention [Problem to be solved by the invention]

[0005] Although OF sensors and WEs can provide the speed of a planar robot, the measurements provided by OF sensors and WEs are prone to errors and failures. For example, if the OF image quality is too low, the OF sensor may fail completely. In addition, wheel slippage is a common source of measurement error for WEs. In dead-reckoning navigation, the accuracy of speed estimation determines the rate of position error growth; therefore, fusing sensors to obtain accurate speed estimates is essential for navigating heterogeneous surfaces (e.g., rugs, tiles, wood). Therefore, there is still room for improvement in planar robot navigation. [Means for solving the problem]

[0006] Exemplary embodiments are directed to systems and methods for providing navigation for planar robots. Linear acceleration from an IMU is fused with speed from an OF sensor and a WE to achieve robust and accurate dead-reckoning position estimation. Bad sensor measurements are rejected, or both OF sensor and WE measurements are appropriately combined for accuracy. Measurements from the OF sensor and WE may not match the robot's motion, and errors in the OF sensor and WE depend on driving conditions. However, the IMU's linear acceleration always matches the robot's motion. Additionally, speed derived from linear acceleration suffers from rapidly growing integration errors, but through signal processing, this derived speed becomes comparable to the speed measured by the OF sensor and WE without motion mismatch errors in the feature domain. Additionally, weights for sensor measurements can be derived for their accuracy.

[0007] According to one embodiment, there is a method for estimating a robot trajectory, the method including fusing multiple robot velocity measurements from multiple robot sensors located within the robot to generate a fused robot velocity, and applying Kalman filtering to the fused robot velocity and linear acceleration measured from an inertial measurement unit to calculate a current robot location.

[0008] According to one embodiment, there is a robot configured for estimating a robot trajectory, the robot including at least one processor configured to fuse multiple robot velocity measurements from multiple robot sensors disposed within the robot to generate a fused robot velocity, the at least one processor configured to apply Kalman filtering to the fused robot velocity and linear acceleration measured from an inertial measurement unit to calculate a current robot location.

[0009] The accompanying drawings illustrate exemplary embodiments. [Brief explanation of the drawings]

[0010] [Figure 1]FIG. 1 is a schematic diagram of a robot according to one embodiment. [Figure 2] 1 is a flowchart illustrating a method for estimating a location of a robot according to one embodiment. [Figure 3] 10 is a graph of a DC blocking filter frequency response according to one embodiment. [Figure 4] FIG. 1 is a diagram of velocity fusion and location and orbit estimation according to one embodiment. [Figure 5] 1 is a flowchart of a method for estimating a trajectory of a robot according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] The following detailed description of the present invention refers to the accompanying drawings, in which the same reference numbers in different drawings identify the same or similar elements, and is not intended to limit the present invention. Instead, the scope of the present invention is defined by the appended claims.

[0012] As used herein, variables that appear in bold indicate matrices or vectors. Whether a variable is a matrix or a vector can be inferred from the context.

number

[0013] In general, the following symbols and variables are used throughout this specification: I IMU body frame. i, o, w Location of the IMU, OF, and WE sensors on the robot. U User frame. ω [0 0 ω z ] the angular velocity of the robot.

number

number

number

number

number

number

number

number

number

[0014] Exemplary embodiments provide improved navigation for planar robots. Suitable planar robots include, but are not limited to, robotic vacuum cleaners (RVCs). Generally, RVC systems include two wheels, a left wheel and a right wheel, each driven by two independent motors with their own encoder. A description of how a robotic vacuum cleaner works can be found in "Robot Vacuum Cleaner System," IEEE Transactions on Robotics, Vol. 1, No. 1, pp. 211-214, 2003, the contents of which are incorporated herein by reference in their entirety.

[0015] 1, one embodiment of a robot 100 is shown. The robot includes a left wheel 102 that includes a left wheel encoder (WE) sensor and a right wheel 104 that includes a right wheel (WE) encoder sensor. The robot also includes an optical flow (OF) sensor 106 and an IMU 108. The IMU 108 includes an accelerometer and a gyroscope, and measures acceleration at the location of the IMU.

number

number

[0016] The OF sensor 106 measures pixel movement in the OF body frame, which is the relative distance the sensor moves between two successive register readings. When sampled at a regular period, the OF sensor 106 can be treated as a velocity sensor. The OF sensor 106 is calibrated to measure velocity 122 at the location of the OF sensor 106, which is

number

number

[0017] After calibration, the left and right WE sensors measure the velocity at the location of the left wheel 126 and right wheel 128 in the IMU frame, respectively, which are

number

number

number

number

number

number

[0018] Optical flow sensors measure change in position by optically acquiring successive surface images (frames) using a low-resolution, high-frame-rate camera to capture images of a surface and mathematically determining the speed and direction of movement. Successive images are fed into an image processing algorithm that produces delta X and Y, the change in position during the interval between them. A well-known application of optical flow sensors is in computer mice, where the image sensor and processing algorithm are integrated into a single chip.

[0019] The accuracy of OF measurements depends on the light source and surface texture. A suitable OF sensor 106 for some embodiments is the Pix Art Imaging PAA 5101, which has both a Light Amplification by Stimulated Emission of Radiation (LASER) and a Light-Emitting Diode (LED) light source. When LASER illumination is selected, image quality values ​​are high when moving over tiles and low when moving over carpet. Conversely, when LED illumination is selected, image quality is high when moving over carpet and low when moving over tiles. Because the sensor operates by measuring optical flow, surfaces with differently behaving light intensity patterns will result in inaccurate measurements. Therefore, as described in [2] and [3], specular reflections from surfaces will affect measurements. To provide fusion of measurements from different sensors, it is necessary to reliably detect OF sensor failures and evaluate the accuracy of the OF sensor as it moves over different surfaces.

[0020] Regarding WE accuracy, robots are typically equipped with wheel odometry to measure their travel distance. The accuracy of WE measurements depends on the terrain surface and is subject to many sources of error, as described in [4]. These sources of error include resolution limitations during integration (e.g., time increment, measurement resolution), wheel misalignment (deterministic), wheel diameter imbalance (deterministic), variations in wheel contact points, uneven floor contact (e.g., slippage, non-flatness), and errors caused by robot collisions and collisions. These errors adversely affect the reliability of WE sensor measurements. These deterministic errors can be significantly reduced by dynamic wheel encoder calibration. Other errors require accuracy estimation for sensor fusion.

[0021] The IMU 108 provides the robot's orientation and linear acceleration. Changes in the robot's orientation can be derived from the orientation. Linear acceleration typically includes a zero-gravity offset (ZGO) and a gravity residual. As a result, direct integration of linear acceleration suffers from severe integration errors. However, the accuracy of the IMU's linear acceleration is insensitive to variations in terrain and surface texture. In other words, the linear acceleration measurements are always consistent with the robot's motion. After filtering and signal processing, the linear acceleration derived from the IMU 108 is used to detect failures of the OF and WE sensors and evaluate their accuracy.

[0022] Referring to FIG. 2, an exemplary embodiment of a method for estimating the current position of a robot 200 with good accuracy is shown. The method fuses velocity measurements from multiple robot sensors and uses the fused velocity in a Kalman filter to estimate the current position. When executed over time, the current position defines the robot trajectory. Suitable robots include, but are not limited to, planar robots. In one embodiment, the robot is an RVC. First, each robot sensor in the plurality of robot sensors is calibrated. In one embodiment, a given robot sensor is calibrated in step 202 using a nominal calibration method provided by the original equipment manufacturer (OEM) for that given robot sensor. Alternatively, the given robot sensor is calibrated using an in-house advanced calibration method. Suitable robot sensors include OF sensors and WE sensors, as described above. The calibrated robot sensors are time-aligned in step 204. Robot velocity measurements are obtained from each robot sensor in step 206. In one embodiment, one OF sensor velocity measurement and at least one WE sensor velocity measurement are obtained. In one embodiment, two separate WE sensor velocity measurements are obtained and the two separate WE sensor velocity measurements are averaged to generate a composite WE sensor velocity measurement that is relative to the center of the axis between the left and right wheels, which is often also the center of the robot's body.

[0023] The robot velocity measurements from the multiple robot sensors are then fused or combined in step 208. In one embodiment, this combination is a weighted combination or weighted sum of the multiple robot velocity measurements. The fused robot velocity measurements are then processed by a Kalman filter in step 210, i.e., subjected to Kalman filtering, to estimate the current robot location in step 212. The collection and fusing of robot velocity measurements and processing the fused velocity measurements with Kalman filtering can be repeated iteratively. This generates a history of the current robot location that can be used to determine the robot's trajectory over a given period of time in step 214. In one embodiment, the fused robot velocity measurements are processed by a Kalman filter in combination with linear acceleration obtained from the IMU 108 and expressed in the user frame of reference in step 216 and robot dynamics in step 218.

[0024] Because each robot sensor measures a quantity at its physical location and the IMU 108 measures linear acceleration at the IMU location, in one embodiment, the robot velocity measurements, e.g., the OF sensor velocity measurements and the WE sensor velocity measurements, are converted, transformed, or translated in step 220 from the robot velocity measurements at each robot sensor location to robot velocity measurements at a common location within the robot. In one embodiment, the robot velocity measurements are converted to any common location. In one embodiment, each robot velocity measurement is converted from each robot sensor location to a corresponding robot velocity measurement at the IMU location within the robot. Thus, each robot velocity measurement is translated from the frame of reference of the robot sensor location to the frame of reference of the IMU location.

[0025] Choosing different transformations to convert the robot velocity measurements will result in different transformation errors. In one embodiment, all quantities from the robot velocity measurements are converted to IMU locations, i.e., the OF sensor velocity measurements and the WE sensor velocity measurements are converted to IMU locations. On a rigid body, the velocity at a second location b is calculated from the velocity at a first location a using the rigid body's angular velocity and the displacement from a to b. This is expressed by the following rigid body transformation equation (1):

number

number

number

number

[0026] In one embodiment, the robot velocity measurements, i.e., the OF sensor velocity measurements and the WE sensor velocity measurements, are transformed to IMU locations using rigid body transformation equations. The corresponding velocities at the IMU calculated from the OF and WE sensors are shown in equations (2) and (3).

number

number

number

number

number

[0027] The resulting robot velocity measurements in the IMU frame of reference are then transformed to corresponding robot velocity measurements in the user frame of reference in step 222. The user frame of reference is the frame of reference associated with the user of the robot. In one embodiment, a quaternion or rotation matrix from the IMU provides the rotation from the IMU frame to the user frame or fixed coordinate frame. With such a quaternion q, the velocity at the IMU location is

number

number

number

[0028] Additionally, measurements from the IMU 108 are translated into the user frame of reference. In one embodiment, the linear acceleration from the IMU 108 in the user frame of reference is calculated as shown in equation (6):

number

[0029] The accuracy of sensor measurements from multiple robot sensors, e.g., OF sensors and WE sensors, depends on the driving or operating conditions of the robot 100. The accuracy of one type of robot sensor under given driving or operating conditions may be different from another type of robot sensor under the same driving or operating conditions. Thus, some types of robot sensors will have greater accuracy than other types of robot sensors under common operating conditions. Therefore, the accuracy of each sensor measurement from each robot sensor is determined based on the driving conditions in step 224, and this accuracy is used in determining how to combine or fuse robot velocity measurements for use in Kalman filtering.

[0030] Given that IMU measurements are not affected by or dependent on the robot's driving conditions, in one embodiment, IMU linear acceleration measurements in feature space are used to reliably access or determine the accuracy of multiple robot velocity measurements. Weights to be applied to robot velocity measurements from each robot sensor, the OF sensor and the WE sensor, are calculated with respect to the accuracy associated with these velocity measurements. Robot velocity is derived from the robot's linear acceleration obtained from the IMU in addition to the robot velocity measurements obtained from the robot sensors. Features are defined for the purpose of comparing the velocity obtained from the robot sensors to the velocity calculated from the linear acceleration derived from the IMU. According to some embodiments, features are generated by a DC blocking filter that transforms the velocity signal, which is generated by one of an optical flow sensor, a wheel encoder, or an inertial measurement unit.

[0031] In step 226, equivalent features from the robot velocity derived from linear acceleration and the robot velocity measurements from multiple robot sensors are calculated and compared for different domains, e.g., the velocity domain and the position domain. In step 228, differences between the equivalent features between the IMU and each robot sensor are identified. These differences represent the accuracy of the velocity measurements from each robot sensor and are used to weight the contribution of each robot velocity measurement in the velocity measurement fusion in step 230. Thus, fusing the robot velocity measurements uses a weighted sum of the robot velocity measurements from each robot sensor based on the accuracy of each robot sensor under the robot's current driving or operating conditions.

[0032] Linear acceleration from the IMU is used because it is not subject to errors caused by driving conditions that adversely affect the OF and WE sensors.

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0033] To overcome the difference in the integration due to ZGO and gravity residuals in the acceleration, a DC blocking filter is applied before and after each integration operation so that only the signal changes are kept. The DC blocking filter is a recursive filter specified by the difference equation (9). y(n)=(x(n)-x(n-1))(1+a) / 2+ay(n-1) (9)

[0034] Referring to FIG. 3, a graph 300 of the frequency response for a=0.5, 306, a=0.8, 304, and a=0.95, 302 is shown.

[0035] To obtain equivalent characteristics between velocities derived from the IMU linear acceleration, OF sensor velocity measurements, and WE sensor velocity measurements, DC blocking filters and appropriate transformations are applied to the OF sensor, WE sensor, and IMU velocity measurements. This converts the sensor measurements into transformed signals in the transform domain. The transformed signals are referred to as features of the original signal, i.e., original velocity measurements. These transformed signals, i.e., features, are equivalent, and differences between these features are used to assess the accuracy of the sensor measurements. In one embodiment, the sensor measurements are transformed for multiple transform domains, e.g., the velocity domain and the position domain.

[0036] A well-designed DC blocking filter f DCB (*) is used to calculate the original signal features, i.e., the original sensor measurements, in the velocity and position domains that are used to compare each robot velocity measurement from each robot sensor with the velocity derived from the IMU linear acceleration. First, the IMU linear acceleration

number

number

number

[0037] Innermost f DCB (*)teeth

number

number

[0038] The same DC blocking filter

number

number

number

[0039] After applying the double DC block filter, the DC component and first-order fluctuation component embedded in the original signal are filtered out, and only the higher-order signal fluctuations are kept. The errors caused by ZGO and gravity residuals are greatly reduced. As shown in equations (12) and (13), the following relationship holds for the actual data:

number

[0040] These relationships bridge the gap between theoretical formulation and practical implementation. To further reduce high frequency sensor noise, the velocity is also transformed in the position domain and a DC blocking filter is applied as shown in equations (14)-(16).

number

number

number

[0041] When assessing feature differences, correlation coefficients between the computed features are calculated and used to determine the closeness or similarity between the IMU measurements and each robot sensor measurement, i.e., between the OF measurements and the IMU measurements and between the WE measurements and the IMU measurements. This similarity metric is calculated between the k0+1 feature samples within the most recent t0 seconds as shown in equations (19) and (20).

number

number

number

[0042] λ ioWhen λ is below a threshold, e.g., 0.5, the OF sensor measurement is affected by driving condition errors, e.g., image defects, and λ io When λ is sufficiently high or exceeds a threshold, e.g., 0.95, the OF sensor measurements have acceptable accuracy. iw When is sufficiently low below a threshold, e.g., 0.5, the WE measurement is affected by driving condition errors, and λ iw When λ is sufficiently high above a threshold, e.g., 0.95, the WE measurement has acceptable accuracy. io is sufficiently high or exceeds a threshold, e.g., 0.95, while λ iw When is sufficiently low, or below a threshold, say 0.5, the fusion rate is

number

number

number

number

[0043] To derive the velocity weights associated with the OF and WE sensors, linear acceleration

number

number

number

number

[0044]

number

number

number

number

number

number

number

number

number

number

number

number

number

[0045] Initial Robot Speed

number

number

number

number

number

number

[0046] In one embodiment, the likelihood

number

number

number

number

[0047] For short periods, the following approximation holds even with the gravity residual and ZGO, as shown in equation (27).

number

number

number

number

number

number

number

[0048] Similarly, the WE velocity residual is calculated as shown in equation (29).

number

number

number

number

number

number

number

number

[0049] In another embodiment, the weights are calculated using features in the velocity domain, as defined in equations (32)-(34):

number

[0050] If both the OF sensor and the WE sensor work properly,

number

number

number

number

number

number

number

number

[0051]

number

number

number

number

number

number

number

number

number

number

[0052] In this embodiment, the likelihood is

number

number

number

number

number

number

[0053] After fusing the velocity of the OF sensor and the WE, the robot position is estimated. In the robot dynamics, the robot is assumed to be moving with a constant acceleration. The state is

number

[0054] The dynamics of the robot are k =Fx k-1 +w k and z k =Hx k +n k where F and H are shown below in equations (40) and (41), respectively.

number

[0055] The measurement value is

number

number

number

[0056] The velocity fusion and position estimation described herein are summarized in Figure 4. Furthermore, Figure 4 can be mapped to Figure 2 for many of the steps, i.e., in several steps, and Figure 4 additionally provides details, e.g., various equations and specific inputs described herein, for some of the steps described in Figure 2. Therefore, the interested reader is encouraged to compare Figures 2 and 4 for ease of understanding.

[0057] Initially, flowchart 400 includes inputs from optical flow 402, IMU 404, and wheel encoder 406. Blocks 408 and 410 represent processing the various inputs shown to provide velocity outputs related to WE 406 and OF 402, respectively. The velocity outputs from blocks 408 and 410 are then merged with information from IMU 404, as shown in step 412, which is described as rotating by R into the user frame. Integration into velocity is performed in step 414. A DC blocking filter is then applied before and after each integration operation to ensure only signal fluctuations are retained, as shown in step 416. Calculating feature differences is performed in step 418, followed by calculating weights in step 420. Fusion of the WE 406 and OF 402 velocities is performed in step 424. Block 424 represents inputting these robot dynamics and linear accelerations, and the output is sent to a Kalman filter for filtering, as shown in step 426.

[0058] According to one embodiment, there is a flowchart 500 of a method for estimating a robot trajectory, as shown in Figure 5. The method includes, in step 502, fusing multiple robot velocity measurements from multiple robot sensors located within the robot to generate a fused robot velocity, and, in step 504, applying Kalman filtering to the fused robot velocity and the linear acceleration measured from the inertial measurement unit to calculate a current robot location.

[0059] The embodiments describe fusing linear acceleration from the IMU 404, velocity measurements from the OF sensor 402, and measurements from the WE sensor 406 to estimate robot position. First, the linear acceleration from the IMU is not subject to driving condition-induced measurement errors to which the OF sensor 402 and WE sensor 406 are prone, such as poor quality caused by encoder wheel slippage or floor texture. Second, although linear acceleration is not directly used to evaluate the accuracy of the OF sensor 402 and WE sensor 406, after converting this information to a feature domain, a similarity metric can be used to reject bad OF and / or WE measurements, and the derived weights can be used to fuse the OF and WE measurements. Third, the robot position is estimated via a Kalman filter. The embodiments described herein in practical systems have demonstrated good performance, robustness, and are computationally efficient.

[0060] Systems and methods for processing data according to exemplary embodiments of the present invention can be implemented by one or more processors executing a set of instructions contained in a memory device. Such instructions can be loaded into the memory device from another computer-readable medium, such as a secondary data storage device. Execution of the set of instructions contained in the memory device causes the processor to operate, for example, as described above. In alternative embodiments, hardwired circuitry can be used in place of or in combination with software instructions to implement the present invention. Such software can execute on a processor housed within a sensor-containing device, such as a robot or other device, or the software can execute on a processor or computer housed within another device that communicates with the sensor-containing device, such as a system controller, game console, or personal computer. In such cases, data can be transferred via wire or wirelessly between the sensor-containing device and a device that includes a processor executing software that performs bias estimation and compensation as described above. According to other exemplary embodiments, some of the above processing can be performed on the sensor-containing device, while the remainder of the processing is performed on the second device after receiving partially processed data from the sensor-containing device.

[0061] The above-described exemplary embodiments are intended to be illustrative in all respects of the present invention, rather than limiting. As such, the present invention is susceptible to many variations in its detailed implementation, which can be derived by those skilled in the art from the description contained herein. For example, while the foregoing exemplary embodiments describe, among other things, the use of inertial sensors to detect device movement, other types of sensors (e.g., ultrasonic, magnetic, or optical), in conjunction with the signal processing described above, can be used in place of or in addition to inertial sensors. All such variations and modifications are believed to be within the scope and spirit of the present invention as defined by the following claims. No element, act, or instruction used in the description herein should be construed as critical or essential to the invention unless explicitly described as such. Also, as used herein, the article "a" is intended to include one or more items.

[0062] Although features and elements are described above in specific combinations, those skilled in the art will understand that each feature or element can be used alone or in any combination with the other features and elements. In addition, the methods described herein can be implemented in a computer program, software, or firmware embodied in a computer-readable medium for execution by a computer or processor. Examples of non-transitory computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs).

[0063] Additionally, in the above-described embodiments, reference is made to processing platforms, computing systems, controllers, and other devices that include processors. These devices may include at least one central processing unit ("CPU") and memory. In accordance with the practices of those skilled in the art of computer programming, references to acts and symbolic representations of operations or instructions may be performed by various CPUs and memories. Such acts and operations or instructions may be referred to as being "executed," "computer-executed," or "CPU-executed." For example, the IMU 108 may also include a processor. Alternatively, a processor / processing unit may be located anywhere desired within the robot 100 to perform various calculations, estimations, image processing, etc., as described herein.

[0064] Those skilled in the art will understand that these acts and symbolically represented operations or instructions include the manipulation of electrical signals by a CPU. The electrical system represents the data bits, causing the resulting transformation or reduction of the electrical signals and the maintenance of data bits in memory locations in a memory system, which can reconfigure or otherwise alter the operation of the CPU, as well as other processing of the signals. The memory locations in which the data bits are maintained are physical locations that have particular electrical, magnetic, optical, or organic properties that correspond to or represent the data bits. It should be understood that exemplary embodiments are not limited to the above-mentioned platforms or CPUs, and that other platforms and CPUs can support the provided methods.

[0065] The data bits may also be maintained on computer-readable media, including magnetic disks, optical disks, and any other volatile (e.g., random access memory ("RAM")) or non-volatile (e.g., read-only memory ("ROM")) mass storage systems readable by a CPU. The computer-readable media may include cooperative or interconnected computer-readable media, which reside exclusively on a processing system or are distributed among multiple interconnected processing systems, which may be local or remote to the processing system. It will be understood that exemplary embodiments are not limited to the memories described above, and that other platforms and memories may support the methods described above.

[0066] In an exemplary embodiment, any of the operations, processes, etc. described herein may be implemented as computer-readable instructions stored on a computer-readable medium, which may be executed by a processor of a mobile unit, a network element, and / or any other computing device.

[0067] Little distinction remains between hardware and software implementations of aspects of a system. The use of hardware or software is generally a design choice representing a cost vs. efficiency trade-off (though not always, in that in some contexts the choice between hardware and software can be important). There may be various means by which the processes and / or systems and / or other technologies described herein (e.g., hardware, software, and / or firmware) can be implemented, and the preferred means may vary with the context in which the processes and / or systems and / or other technologies are deployed. For example, if an implementer determines that speed and accuracy are paramount, the implementer may select a primarily hardware and / or firmware means. If flexibility is paramount, the implementer may select a primarily software implementation. Alternatively, the implementer may select some combination of hardware, software, and / or firmware.

[0068] The foregoing detailed description has defined various embodiments of devices and / or processes through the use of block diagrams, flowcharts, and / or examples. To the extent that such block diagrams, flowcharts, and / or examples include one or more functions and / or operations, it will be understood by those skilled in the art that each function and / or operation within such block diagrams, flowcharts, or examples, individually and / or collectively, can be implemented by a wide variety of hardware, software, firmware, or virtually any combination thereof. Suitable processors include, by way of example, general-purpose processors, special-purpose processors, conventional processors, digital signal processors (DSPs), multiple microprocessors, one or more microprocessors associated with a DSP core, controllers, microcontrollers, application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), field-programmable gate array (FPGA) circuits, any other type of integrated circuit (IC), and / or state machines.

[0069] The present disclosure is not limited by the specific embodiments described herein, which are intended as illustrations of various aspects. As will be apparent to those skilled in the art, many modifications and variations can be made without departing from its spirit and scope. No element, act, or instruction used in the description herein should be construed as critical or essential to the invention unless explicitly provided as such. Functionally equivalent methods and apparatuses within the scope of the present disclosure, in addition to those enumerated herein, will be apparent to those skilled in the art from the foregoing description. Such modifications and variations are intended to fall within the scope of the appended claims. The present disclosure should be limited only by the terms of the appended claims, along with the full scope of equivalents to which such claims are entitled. It should be understood that the present disclosure is not limited to any particular method or system.

[0070] In some representative embodiments, portions of the subject matter described herein may be implemented via application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), and / or other integrated formats. However, those skilled in the art will recognize that some aspects of the embodiments disclosed herein may equally be implemented in an integrated circuit, in whole or in part, as one or more computer programs running on one or more computers (e.g., one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as substantially any combination thereof, and that designing circuitry and / or writing code for software and / or firmware would be well within the skill of one of ordinary skill in the art in light of this disclosure. Additionally, those skilled in the art will understand that the mechanisms of the subject matter described herein can be distributed as a program product in various forms, and that one exemplary embodiment of the subject matter described herein applies regardless of the particular type of signal-bearing medium actually used to effect this distribution. Examples of signal bearing media include, but are not limited to, the following: recordable type media such as floppy disks, hard disk drives, CDs, DVDs, digital tape, computer memory, etc., and transmission type media such as digital and / or analog communications media (e.g., fiber optic cables, wave guides, wired communications links, wireless communications links, etc.).

[0071] The subject matter described herein sometimes depicts different components contained within or connected with different other components. It should be understood that such illustrated architectures are merely examples, and that in fact, many other architectures that achieve the same functionality may be implemented. In a conceptual sense, any arrangement of components that achieve the same functionality is effectively "associated" such that the desired functionality can be achieved. Thus, any two components combined herein to achieve a particular functionality can be viewed as "associated" with each other such that the desired functionality is achieved, regardless of the architecture or intermediate components. Similarly, any two components so associated can also be viewed as being "operably connected" or "operably coupled" to each other such that the desired functionality is achieved, and any two components that can be associated in this manner can also be viewed as being "operably couplable" to each other such that the desired functionality is achieved. Specific examples of operably couplable include, but are not limited to, physically matable and / or physically interacting components and / or wirelessly interacting and / or wirelessly interacting components and / or logically interacting and / or logically interacting components.

[0072] With respect to the use of virtually any plural and / or singular term herein, one of ordinary skill in the art can translate from plural to singular and / or from singular to plural as appropriate to the context and / or application. For clarity, various singular / plural permutations may be expressly provided herein.

[0073] In general, it will be understood by those skilled in the art that the terms used in this specification, and particularly in the appended claims (e.g., the body of the appended claims), are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "includes" should be interpreted as "including but not limited to," etc.). If a specific number of introduced claim provisions is intended, such intent will be expressly set forth in the claim; it will be further understood by those skilled in the art that, in the absence of such a provision, no such intent exists. For example, where only one item is intended, the term "single" or similar language can be used. As an aid to understanding, the following claims and / or description herein may include the use of the introductory phrases "at least one" and "one or more" to introduce claim provisions. However, the use of such phrases should not be construed as suggesting that the introduction of a claim provision with the indefinite article "a" or "an" limits any particular claim containing such introduced claim provision to embodiments containing only one such provision, even when the same claim includes the introductory phrase "one or more" or "at least one" and an indefinite article such as "a" or "an" (e.g., "a" and / or "an" should be interpreted to mean "at least one" or "one or more"). The same applies to the use of definite articles used to introduce claim provisions. Additionally, those skilled in the art will recognize that even if a specific number of introduced claim provisions is explicitly specified, such provisions should be interpreted to mean at least the recited number (e.g., the mere provision "two provisions" without other modifiers means at least two provisions, or two or more provisions).Furthermore, when a convention similar to "at least one of A, B, and C, etc." is used, such a configuration is generally intended in the sense that one of ordinary skill in the art would understand this convention (e.g., "a system having at least one of A, B, and C" includes, but is not limited to, systems having A only, B only, C only, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). When a convention similar to "at least one of A, B, or C, etc." is used, such a configuration is generally intended in the sense that one of ordinary skill in the art would understand this convention (e.g., "a system having at least one of A, B, or C" includes, but is not limited to, systems having A only, B only, C only, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). It will be further understood by those skilled in the art that virtually any disjunctive word and / or phrase presenting two or more alternative terms, whether in the specification, claims, or drawings, should be understood to consider the possibility of including one of those terms, either of those terms, or both terms. For example, the phrase "A or B" will be understood to include the possibilities of "A" or "B" or "A and B." Furthermore, the term "any of," followed by a list of multiple items and / or multiple categories of items, as used herein, is intended to include "any of," "any combination of," "any plurality of," and / or "any combination of a plurality of" those items and / or categories of items, individually or in conjunction with other items and / or other categories of items. Also, as used herein, the terms "set" or "group" are intended to include any number of items, including zero. Additionally, as used herein, the term "number" is intended to include any number, including zero.

[0074] Additionally, where features or aspects of the present disclosure are described in terms of a Markush group, those skilled in the art will recognize that the present disclosure is also hereby described in terms of any individual element or subgroup of elements of the Markush group.

[0075] As will be understood by those skilled in the art, for any and all purposes, including with respect to providing a written description, all ranges disclosed herein encompass any and all possible subranges and combinations thereof. Any recited range can be readily recognized as fully descriptive and allows for the same range to be broken down into at least equal halves, thirds, quarters, fifths, tenths, etc. As a non-limiting example, each range discussed herein can be readily broken down into a lower third, middle third, and upper third, etc. As will also be understood by those skilled in the art, all terms such as "up to," "at least," "greater than," "less than," etc., refer to ranges that are inclusive of the stated number and can subsequently be broken down into subranges as described above. Finally, as will be understood by those skilled in the art, a range includes each individual element. Thus, for example, a group having 1 to 3 cells refers to groups having 1, 2, or 3 cells. Similarly, a group having 1 to 5 cells refers to groups having 1, 2, 3, 4, or 5 cells, and so on.

[0076] Also, the claims should not be read as limited to the order or elements provided unless stated to that effect. Additionally, the use of the term "means for" in any claim is intended to invoke 35 U.S.C. 112, paragraph 6 or means-plus-function claim format, and any claim without the term "means for" is not so intended. [Explanation of symbols]

[0077] 100 robots 102 left wheel 104 Right Wheel 106 OF sensor 108 IMU 116 IMU frame 118 User Frames 136 processors

Claims

1. 1. A method for estimating a trajectory of a robot, comprising: fusing multiple robot velocity measurements from multiple robot sensors disposed within the robot to generate a fused robot velocity; applying Kalman filtering to the fusion robot velocity and linear acceleration measured from an inertial measurement unit; Calculating the current robot location; A method comprising:

2. The method of claim 1 , wherein fusing the plurality of robot velocity measurements comprises fusing optical flow sensor velocity measurements and wheel encoder sensor velocity measurements.

3. The method of claim 2 , wherein fusing the optical flow sensor velocity measurements and the wheel encoder sensor velocity measurements comprises calculating a weighted sum of the optical flow sensor velocity measurements and the wheel encoder sensor velocity measurements.

4. 4. The method of claim 3, wherein the step of fusing the optical flow sensor velocity measurements and the wheel encoder sensor velocity measurements further comprises the step of determining weights for the optical flow sensor velocity measurements and weights for the wheel encoder sensor velocity measurements using velocities derived from linear accelerations of an inertial measurement unit.

5. using a velocity derived from the linear acceleration of the inertial measurement unit, converting the optical flow sensor velocity measurements, the wheel encoder sensor velocity measurements, and the velocity derived from the linear acceleration of the inertial measurement unit into transform signals in a transform domain, each transform signal comprising one feature; Identifying a difference between a velocity signature derived from the linear acceleration of the inertial measurement unit and a signature of the optical flow sensor velocity measurement; identifying a difference between a velocity signature derived from the linear acceleration of the inertial measurement unit and a wheel encoder sensor velocity measurement signature; using the determined difference to determine a weight for the optical flow sensor velocity measurements and a weight for the wheel encoder sensor velocity measurements; The method of claim 4, comprising:

6. using the determined difference assigning a value of 1 to a weight of the optical flow sensor velocity measurement and a value of 0 to a weight of the wheel encoder sensor velocity measurement when a similarity metric between the velocity features derived from the linear acceleration of the inertial measurement unit and the optical flow sensor velocity measurement features is at least 0.95 and a similarity metric between the velocity features derived from the linear acceleration of the inertial measurement unit and the wheel encoder sensor velocity measurement features is less than 0.5; assigning a value of 0 to a weight of the optical flow sensor velocity measurement and a value of 1 to a weight of the wheel encoder sensor velocity measurement when the similarity metric between the velocity features derived from the linear acceleration of the inertial measurement unit and the optical flow sensor velocity measurement features is less than 0.5 and the similarity metric between the velocity features derived from the linear acceleration of the inertial measurement unit and the wheel encoder sensor velocity measurement features is at least 0.95; The method of claim 5 , comprising:

7. 6. The method of claim 5, wherein using the determined difference comprises assigning weights using recent samples of linear acceleration extending from a given time and previous velocity measurements at the given time.

8. 6. The method of claim 5, wherein using the identified difference comprises assigning weights in a velocity domain using features of the optical flow sensor velocity measurement, features of the wheel encoder sensor velocity measurement, and velocity features derived from the linear acceleration of the inertial measurement unit.

9. 2. The method of claim 1, wherein applying Kalman filtering further comprises using the linear acceleration of the inertial measurement unit, a weighted combination of the optical flow velocity and wheel encoder measurements, and robot dynamics in the Kalman filtering.

10. The method of claim 1 , wherein the robot is a robotic vacuum cleaner.

11. 1. A robot configured for estimating a robot trajectory, comprising: at least one processor configured to fuse multiple robot velocity measurements from multiple robot sensors disposed within the robot to generate a fused robot velocity; Including, the at least one processor is configured to apply Kalman filtering to the fusion robot velocity and the linear acceleration measured from an inertial measurement unit; the at least one processor is configured to calculate a current robot location; robot.

12. The robot of claim 11 , wherein fusing the plurality of robot velocity measurements includes the at least one processor fusing optical flow sensor velocity measurements and wheel encoder sensor velocity measurements.

13. 13. The robot of claim 12, wherein fusing the optical flow sensor velocity measurements and the wheel encoder sensor velocity measurements includes the at least one processor calculating a weighted sum of the optical flow sensor velocity measurements and the wheel encoder sensor velocity measurements.

14. 14. The robot of claim 13, wherein fusing the optical flow sensor velocity measurements and the wheel encoder sensor velocity measurements further includes the at least one processor determining weights for the optical flow sensor velocity measurements and weights for the wheel encoder sensor velocity measurements using velocities derived from linear accelerations of an inertial measurement unit.

15. Using a velocity derived from the linear acceleration of the inertial measurement unit the at least one processor converts the optical flow sensor velocity measurements, the wheel encoder sensor velocity measurements, and the velocity derived from the linear acceleration of the inertial measurement unit into a transform signal in a transform domain, each transform signal including one feature; the at least one processor determining a difference between a velocity signature derived from the linear acceleration of an inertial measurement unit and a velocity signature of an optical flow sensor; the at least one processor determining a difference between a velocity characteristic derived from the linear acceleration of the inertial measurement unit and a characteristic of a wheel encoder sensor velocity measurement; the at least one processor using the determined difference to determine weights for the optical flow sensor velocity measurements and weights for the wheel encoder sensor velocity measurements; The robot of claim 14 , comprising:

16. Using the identified difference the at least one processor assigns a value of 1 to a weight of the optical flow sensor velocity measurement and a value of 0 to a weight of the wheel encoder sensor velocity measurement when a similarity metric between the velocity features derived from the linear acceleration of the inertial measurement unit and the optical flow sensor velocity measurement features is at least 0.95 and a similarity metric between the velocity features derived from the linear acceleration of the inertial measurement unit and the wheel encoder sensor velocity measurement features is less than 0.5; the at least one processor assigns a value of 0 to a weight of the optical flow sensor velocity measurement and a value of 1 to a weight of the wheel encoder sensor velocity measurement when the similarity metric between the velocity features derived from the linear acceleration of the inertial measurement unit and the optical flow sensor velocity measurement features is less than 0.5 and the similarity metric between the velocity features derived from the linear acceleration of the inertial measurement unit and the wheel encoder sensor velocity measurement features is at least 0.95; The robot of claim 15 , comprising:

17. 16. The robot of claim 15, wherein using the determined difference includes the at least one processor assigning weights using recent samples of linear acceleration extending from a given time and previous velocity measurements at the given time.

18. 16. The robot of claim 15, wherein using the determined difference includes the at least one processor assigning weights in a velocity domain using the optical flow sensor velocity measurement features, the wheel encoder sensor velocity measurement features, and velocity features derived from the linear acceleration of the inertial measurement unit.

19. 12. The robot of claim 11, wherein applying Kalman filtering further comprises using the linear acceleration of the inertial measurement unit, a weighted combination of the optical flow velocity and wheel encoder measurements, and robot dynamics in the Kalman filtering.

20. The robot of claim 11 , wherein the robot is a robotic vacuum cleaner.

21. 20. The method or robot of claim 5, 6, 8, 15, 16, or 18, wherein the features are generated by a DC blocking filter that converts a velocity signal, the velocity signal being generated by one or more of the optical flow sensor, the wheel encoder, or the inertial measurement unit.

Images