Method of estimating metric of interest related to motion of subject

Abstract

There is disclosed a computer-implemented method performed in a tracking system that tracks motion of a subject that varies over time, comprising: during a first time period, obtaining first data relating to motion of the subject from at least one primary positioning unit; obtaining second data from the one or more auxiliary sensors during the first time period; generating first training data comprising the first data and the second data, the first training data comprising orientation information of the first platform and the one or more second platforms, and at least one training metric relating to motion of the subject during the first time period; obtaining, during the second time period, third data from the one or more auxiliary sensors, the third data comprising measurements from which the position or movement can be determined; and analyzing the third data using a first algorithm trained with the first training data to estimate at least one first metric related to motion of the subject during the second time period.

Description

[0001] This application is a divisional application with an application date of October 11, 2019, application number 201980077769.4, and invention name “Method for estimating a metric of interest associated with subject motion”. The parent application is an application entering the Chinese national phase of international application number PCT / GB2019 / 052901 submitted to the International Patent Office on October 11, 2019, claiming priority to application number 1816655.3 submitted to the UK Patent Office on October 12, 2018 and application number 1905586.2 submitted to the UK Patent Office on April 18, 2019. The above applications are fully incorporated herein by reference. Technical Field

[0002] The present invention relates to a method and system for estimating a metric of interest related to subject motion. The present invention has particular application in the field of tracking and navigation. Background Art

[0003] Conventional inertial navigation systems use standard mechanical equations to convert measurements from inertial sensors (e.g., accelerometers, gyroscopes, etc.) into tracking and navigation data such as velocity, orientation, and position. A known problem with such systems is numerical integration errors, which manifest as rapidly increasing errors in the derived tracking and navigation data. Inertial navigation equations typically involve determining orientation, velocity, and position by numerical integration of continuous rotation rate and acceleration measurements. The integration of measurement errors (e.g., due to sensor noise, bias, scale factor, and alignment errors) results in an ever-increasing velocity error, which in turn is accumulated into an even more rapidly increasing position error. The impact of numerical integration errors is particularly pronounced in the case of low-cost inertial measurement units (IMUs), whose sensors are relatively noisy, have large alignment and scale factor errors, and have biases that may drift significantly over time or over temperature.

[0004] The availability of low-cost IMUs has become more widespread, and as a result, positioning, navigation and tracking systems have become more common, for example, in smart devices such as smartphones and fitness trackers. However, these low-cost IMUs are typically of relatively low quality, which means that error drift is often large, providing spurious and useless results to the user. Global Navigation Satellite System (GNSS) receivers (such as GPS and GLONASS and Galileo) can be used in combination with an IMU to help improve the accuracy of the navigation solution. However, GNSS coverage is not always readily available (e.g. inside buildings), and GNSS units themselves may provide inaccurate navigation data in certain situations, such as in "urban canyons" where tall buildings may block visibility of the satellite constellation.

[0005] In addition, smart devices such as smartphones can be carried in a variety of different positions and orientations relative to the user, and the user may frequently change his or her movement pattern. For example, a smartphone user may start a journey by walking with the smartphone in his pocket. Upon receiving a call, the phone may be moved to the user's ear, during which the user must run to catch a bus, after which the user rides the bus for the remainder of the journey. Compared to, for example, a "strapdown" system (where a highly accurate IMU is provided in a fixed position and orientation relative to its host body) or a gimballed system, the user's movement and such changes in the relative position and orientation of the device relative to the user may introduce further errors into the final navigation solution.

[0006] However, users of tracking and navigation systems using low-cost IMUs still require accurate and reliable positioning, tracking and navigation information, and therefore there is a need in the art to address the above problems. In particular, there is a need to provide accurate positioning, tracking and navigation systems in situations where GNSS positioning data is inaccurate or unavailable. Summary of the invention

[0007] According to a first aspect of the present invention, there is provided a computer-implemented method performed in a tracking system for tracking the movement of a subject over time, the method comprising: (a) obtaining first data related to the movement of the subject from at least one main positioning unit during a first time period, wherein the at least one main positioning unit is mounted on a first platform carried by the subject, or wherein the at least one main positioning unit is separated from the subject, and the main positioning unit is operative during the first time period; (b) obtaining second data from one or more auxiliary sensors during the first time period, wherein the one or more auxiliary sensors are configured to perform measurements based on which position or movement can be determined, and wherein the one or more auxiliary sensors are mounted on one or more second platforms carried by the subject; (c) generating first training data including the first data and the second data; (d) obtaining third data from the one or more auxiliary sensors during a second time period, and (e) analyzing the third data using a first algorithm trained using the first training data to estimate at least one first metric related to the movement of the subject during the second time period.

[0008] The first data is usually obtained by a main positioning unit mounted on a first platform carried on the main body, and the second data is usually obtained by one or more auxiliary sensors mounted on a second platform carried on the main body. The second data can be obtained by auxiliary sensors mounted on multiple second platforms carried on the main body. The first platform and the second platform are separated from each other. In other words, they are physically separated and are usually different devices. For example, and as will be explained in more detail herein, the first platform can be a smartphone including a GNSS sensor acting as a main positioning unit and carried on the main body, and one or more second platforms can be a headset, wearable glasses, a foot-worn platform, or a wrist-worn watch or a fitness tracker including an inertial sensor suite acting as an auxiliary sensor.

[0009] The present invention advantageously allows the second platform(s) to provide a reliable estimate of a first metric of interest using data obtained only from its auxiliary sensors, using an algorithm trained using the first training data. As will be explained in more detail below, the first metric is typically used to constrain a tracking solution (e.g., trajectory) estimated by analyzing the third data. The use of at least one first metric (which may be referred to as a "motion model" of the subject during the second time period, or a portion thereof) allows an improved tracking solution to be obtained, compared to conventional techniques (e.g., dead reckoning) that use such auxiliary sensors, which are typically only able to provide reliable tracking information for a few seconds before error drift renders the estimate unusable.

[0010] The main positioning unit may be separate from the main body ("external to the main body"); in other words, the main positioning unit is not carried by the main body. For example, the main positioning unit may be an external camera (e.g., one or more CCTV cameras). Also in this case, the main positioning unit and the one or more second platforms are independent of each other.

[0011] In some embodiments, the first data is obtained from at least one main positioning unit mounted on the first platform and from at least one main positioning unit separate from the main body.

[0012] In general, a subject is any subject whose movement may be desired to be tracked. The subject is typically a human user, i.e. a pedestrian, but in embodiments the subject may be a vehicle, such as a car or a bicycle.

[0013] Typically, at least a portion of the second time period does not overlap with the first time period. In other words, typically, the first time period and the second time period do not extend between the same moments. Preferably, the first time period and the second time period do not overlap. Typically, the second time period is in the future relative to the first time period, and thus, the second time period occurs after the first time period. However, the first time period may be in the future relative to the second time period, for example when estimating a metric during the second time period in a post-processing scenario.

[0014] The first time period refers to any time period during which the main positioning unit is operating, which means that the main positioning unit is able to provide measurements and therefore reliable first data can be obtained for providing first training data. The second time period is usually (although not always) any time period during which measurements from the main positioning unit are not available or are considered unreliable. In an embodiment, during the second time period, the first platform on which the main positioning unit is mounted may not be carried by the subject. During the second time period, the first algorithm is required in order to estimate the desired first metric using data obtained from one or more auxiliary sensors.

[0015] In general, the first time period and the second time period may occur in either order, may be consecutive or non-consecutive, and may or may not be from the same "trip" taken by the subject. The first time period may vary in length from the order of seconds (e.g., 10 seconds) to an hour or more. In contrast to conventional techniques (such as dead reckoning) that can only provide a good estimate of the metric of interest for a few seconds, the present invention allows a good estimate of the first metric during the second time period, which may vary from a few seconds to a few hours.

[0016] As used herein, one or more auxiliary sensors are "auxiliary" sensors because they do not directly provide positioning, tracking, or navigation data; instead, they provide measurements from which position or movement can be inferred. At least one auxiliary sensor typically includes an inertial sensor, such as an accelerometer (typically suitable for measuring linear acceleration) or a gyroscope (typically suitable for measuring rotational speed). Other examples of such auxiliary sensors include magnetometers (typically suitable for measuring heading reference) and pressure sensing devices such as barometers (typically suitable for measuring changes in altitude). Another example of an auxiliary sensor that can be used is a camera that is not part of a dedicated camera-based or visual range-based positioning system. In some embodiments, the auxiliary sensor can use a signal received from a ground transmitter (e.g., a cellular transmitter) to infer heading and / or velocity, for example by measuring the Doppler shift of a signal received from a transmitter whose orientation relative to the sensor is known. The at least one auxiliary sensor is typically part of an inertial navigation system.

[0017] The present invention is particularly suitable for using micro-electromechanical system (MEMS) sensors as auxiliary sensors. MEMS sensors are generally low-cost and low-quality, resulting in large error drift, and are not generally used in strapdown and gimbaled systems. However, their low cost and wide availability make them commonly used in platforms such as smartphones and fitness trackers, for which the present invention has particular application.

[0018] In contrast, the primary positioning unit typically provides position and navigation data directly based on information received from a source external to the unit. An example of a primary positioning unit is a GNSS sensor that receives external information from its corresponding satellite constellation. Other examples of primary positioning units include units adapted to provide positioning measurements based on local location identification techniques (such as WiFi fingerprinting or visual identification methods) or using signals from other navigation and tracking systems (such as those provided by the LORAN system). During the first time period, the primary positioning unit is operational, meaning that it is able to provide credible and reliable positioning and navigation data. This is typically because the primary positioning unit can receive the required external information during the first time period. Typically, the primary positioning unit is operational during the entire first time period. For example, in the case where the primary positioning unit includes a GNSS sensor mounted on a first platform carried by the subject, during the first time period, the platform is typically located at a position where the GNSS sensor has good visibility of the satellite constellation, and therefore reliable speed, position, and timing information can be provided.

[0019] In an embodiment, the first data and the second data may be obtained based on at least one auxiliary sensor and at least one main positioning unit using simulation techniques.

[0020] One or more auxiliary sensors are mounted on one or more second platforms. Typically, the second platform includes only auxiliary sensors and does not include a primary positioning unit. Examples of second platforms that can be used for the present invention include headphones, wearable glasses, wearable devices (smart watches, fitness trackers and other jewelry), and foot-worn platforms.

[0021] Typically, during the second time period, the primary positioning unit is inoperative. For example, in the case where the primary positioning unit includes a GNSS sensor, during the second time period, the GNSS sensor mounted on the first platform does not have sufficient satellite constellation visibility. In an embodiment where the primary positioning unit (e.g., a closed-circuit television system) is separate from the subject, the second time period may be a time period during which the subject is no longer within (e.g., visual) range of the primary positioning unit. Thus, typically, during the second time period, only third data is obtained from one or more auxiliary sensors. In fact, in an advantageous embodiment, during the first time period, both the first platform and the second platform may be carried on the subject, or the subject is within (e.g., visual) range of a separate primary positioning unit so that training data may be generated. During the second time period, the subject may carry only the second platform, wherein only third data obtained from one or more auxiliary sensors is used to estimate the metric during the second time period. This also exploits the fact that auxiliary sensors such as accelerometers and gyroscopes (and other inertial sensors) are typically updated faster than GNSS units (on the order of 100 Hz for inertial sensors, compared to typically 1 Hz for GPS), enabling high time resolution estimates of the metric of interest during the second time period.

[0022] Typically, the at least one first metric estimated during the second time period is at least one of: direction of motion, rate, speed, the subject's motion environment, and the second platform's position environment relative to the subject. Thus, the first metric of interest may be (and in a preferred embodiment is) the subject's velocity. The at least one first metric may be considered a "motion model" of the subject, and may typically include any parameter that quantitatively represents an aspect of the subject's motion (e.g., step length).

[0023] The first metric may be a desired solution; for example, it may be the velocity of a subject that one wishes to know at a particular time. However, particularly advantageously - and as will be described in more detail below - the method of the present invention allows estimating the evolution of a metric of interest (typically a trajectory) relating to the subject's motion during a second time period by further analysis of third data constrained by a series of first metrics obtained using the first algorithm during the second time period. In other words, the present invention uses a trained machine learning algorithm to estimate a motion model of the subject during the second time period, which motion model can be used to constrain an estimate of the evolution of the metric of interest (e.g., trajectory) during that time period. Compared to conventional techniques (e.g., dead reckoning) that can only provide a good trajectory estimate for a few seconds due to error drift, the present invention allows a good estimate of such a trajectory during a second time period varying from a few seconds to a few hours.

[0024] Using a trained machine learning algorithm to estimate the first metric (motion model) during the second time period is particularly advantageous over conventional techniques that attempt to define the motion model using data obtained only during the current period. For example, by training the first algorithm using training data obtained over multiple "trips" previously taken by the user, the motion model of the user may become more accurate over time, thereby allowing a more accurate estimate of their trajectory during the second time period.

[0025] The inventors have found that the method of the present invention provides an excellent estimate of the trajectory of the subject during the second time period in the absence of a primary positioning unit, particularly in the case where the subject and the (one or more) second platform are not in a conventional strapdown or gimbaled arrangement. In other words, the method of the present invention provides particular advantages in scenarios when the subject and the (one or more) second platform have an unpredicted motion relationship during the second time period. It can be seen that the (one or more) second platform is carried by the main motion of the subject and can move independently relative to the main motion of the subject. For example, the present invention finds particular application in scenarios where the subject is a pedestrian and the second platform is a wearable health tracker worn by the pedestrian or a headset worn by the pedestrian. In such scenarios, although the second platform is carried by the pedestrian, it can move independently relative to the main motion of the pedestrian. For example, a health tracker worn on the wrist can move in response to the pedestrian's arm movement, which arm movement is different from the direction in which the pedestrian is walking. Similarly, the headset can move with the pedestrian's head movement, but can be independent of the pedestrian's direction of motion. In particular, the second platform is independently movable relative to a fixed point on the subject. The fixed point is typically the center of mass of the subject. Typically, the metric of interest is estimated with respect to a fixed point of the body (eg, the center of mass).The second platform may be said to be "flexibly carried" by or "flexibly connected to" the body.The second platform may be said to be arbitrarily oriented relative to the body.

[0026] One or more secondary platforms may be connected to the main body in a temporary manner. The term "temporary" is used herein to mean that the platform is easily movable (engaged / disengaged) relative to the main body, for example the secondary platform may be carried by a pedestrian for a short period of time (i.e., temporarily carried), or attached to a moving object such as a vehicle using a mount designed for temporary mounting.

[0027] In some preferred embodiments, the first metric estimated in step (e) using the first algorithm is only the direction of motion of the subject, wherein the velocity of the subject is estimated from the third data using a technique such as regression or integration of data obtained from at least one auxiliary sensor during the second time period. As will be further described herein, using the first algorithm to estimate only the direction of motion in step (e) advantageously reduces the use of computer resources compared to using the first algorithm to estimate other metrics of interest during the second time period.

[0028] Typically, a first algorithm may be used to estimate the direction of motion in a sensor reference frame, i.e., a coordinate system attached to the center of mass of the auxiliary sensor or to a second platform to which the auxiliary sensor is mounted. If the orientation of the sensor reference frame relative to the navigation reference frame is known or can be determined, this metric (i.e., the direction of motion in the sensor reference frame) may then be used to determine the direction of motion of the subject in the navigation reference frame (i.e., the coordinate system through which the subject is moving - typically the earth's coordinate system). This offset between the sensor and the navigation reference frame may be determined by using a magnetometer and / or gyroscope used as an auxiliary sensor in a second time period. The magnetometer and / or gyroscope may provide a heading in the navigation reference frame of the second platform to which the auxiliary sensor(s) are mounted, and thus the offset between the sensor and the navigation reference frame may be measured.

[0029] Alternatively, the direction of motion in the navigation reference frame (e.g., determined based on GNSS if the primary positioning unit is active during the second time period) can be aligned with the estimated direction of motion in the sensor reference frame to determine the offset between the navigation and sensor reference frames (and therefore the orientation of the second platform) without the need for a magnetometer / gyroscope.

[0030] It is a particular advantage of the present invention to use the first algorithm to estimate the direction of motion in the sensor reference frame (which can then be transformed to the navigation reference frame). This allows the velocity of the subject to be inferred (and input into the first algorithm) from a window or "snapshot" of data obtained from at least one auxiliary sensor during the second time period, without knowledge of past or future measurements or system states.

[0031] For the sake of completeness, it should be noted that the first algorithm can be trained to estimate the orientation of the second platform in the navigation reference frame (and therefore the direction of motion), for example by estimating the roll, pitch and yaw of the second platform based on accelerometer, magnetometer and gyroscope measurements obtained during the second time period.

[0032] Particularly preferably, the method further comprises analyzing the third data to estimate the evolution of at least one second metric associated with the motion of the subject during the second time period, wherein the evolution of the at least one second metric is constrained by at least one first metric estimated using the first algorithm ("motion model"). Typically, the second metric is at least one of position, orientation and velocity. The analysis of the third data (i.e., obtained from one or more auxiliary sensors) to estimate the evolution of the at least one second metric typically first comprises conventional inertial navigation techniques to infer the second metric from the data obtained by the at least one auxiliary sensor. For example, data from an accelerometer may be integrated (while correcting for gravity) to infer velocity, and integrated again to infer position. The estimate of the second metric is then constrained (i.e., constrained by the motion model) using the estimate of the first metric obtained using the first algorithm. For example, analyzing the third data may comprise comparing the third data with the at least one first metric to obtain corrected third data, and wherein the estimate of the evolution of the second metric is based on the corrected third data. Preferably, obtaining the corrected second data comprises determining a measurement bias of the at least one auxiliary sensor, and correcting the measurement bias to obtain the corrected second data. In a preferred embodiment, the estimate of the second metric is constrained by the at least one first metric using a Kalman filter.

[0033] The third data are analyzed in order to estimate an evolution of at least one second metric during the second time period. "Evolution" refers to a change in the second metric over time. Particularly preferably, the third data are analyzed in order to estimate the trajectory of the subject during the second time period (i.e. the evolution of the subject's position over time during the second time period). In case the goal is to track the subject's position in a navigation reference system, it is usually sufficient to track the position of an auxiliary sensor carried by the subject if the position of the auxiliary sensor meets the desired positioning accuracy. For example, the position of an auxiliary sensor mounted on a headset worn by a pedestrian or mounted on a portable tracking device is usually a reliable indication of the subject's position.

[0034] Thus, a particular advantage of the present invention is that a tracking solution for the subject during a second time period can be obtained even when the primary positioning unit is not operational or even not carried by the subject, and in particular in scenarios where the (one or more) second platforms and the subject are not in a conventional strapdown or gimbaled relationship. For example, the primary positioning unit comprises a GNSS sensor mounted on a first platform carried on the subject and the subject (e.g. a pedestrian) moves from an area with good satellite visibility to an area with poor or zero satellite visibility (such as indoors or within an "urban canyon"); during the time of poor or zero satellite visibility (the second time period), data obtained from the auxiliary sensor mounted on the (one or more) second platform and the trained first algorithm can therefore be used to maintain an accurate track of the subject.

[0035] Traditionally, efforts to estimate the evolution of a subject's motion-related metric of interest using only data from auxiliary sensors (e.g., inertial sensors) in a non-strapdown or non-gimbaled setup have proven to be unsatisfactory. Together with the fact that widely available low-cost inertial sensors are typically of low quality that leads to particularly rapid accumulation of errors in dead reckoning techniques, the typical motion patterns of subjects such as pedestrians are not conducive to accurate estimation of motion when the data is obtained in a non-strapdown or non-gimbaled manner from sensors on a platform carried by the subject. Using the headset described above as an example, when the wearer turns his or her head, data from an inertial sensor such as an accelerometer may suggest that the pedestrian has undergone a sudden acceleration in the direction of the head's rotation. However, in reality, the pedestrian is likely to maintain a constant direction and rate of motion.

[0036] The present invention at least partially overcomes this problem by being robust to such "environmental changes" of the first and second platforms. This is particularly advantageous during the second time period.

[0037] Environments can be categorized into "location environments" and "motion environments." The motion environment refers to the manner in which the subject moves, for example, described as being stationary, walking, running, sprinting, jumping, dancing, kicking a ball, playing a game with a racket, side-stepping, backing up, crawling, going up / down stairs, riding an escalator, riding an elevator, climbing a ladder, riding a bicycle, traveling by car, etc. Other motion environments will be readily understood by those skilled in the art.

[0038] The location environment is the manner in which the platform is carried by the subject, and generally includes the location of the platform on the subject. Two location environments are different if the platform moves through space in different ways. For example, the left foot and the right foot will move very similarly, and therefore the platform mounted on the right foot or the left foot can be considered to have the same location environment. However, the platform mounted on the dashboard of a car moves through space in a different manner than the platform mounted on the foot, so the platform mounted on the dashboard of a car will be considered to have a different location environment. As a result of the subject's movement, the location environment involves the precise movement of the platform in space over time. Examples of location environments include: in the hand / wrist in front of the subject; in the hand / wrist swinging to the side of the subject; in the hand / on the subject near the ear; upper arm; head; center of the shoulder / back / waist; buttocks; side pocket of pants; back pocket of pants; jacket pocket; backpack; handbag; thigh / knee; ankle / foot; mounted on a vehicle, etc. Those skilled in the art will readily understand other location environments.

[0039] The motion environment and location environment can be identified by patterns in the acquired data (training data and / or third data acquired during the second time period). For example, regular periodic accelerometer and gyroscope signals characteristic of a user's gait and (if the primary positioning unit is operational) GNSS velocity measurements indicating a rate of approximately 1 to 10 m / s will infer a motion environment of walking or running. As another example, for a user in a moving vehicle, noise-like signals from the accelerometer and gyroscope due to vehicle vibrations and GNSS velocity measurements between 5 and 120 km / h will typically be observed.

[0040] As another example, consider the location environment of a mobile phone in a pocket and the location environment of a mobile phone held near the ear. In both cases, we will observe periodic (approximately sinusoidal) accelerometer and gyroscope time series signals, and the measured acceleration and rotation are characterized by pedestrian gait. However, the amplitude of the acceleration and rotation measured by the mobile phone in the pocket will likely be greater than that of the mobile phone held at the ear, because the mobile phone in the pocket swings back and forth with the legs, while the mobile phone at the ear maintains approximately the same pitch and roll throughout the gait cycle. In addition, the mobile phone in the trouser pocket will experience asymmetric acceleration when the left and right feet hit, where the peak acceleration experienced when the foot hits is larger on the same side of the body as the mobile phone. In contrast, the mobile phone held at the ear will show approximately symmetrical acceleration when the left and right feet hit because it is held closer to the center line of the body. Light sensors can also be used to help distinguish whether it is in a pocket or kept next to the ear. We may always expect the mobile phone to be in a dark place in the pocket, while during the day or under artificial light, the mobile phone at the ear will detect a relatively bright light level (assuming that the light sensor is not pressed against the earlobe).

[0041] The above examples illustrate clear differences between time series signals that are easily discernible even by a human observer.As will be appreciated, during training of the first algorithm using the first training data, the first algorithm may identify other less obvious features and combinations thereof that allow for robust distinctions between environments.

[0042] The inventors have found that implementations where the first algorithm includes a neural network are particularly robust to changes in position and motion environments. Therefore, in a preferred embodiment, the first algorithm is a neural network. This "robustness" is typically achieved by obtaining first data and second data for a plurality of motion environments of the subject, wherein the motion environment of the subject during the second time period substantially corresponds to the motion environment of the subject during the first time period. In addition, the method typically includes obtaining first data and second data for a plurality of position environments of one or more second platforms relative to the subject, wherein the position environment of the one or more second platforms relative to the subject during the second time period substantially corresponds to the position environment during the first time period.

[0043] In other words, in a preferred embodiment, a neural network is trained using first training data that spans an expected range of motion and positional context, particularly for one or more second platforms. When third data is obtained within the range of the context, the trained neural network can be used to estimate at least one first metric associated with the motion of the subject during the second time period, wherein the first metric is at least one of a direction of motion, a rate, a velocity, the motion context of the subject, and the positional context of the second platform relative to the subject. The at least one first metric can then be used to constrain a tracking solution for the subject during the second time period.

[0044] This is a particularly advantageous feature of the invention, since neither the training data nor the tertiary data need to be explicitly categorized ("labeled") with corresponding motion and positional contexts.

[0045] However, in some embodiments, the third data may be analyzed to determine at least one of: (i) the position environment of one or more second platforms relative to the subject during the second time period, and (ii) the motion environment of the subject during the second time period. This clear classification may be advantageous, particularly when a first algorithm other than a neural network is used (such as an algorithm based on support vector machines and support vector regression classification techniques). In some embodiments, the first algorithm used in step (e) is selected from a set of predetermined algorithms for estimating at least one first metric, wherein the selection is based on the position environment determined during the second time period and the motion environment determined. Based on the determination of the motion environment and the position environment during the second time period, the first algorithm may be selected in step (e) based on the best maximum likelihood method to estimate the first metric associated with the motion of the subject. In such an embodiment, the first metric is typically at least one of a direction, rate, or speed of motion. The selection of the first algorithm is typically based on a lookup table of motion and position environments and their respective maximum likelihood methods.

[0046] In such embodiments where the third data is analyzed to determine the location and / or motion environment, the determination may be performed by a second and / or third machine learning algorithm (e.g., different from the first machine learning algorithm). Such a second and / or third algorithm is typically a machine learning algorithm based on, for example, a neural network and / or a support vector machine classifier trained on corresponding training data, or a naive Bayes classifier or related techniques. Preferably, the previously acquired training data is obtained from one or more auxiliary sensors, or from one or more sensors of the same type as the auxiliary sensor(s) carried by the subject, and the data has been labeled into different motion and / or location environments. The labeling is typically manual. For example, the previously acquired training data for determining the location environment may include a plurality of data subsets obtained from at least one inertial sensor, each subset having been obtained in a different location environment such as "in a pocket", "in the hand", etc., and manually labeled as such. Similarly, the previously acquired training data for determining the motion environment may include a plurality of data subsets from at least one inertial sensor, each subset having been obtained in a different motion environment such as "walking", "running", etc. and labeled as such.

[0047] As yet another alternative, the motion and location context may be provided via application programming interface (API) calls to an operating system (e.g., Google ) to determine.

[0048] In an embodiment, at least one of the location environment and the motion environment is determined during the second time period based on the size of the data obtained from one or more auxiliary sensors. For example, the second platform may include a three-axis accelerometer and a three-axis gyroscope used as auxiliary sensors. At least one of the location environment and the motion environment during the second time period can be determined by reducing the three-axis data stream to just the total size of the acceleration or rotation. This advantageously reduces the dimensions, working time, and processing power required to perform the determination of the motion and location environment during the second time period, and thus reduces the dimensions, working time, and processing power required to estimate the first metric.

[0049] In some embodiments, the first algorithm in step (e) is selected from a predetermined set of algorithms for estimating at least one first metric, wherein the selection is based on the at least one first metric to be estimated. For example, if the first metric to be estimated is a direction of motion, one algorithm may be used, while if the first metric to be estimated is a velocity (without a direction of motion), a different algorithm may be selected.

[0050] Preferably, in step (e), the third data is arranged as a plurality of frames, each frame comprising a plurality of measurements from one or more auxiliary sensors in time order, and wherein a first algorithm is used to provide an estimate of at least one first metric for each of the frames. For example, for an accelerometer sampling at a rate of 200 times per second, a "frame" of data may represent one second of motion and include 200 measurements. In step (e), the first metric may be estimated for each frame of data obtained from at least one auxiliary sensor during the second time period.

[0051] Therefore, in a preferred embodiment in which the third data are further analyzed to estimate the evolution of a second metric (e.g., the trajectory of the subject) during a second time period, multiple position estimates obtained from at least one auxiliary sensor are constrained by an estimate of the first metric that has been obtained using the first algorithm for each of the frames (or "windows") of the second data.

[0052] Furthermore, preferably, in step (c), the second data is arranged as a plurality of frames, each frame comprising a plurality of measurement values ​​from at least one auxiliary sensor in time sequence, and wherein the time length of the frames of the second data and the frames of the third data are substantially the same. In other words, when the trained first algorithm is used to estimate the first metric, a frame length of the data obtained from one or more auxiliary sensors that is the same as the frame length used during the second time period is used to train the first algorithm.

[0053] The frame length is typically determined by the typical time scale of variation (periodic or otherwise) in the auxiliary sensor data, which is characteristic of the type of motion. For example, in some embodiments, a frame size of the order of 1 second may be used with accelerometer measurements from a walking pedestrian, since the stride rate is approximately 1 Hz, where a frame of 1 second in length is expected to contain sufficient information to allow a reliable estimate of the first metric. Each of the multiple frames (e.g., of the second data and the third data) may have the same (e.g., fixed) time length, e.g., 1 second as described above.

[0054] Where the first algorithm is a neural network, each frame of the second data used to train the neural network and each frame of the third data during the analysis of step (e) have the same (i.e., "fixed") temporal length. In order to achieve greater temporal resolution over a given time period, multiple short frames (on the order of 0.1 seconds) are preferably concatenated in the recurrent neural network. Multiple short frames are used in this way to prevent the subject from actually experiencing multiple changes in motion direction during a frame, which would make it more difficult to provide an accurate estimate of the first metric during that frame.

[0055] In some embodiments, multiple frames (e.g., second data and third data) have a time length based on at least one of the position environment and the determined motion environment during the corresponding first time period and the second time period. For example, based on at least one of the position and the motion environment determined during the second time period, the frame length of the third data can be changed accordingly. For example, when the determined motion environment is "running", since the frequency of heel strikes in running motion is increased compared to walking motion, a more accurate estimate of the first metric (here, it is usually at least one of the motion direction, rate, or speed) can be obtained by using a shorter frame length than when the determined motion environment is "walking".

[0056] The trained neural network can be trained to generate multiple estimates of the first metric over an arbitrary duration, which is typically relevant to the context of the movement. For example, the duration can be a stride period for a peripatetic movement such as walking or running. These multiple estimates can be obtained by overlapping frames of third data from which the metric is estimated. These multiple estimates can then be averaged to give a more reliable estimate of the stride period overall, compared to a single frame having a time length spanning the stride period.

[0057] At least one frame length of the third data in step (e) and at least one frame length of the second data in step (c) may be determined based on at least one parameter that quantitatively describes an aspect of the motion environment determined during the corresponding time period, rather than on an arbitrary or fixed time basis. Preferably, the at least one parameter is a repetition parameter. For example, if the motion environment determined during the corresponding time period is a peripatetic motion (e.g., walking, running, jogging, etc.), data obtained from an accelerometer as an auxiliary sensor may be used to define a data frame by heel-to-heel strikes. In other words, the start of a frame may be defined by the first heel strike determined by the data from the accelerometer, and the end of a frame may be determined by a subsequent heel strike. Preferably, this is a right heel strike on a right heel (or a left heel strike on a left heel) so as to average out the left / right oscillatory motion over the entire stride.

[0058] The use of frame selection based on at least one parameter that quantitatively describes an aspect of the determined motion environment (such as heel strike) advantageously means that any actual changes in the direction of motion of the subject within the frames are minimised, thereby providing a more accurate estimate of the metric of interest during the second time period.

[0059] When the first algorithm comprises a neural network, a particular advantage of the present invention is that there is no need to initially classify the first training data into a specific environment. However, in some embodiments, the method may further include the following steps: (a1) determining a training position environment of one or more second platforms relative to the subject during a first time period, wherein the first training data includes the (one or more) training position environment. Similarly, in some embodiments, the method may further include step (a2): determining a training motion environment of the subject during the first time period, wherein the first training data includes the determined motion environment. In embodiments where the first algorithm is not a neural network, such as an algorithm based on SVM technology, this method of initially classifying the training data into motion and / or position environments is particularly beneficial.

[0060] Preferably, where the method comprises classifying the first training data into a motion and / or position context, the method further comprises analysing the third data to determine the motion and / or position context during the second time period.

[0061] At least one of the training position environment and the training motion environment may be determined by user input. For example, data obtained during the first time period may be manually labeled with the appropriate training position environment and training motion environment. In other embodiments, at least one of the training motion environment and the training position environment may be determined by analyzing the first and second data in the same manner as described above for the second time period (i.e., by using a machine learning algorithm or a lookup table).

[0062] The determination of the position context and the motion context in the first or second time period may be performed in either order and may be performed in an iterative manner without loss of generality.

[0063] Preferably, the first algorithm comprises (eg is) a neural network trained using said first training data.However, other algorithms that can be trained using training data may be used, such as algorithms based on support vector regression (SVR) techniques.

[0064] In an embodiment, the method may further comprise the step of determining, based at least in part on the first data, an evolution of a training metric related to the movement of the subject during the first time period, and wherein the first training data comprises the evolution of said training metric. The training metric is determined at least in part using the first data, i.e. data from the primary positioning unit. In an embodiment, the evolution of the training metric may be determined solely using the first data. Using data obtained from the primary positioning unit to determine the first training metric advantageously provides reliable training data for training the first algorithm. The first training metric is typically at least one of: position, rate, speed, direction of movement. Preferably, the training data comprises a trajectory of the subject during the first time period, in other words, the method comprises determining the evolution of the position of the subject during the first time period, for example as a plurality of positionings from the primary positioning unit.

[0065] Since the primary positioning unit is active during the entire first time period (i.e., at every moment within the first time period), a highly accurate and reliable determination of at least one metric of interest can be obtained during the first time period, thereby generating accurate and reliable training data. For example, if the primary positioning unit includes a GNSS sensor that has good visibility of the satellite constellation at all times during the first time period, accurate velocity and positioning information can be obtained throughout the first time period, and this velocity and positioning information can be used to determine the evolution of the training metric, and in particular, determine the trajectory of the subject during the first time period. Similarly, the primary positioning unit can include a visual odometry unit such as a Google Tango device to provide accurate velocity and positioning information during the first time period to determine the evolution of the training metric. Using such a primary positioning unit during the first time period advantageously provides training data that can be relied upon.

[0066] Assuming that the data obtained from the primary positioning unit is accurate and reliable, the evolution of the training metric during the first time period can be determined. This may be a reasonable assumption. However, this may not always be the case; for example, there may be durations during the first time period when the GNSS sensor provides corrupted data. Alternatively or additionally, the primary positioning unit may have inherent biases, particularly in the case of low-cost devices. If the primary positioning unit is based on visual odometry, then although it will be able to provide an accurate determination of the metric of interest during the first time period, the use of a camera may not be appropriate in many cases. This may be for privacy reasons or availability, for example, the motion and position environment during the first time period may not be suitable for visual odometry.

[0067] Thus, in a preferred embodiment of the invention, a first training metric may be determined based on first data and second data obtained during a first time period. In other words, data from at least one auxiliary sensor may be used in combination with data obtained from the main positioning unit in order to determine the evolution of the training metric during the first time period. In a further embodiment of the invention, the method may include obtaining fourth data from at least one additional auxiliary sensor mounted on a first platform carried on the subject during the first time period, and wherein the evolution of the training metric is determined using at least one of the second data and the fourth data and the first data. For example, the first platform may be a smartphone carried by a user, the smartphone comprising both a main positioning unit and one or more auxiliary sensors. In a preferred embodiment, data obtained from the main positioning unit and at least one additional auxiliary sensor are used to determine the evolution of the training metric, both of which are mounted on the first platform. In other embodiments, the main positioning unit (e.g., a CCTV camera system) is separate from the subject, and at least one additional auxiliary sensor is mounted to the platform carried on the subject.

[0068] The step of determining the evolution of the training metric of interest may include: obtaining first sub-data from the at least one primary positioning unit in a first time sub-period within the first time period; obtaining second sub-data from one or more auxiliary sensors and / or at least one further auxiliary sensor mounted on the first platform in a second time sub-period within the first time period; comparing the first sub-data and the second sub-data with each other and / or with a model of the subject's motion during the first time period to obtain corrected first sub-data and / or corrected second sub-data, and determining the evolution of the training metric associated with the subject's motion during the first time period based on (e.g., constrained by) the corrected first sub-data and / or the corrected second sub-data. The second sub-data is typically obtained from the auxiliary sensor mounted on the first platform. In other words, the evolution of the training metric may be determined based on data obtained only from the primary positioning unit and at least one auxiliary sensor mounted on the first platform. However, alternatively or additionally, the second sub-data may be obtained from at least one auxiliary sensor mounted on the second platform.

[0069] The first time subsegment may be shorter than the first time period, or may be substantially the same as the first time period. Similarly, the second time subsegment may be shorter than the first time period, or may be substantially the same as the first time period.

[0070] Each of the first and second time subsegments may be substantially identical to the first time period, in which case the step of comparing the first sub-data and the second sub-data to each other and / or to a model of the subject's motion during the first time period to obtain corrected first sub-data and / or corrected second sub-data may be considered to obtain corrected first data and / or second data. However, typically, the first and second time subsegments are distinct and may be used to provide a "moving window" of a series of metrics (e.g., user position or velocity) at a sequence of corresponding moments in the first time period based on the perceived reliability of the data in the first and second time subsegments corresponding to a particular moment in time.

[0071] The motion model typically includes at least one of the following: velocity, direction of motion, speed, positional context of one or more second platforms relative to the subject during the first time period, and motion context of the subject during the first time period, preferably wherein the motion model is determined at least in part based on an analysis of the first sub-data and / or the second sub-data. The motion model may be provided using a trained first algorithm.

[0072] The position and motion environment can be determined, for example, by analysis of the sub-data, or by user input in the manner described above. In some embodiments, the motion model includes at least one parameter that quantitatively describes an aspect of the motion, and / or at least one function that can be used to determine a parameter that quantitatively describes an aspect of the motion. For example, the at least one parameter can be one of the following: pedestrian step length, pedestrian speed, pedestrian height, pedestrian leg length, stair step height, stair level distance, or compass heading offset to the direction of motion.

[0073] As will be further explained herein, the use of corrected first sub-data and / or corrected second data advantageously allows for an improved accuracy determination of the evolution of the training metric to be generated (and hence the first training data). By cross-comparing the first sub-data and the second sub-data (particularly if obtained from different sensors and / or time sub-periods) with each other and / or with the motion model, errors or inaccuracies in the originally obtained first sub-data and / or second sub-data may be corrected prior to determining the evolution of the training metric during the first time period. This is of particular benefit when (e.g. as seen in smartphones) one or more auxiliary sensors and / or the primary positioning unit are low-cost sensors, which are typically of low quality and have large inherent biases.

[0074] A particular advantage of this aspect of the invention is that the first training data can be continuously updated during substantially any time period in which the primary positioning unit is operational. For example, the first training data can be obtained during a first time period when a user carrying a first platform (such as a smartphone) is outdoors (where a GNSS sensor used as the primary positioning unit has visibility of a satellite constellation). During a subsequent second time period, when the user moves indoors and GNSS data is not available, data obtained from at least one auxiliary sensor mounted on a second platform carried by the user (such as a fitness tracker) can be used to estimate the desired first metric using a first algorithm trained using data obtained during the first time period. Once the user moves outdoors again and the primary positioning unit becomes operational, appropriate training data can be obtained again.

[0075] Comparing the data obtained during the first and second time sub-periods with each other and / or with a motion model of the subject provides a further advantage over prior art methods that do not perform such analysis before processing the obtained data to provide a tracking solution. Thus, the present invention allows more reliable and / or more accurate data to be used in the first training data used to train the first algorithm.

[0076] Comparing the first sub-data with the second sub-data with each other and / or with the motion model may include performing a self-consistency check. These may, for example, include analyzing data from a given sensor in its corresponding time sub-segment. Typically, this may include analyzing the data to ensure that it does not violate the recognized physical laws of the motion model. For example, the first sub-data may be obtained from a GNSS sensor and show a smooth positioning trajectory with a distance of 1m between positions with an update interval of 1s, which is consistent with the determined training motion environment (e.g., walking), and then suddenly shows a single value of 1km away before returning to the vicinity of the original position sequence. From this, we can infer that the "1km" data point is inconsistent with the rest of the data and can therefore be ignored when determining the first training metric. Other examples of data that are not subjected to this self-consistency analysis include magnetometer values ​​other than the magnetometer values ​​that are effective for the local geomagnetic field, and spikes in smooth or unchanged barometer data that are inconsistent with spikes in acceleration data that suggest actual altitude changes.

[0077] During a first time period, the first sub-data and the second sub-data are compared to each other and / or to a motion model of the subject. For example, the analysis can include determining at least one data point from the data of the first sub-data and / or the second sub-data that does not correspond to the training position and / or motion environment determined during the first time period, and correcting for the at least one data point. Assuming that the training motion environment is a training motion environment for a running user ("running"), and when analyzing the first data, highlighting false data points that do not correspond to such motion - for example, data points that suggest that the platform has moved a distance inconsistent with someone who is running in the time frame - such erroneous data points can be corrected (e.g., removed) to determine the evolution of the training metric with improved accuracy. This analysis helps to avoid false results, which are, for example, a common problem in GNSS systems.

[0078] Comparing the first sub-data with the second sub-data with each other and / or with the motion model may include determining a measurement deviation of at least one auxiliary sensor and / or at least one main positioning unit, and correcting the deviation in order to obtain corrected sub-data. This is particularly advantageous when using low-cost sensors (e.g. MEMS sensors) that are prone to such deviations. Assume that the motion model is determined as "running" during the first time period, where it is known that the speed of each foot of the user is zero at regular intervals when each foot is located on the ground. If the auxiliary sensor includes an inertial sensor mounted on a second platform located on the user's feet, the zero speed update analysis can be used to determine the deviation of the inertial sensor during each stationary time period. This deviation measurement can be used to constrain the evolution of the training metric during the first time period by taking into account the determined sensor deviation, so as to improve the accuracy of the evolution of the training metric and therefore improve the accuracy of the first training data. In addition, when estimating the metric of interest in the second time period, the deviation in at least one auxiliary sensor determined in the analysis during the first time period can be taken into account.

[0079] Advantageously, the duration and / or amount of overlap of the first and second time sub-periods may be selected based on an analysis of the reliability and / or accuracy of data obtained from one or more auxiliary sensors and at least one primary positioning unit during the first time period. Typically, the duration of the first and second time sub-periods is determined based on at least one of the sampling frequency of the respective sensors, the required statistical confidence in the data obtained, and the amount of data required to identify any erroneous results. For example, to determine whether a platform is static, a time period is required to obtain the data necessary to make such a determination. One hundred measurements may be required in order to obtain a clear measure of the statistical standard deviation of the data obtained, thus requiring a duration of 100 / fs, where fs is the sampling frequency of the respective sensors. As a further example, if the data is obtained from a GNSS sensor used as a primary positioning unit, and we wish to identify and remove spurious data points (e.g., GNSS data points that move 1 km in 1 second and then back again), a rolling window of only 5-10 data points may be required. This assumes relatively smooth motion of the platform of interest.

[0080] It is assumed that a GNSS sensor providing a time series of position measurements (serving as a primary positioning unit) experiences a short period of disturbance during a first time period in which the data is identified as corrupted. Typically, the trajectory of the position will be similarly corrupted, since the data is analyzed point by point in time sequence without reference to future data points. Advantageously, in the present invention, the analysis of the data obtained during the first time period allows the corrupted data to be ignored, and only data obtained before and after the corrupted time period is used to interpolate the track during the corrupted time period. Alternatively, after the corrupted data is identified, the time sub-segments can be dynamically adjusted so as to remove the corrupted time period, wherein the evolution of the training metric is interpolated during the corrupted time period.

[0081] Typically, the acquired first and second sub-data are evaluated on a time scale appropriate to the sensor being used or the training metric of interest being determined. In some embodiments, parameters (e.g., length and relative overlap) of the first and second time sub-segments are selected so as to allow identification and correction of at least one error in the first and / or second sub-data.

[0082] In an embodiment, at least one of the first and second sub-data is analyzed backwards in time, preferably iteratively forwards and backwards in time. This exploits the fact that the first training data need not be generated in real time. By analyzing at least one of the first and second sub-data backwards in time, it can be advantageously achieved that the evolution of the training metric is determined with improved accuracy during the first time period. For example, if the same spurious data point is identified when analyzing the data forwards and backwards in time, this data point can be ignored or corrected when transmitting the data set in the future.

[0083] Analyzing the sub-data backwards in time also advantageously allows asymmetries in the obtained sub-data to be used in order to assess and / or correct the reliability and / or accuracy of the sub-data. For example, an erroneous "event" in the data obtained from a sensor (an auxiliary sensor or a primary positioning unit) may appear as a gradual change (and therefore not necessarily erroneous) when analyzed in a forward direction, but may appear as an unreliable discontinuous jump when analyzed in a backward direction. Therefore, by analyzing the data backwards in time, the obtained data can be corrected accordingly (e.g., by removing data points that constitute the event). In a preferred embodiment, analyzing the first and / or second sub-data backwards in time allows erroneous data to be removed from subsequent transmission of the data and determination of the first training metric, thereby providing a significant improvement over simple averaging of the forward and backward data.

[0084] In a particularly advantageous embodiment, the first and / or second sub-data are analyzed iteratively forward and / or backward in time. Preferably, the first analysis of these sub-data is based on at least one confidence threshold, and at least one confidence threshold is modified after the first analysis; wherein the subsequent analysis of the first sub-data and / or the second sub-data is based on the modified confidence threshold. In other words, for this iterative analysis, at least two passes (analyses) are made through these data, and the confidence threshold can be modified after each pass. The modification can be based on the results of the first analysis. For example, in the first pass, the sub-data can be analyzed using a first confidence threshold that may not filter out any data. In subsequent passes of the sub-data, some data can be excluded (or assigned a greater uncertainty) when a stricter confidence threshold is used. The confidence threshold that is changed when the first and / or second sub-data is subsequently transmitted advantageously allows the gradual removal of erroneous measurements that only become apparent when the data is transmitted multiple times.

[0085] The confidence threshold may be initially set ("predetermined") based on known statistical noise characteristics of at least one auxiliary sensor or the primary positioning unit, or from information derived from other sensors, where information from a first sensor may be used to determine a threshold suitable for testing the validity of a measurement from a second sensor. For example, an inertial auxiliary sensor providing acceleration information may be used to set a threshold for expected changes in GNSS frequency measurements obtained from the primary positioning unit during dynamic motion. Here, changes in position may be derived from measurements of acceleration, from which limits on acceptable changes in position derived from the GNSS or other position sensor may be set.

[0086] When the data is subsequently transferred, confidence in the information from the other sensors can be improved; for example, the bias associated with the inertial sensor can be more accurately known, achieving an improved estimate of acceleration and therefore tighter bounds on any GNSS frequency changes obtained from the primary positioning unit. Mistakes or errors in the frequency measurement that were allowed in the first pass can now be removed in the second pass. Iterative analysis of the sub-data also advantageously allows any bias in the at least one auxiliary sensor and / or the primary positioning unit to be determined more reliably.

[0087] In an embodiment, the evolution of the first training metric during the first time period is determined in near real time (e.g. real time within 1 second). However, in a particularly advantageous embodiment, at least one first training metric during the first time period is determined in a batch processing subsequent to the first time period. The batch processing may include a least squares or other estimation function applied to the data obtained during the first time period, and advantageously provides the most accurate and reliable first training data during the first time period.

[0088] The inventors have recognized that in many cases, the data obtained during the first time period can provide redundant information about training the first algorithm, resulting in unnecessary inefficient use of computer resources and processing power during the training phase. For example, if during the first time period the subject is a user having a jogging motion environment, then due to the repetitive nature of the jogging motion, most of the data frames will be similar to each other.

[0089] Therefore, preferably, the method of the present invention may further include the following steps: (f) obtaining fifth data related to the movement of the subject from at least one main positioning unit during the third time period; (g) obtaining sixth data from one or more auxiliary sensors during the third time period; (h) determining the evolution of a training metric related to the movement of the subject during the third time period based at least in part on the fifth data; (i) analyzing the sixth data to estimate the evolution of the training metric during the third time period, wherein the evolution of the estimated training metric is constrained by at least one first metric, the at least one first metric is estimated using a first algorithm, and the first algorithm is trained using first training data; (i) comparing the determined and estimated evolutions of the training metric, and if the difference between the determined evolution and the estimated evolution of the training metric is greater than a predetermined threshold, updating the first training data using second training data, wherein the second training data includes the fifth data and the sixth data. The difference being greater than the predetermined threshold means that the second training data includes information that is not yet present in the first algorithm, and training of the first algorithm should be performed using the second training data.

[0090] On the other hand, if the difference between the determined second training metric and the estimated second training metric is smaller than the predetermined threshold, this means that the second training data will not further improve the first algorithm and thus the second training data may be ignored.

[0091] This comparison with the second training data may be performed at regular intervals.

[0092] As described above, a particular advantage of the present invention is that a first metric obtained using a first algorithm can be used to constrain an estimate of the evolution of a second metric related to the motion of the subject during a second time period. In a preferred embodiment, this is the evolution of the subject's position over time during the second time period (i.e., the subject's trajectory). In other words, the trained first algorithm is able to provide some aspects of the subject's motion during the second time period (i.e., the first metric or multiple first metrics), such as a quantitative estimate of the rate and direction of motion, which can be used to constrain an estimate of the second metric (e.g., trajectory) based on data obtained from one or more auxiliary sensors mounted on one or more second platforms (e.g., an inertial navigation system) during the second time period.

[0093] The evolution of the second metric during the second time period can be considered as being constrained by a motion model of the subject during the second time period, wherein the motion model includes the first metric (or multiple first metrics) obtained by the first algorithm. In other words, at least one first metric estimated using the trained first algorithm can be considered as a motion model of the subject during the second time period.

[0094] Estimation of the evolution of the second metric may include: obtaining third sub-data from a first auxiliary sensor mounted on a second platform in a third time sub-segment within the second time period; obtaining fourth sub-data from the first auxiliary sensor and / or another auxiliary sensor mounted on a second platform carried on the body or another second platform in a fourth time sub-segment within the second time period; comparing the third sub-data and the fourth sub-data with each other and / or with the estimated first metric (e.g., a motion model) to obtain corrected third sub-data and / or corrected fourth sub-data, and estimating the evolution of the second metric during the second time period based on (e.g., constrained by) the corrected third sub-data and / or the corrected fourth sub-data.

[0095] The third time subsegment may be shorter than the second time period, or may be substantially the same length as the second time period. Similarly, the fourth time subsegment may be shorter than the second time period, or may be substantially the same length as the second time period. Typically, the third and fourth time subsegments are different. The duration and / or amount of overlap of the third and fourth time subsegments may be selected based on an analysis of the reliability and / or accuracy of data obtained from one or more auxiliary sensors during the first time period.

[0096] When estimating the evolution of the second metric, the same techniques as outlined above with respect to estimating the evolution of the first training metric may be used. Thus, using such time sub-segments and comparison of the third and fourth sub-data with each other and / or the estimated first metric (e.g., a "motion model") has the same advantages as outlined above with respect to determining the evolution of the first training metric.

[0097] Preferably, comparing the first sub-data and / or the second sub-data with the first metric comprises: performing a self-consistency check.

[0098] Preferably, at least one of the third sub-data and the fourth sub-data is analyzed backward in time, preferably iteratively forward and backward in time, in order to obtain corrected third sub-data and / or corrected fourth sub-data.

[0099] Preferably, comparing the third sub-data and the fourth sub-data with each other and / or with the estimated first metric comprises determining a measurement deviation of at least one auxiliary sensor and correcting the deviation in order to obtain corrected sub-data.

[0100] The evolution of the second metric may be estimated in near real time (e.g., real time within 1 second). However, in particularly advantageous embodiments, the evolution of the second metric may be estimated in a batch process subsequent to the second time period. The batch process may include a least squares or other estimation function applied to the data obtained during the first time period, and advantageously provides the most accurate and reliable estimate of the second metric during the second time period.

[0101] According to a second aspect of the present invention, there is provided a computer readable medium comprising executable instructions which, when executed by a computer, cause the computer to perform the method of the first aspect of the present invention.

[0102] The computer readable medium may be provided at the download server. Therefore, the computer may obtain the executable instructions through the software upgrade.

[0103] According to a third aspect of the present invention, there is provided a tracking system for tracking the movement of a subject over time, the system comprising: at least one main positioning unit; one or more auxiliary sensors configured to perform measurements based on which a position or movement can be determined, the one or more auxiliary sensors being mounted on one or more second platforms that can be carried on the subject, and a processor adapted to perform the following steps: (a) obtaining first data from the at least one main positioning unit during a first time period in which the at least one main positioning unit is operating; (b) obtaining second data from one or more auxiliary sensors during the first time period, the one or more second platforms being carried on the subject during the first time period; (c) generating first training data comprising the first data and the second data; (d) obtaining third data from the one or more auxiliary sensors during a second time period in which the one or more second platforms are carried on the subject, and (e) analyzing the third data using a first algorithm trained using the first training data to estimate at least one first metric associated with the movement of the subject during the second time period.

[0104] The at least one main locating unit may be mounted on a first platform capable of being carried on the subject, the first platform being carried on the subject during the first time period. Alternatively or additionally, the at least one main locating unit may be detachable from the subject. Typically, the tracking system further comprises at least one further auxiliary sensor mounted on the first platform.

[0105] Preferably, the processor is further adapted to perform the following steps: analyzing the third data to estimate the evolution of at least one second metric associated with the motion of the subject during the second time period, wherein the evolution of the at least one second metric is constrained by at least one first metric estimated using the first algorithm. Typically, the processor of the third aspect of the invention may be adapted to perform the method as set out in the first aspect of the invention. Thus, the third aspect of the invention provides the same advantages as outlined above.

[0106] In the present invention, the one or more auxiliary sensors may include at least one of the following: an accelerometer, a gyroscope, a magnetometer, a barometer, a pedometer, a light sensor, a pressure sensor, a strain sensor, a proximity sensor, a camera, and any other sensor that can provide position-related measurements that can be thought of by a person skilled in the art. Typically, the one or more auxiliary sensors are part of an inertial navigation system.

[0107] In any of the first, second or third aspects of the invention, at least one primary positioning unit may include at least one of the following: a GNSS unit, a camera, a radar, a lidar and any other positioning system that would be conceivable to a person skilled in the art. Where the primary positioning unit includes a camera carried by the subject being tracked, this is typically part of a visual odometry unit.

[0108] The present invention can be implemented on a range of devices. The first platform can be any device on which the main positioning unit is installed. The first platform is typically a smartphone carried by a user, or can be a tablet computer, laptop computer, etc. The second platform is typically a wearable device such as a smart watch, fitness tracker or headphones worn by the user. One or more first platforms can be advantageously used to provide training data for one or more second platforms. For example, a single first platform can be used to provide training data for multiple second platforms carried by a user, and multiple first platforms can be used to provide training data for a single second platform.

[0109] It is particularly advantageous to use data from sensors mounted on two or more independent platforms during the first or second time period so that the data from the two or more platforms can be cross-referenced. For example, during the second time period, it is desirable to estimate the trajectory of the subject using auxiliary sensors mounted on (for example) headphones and fitness trackers in order to avoid reusing information in the trajectory solution (referred to as "next-generation data").

[0110] As already outlined above, the first metric obtained using the first algorithm is typically at least one of: direction of motion, rate, speed, location context, and motion context; while the evolution of the second metric of interest is typically a tracking solution such as a trajectory. In general, examples of metrics that can be estimated using the present invention include: position, range, rate, speed, trajectory, altitude, compass heading, pace, step length, distance traveled, motion context, location context, output power, calorie count, sensor bias, sensor scale factor, and sensor alignment error.

[0111] The present invention is particularly suitable for the case where the GNSS sensor is not operational during the second time period, and only the data from the auxiliary sensor will be used to track the evolution of the metric of interest. However, it will be appreciated that the present invention may also be used in the case where the GNSS sensor is operational, where the first metric estimated by the trained machine learning algorithm is used to further constrain and improve the accuracy of the tracking solution.

[0112] Also disclosed herein is a computer-implemented method performed in a tracking system for tracking motion of a subject over time, the method comprising:

[0113] (a) obtaining first data from at least one auxiliary sensor and at least one primary positioning unit during a first time period, the at least one auxiliary sensor being configured to make measurements from which position or movement can be determined, the at least one primary positioning unit being active during the first time period, wherein the at least one auxiliary sensor and the at least one primary positioning unit are carried on a subject; (b) generating first training data including the first data; (c) obtaining second data from the at least one auxiliary sensor carried by the subject during a second time period, and (d) analyzing the second data using a first algorithm trained using the first training data to estimate a metric associated with the movement of the subject during the second time period. The at least one auxiliary sensor and the at least one primary positioning unit may be mounted on a common platform carried by the subject.

[0114] The present invention also discloses a computer-implemented method performed in a tracking system for tracking the movement of a subject over time, the method comprising: (a) obtaining first data from at least one main positioning unit mounted on a first platform carried by the subject during a first time period, the main positioning unit being operational during the first time period; (b) obtaining second data from at least one auxiliary sensor during the first time period, the at least one auxiliary sensor being configured to perform measurements based on which a position or movement can be determined, the at least one auxiliary sensor being mounted on a second platform carried by the subject; (c) generating first training data comprising the first data and the second data; (d) obtaining third data from at least one auxiliary sensor carried by the subject during a second time period, and (e) analyzing the third data using a first algorithm trained using the first training data to estimate a metric associated with the movement of the subject during the second time period.

[0115] The present invention also discloses a tracking system for tracking the movement of a subject over time, the system comprising: at least one auxiliary sensor, at least one main positioning unit and a processor, wherein the at least one auxiliary sensor is configured to perform measurements based on which a position or movement can be determined, and the at least one auxiliary sensor and the main positioning unit are capable of being carried by the subject, and the processor is suitable for performing the following steps: (a) obtaining first data from the at least one auxiliary sensor and the at least one main positioning unit during a first time period in which the at least one auxiliary sensor and the at least one main positioning unit are carried by the subject, and the main positioning unit is active during the first time period; (b) generating first training data including the first data; (c) obtaining second data from the at least one auxiliary sensor carried by the subject during a second time period; and (d) analyzing the second data using a first algorithm trained using the first training data to estimate a metric related to the movement of the subject during the second time period. BRIEF DESCRIPTION OF THE DRAWINGS

[0116] Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which:

[0117] Figure 1 is a schematic diagram illustrating an example embodiment of the present invention;

[0118] Figure 2a is a schematic logic diagram of a first portable device that may implement the present invention;

[0119] Figure 2b is a schematic logic diagram of a second portable device that may implement the present invention;

[0120] Figure 3 is a flow chart outlining a preferred series of method steps that may be performed during a first time period;

[0121] Figure 4 is a flow chart outlining a preferred series of method steps that may be performed during a second time period;

[0122] Figure 5 is a diagram showing an estimated trajectory of a pedestrian obtained using the present invention;

[0123] Figure 6 is a schematic diagram showing how to use the present invention to estimate the trajectory of a pedestrian;

[0124] Figure 7 The technique used to obtain the position solution at one instant in time is schematically illustrated;

[0125] Figure 8 The technique of obtaining trajectory solutions using batch processing is schematically illustrated;

[0126] Fig. 9 schematically illustrates example position data obtained by a GNSS sensor between two moments in time;

[0127] Fig.10 Schematically illustrating obtaining trajectory solutions in near real time;

[0128] Fig.11 is a flow chart outlining the main steps of one embodiment of the present invention for determining the evolution of a metric of interest;

[0129] Fig.12 is a flow chart outlining the main steps of a further embodiment of the invention for determining the evolution of a metric of interest;

[0130] Fig.13 is a schematic overview of how data from at least one sensor can be used to determine the evolution of a metric of interest. DETAILED DESCRIPTION

[0131] Figure 1It is schematically illustrated how the invention may be implemented to provide an estimate of a metric of interest related to subject motion.

[0132] like Figure 1 , in this embodiment, a pedestrian P wears a headset 110 mounted on the head and carries a smartphone 100. In this embodiment, the smartphone 100 is a first platform and the headset 110 is a second platform. The smartphone 100 and the headset 110 are separate. It should also be noted that in this example, the pedestrian P, the smartphone 100, and the headset 110 are not in a particular fixed relationship, such as in a strapdown or gimbal arrangement. In other embodiments, the pedestrian P may carry an additional second platform and / or the first platform.

[0133] (will be referenced below Figure 2a The smartphone 100 described in more detail comprises a primary positioning unit in the form of a GNSS sensor 20 and an auxiliary sensor suite 10. Figure 1 Schematically illustrated, during a first time period, the GNSS sensor 20 has a clear line of sight to the satellite constellation 90, and is therefore operational to provide positioning and timing data. However, the pedestrian P subsequently moves indoors, where the GNSS sensor 20 no longer has a clear line of sight to the satellite constellation 90, and is therefore no longer able to provide positioning and timing data. This time period during which the GNSS sensor is not operational is a second time period. The present invention allows accurate estimation of the trajectory of the pedestrian during the second time period during which the GNSS sensor is not operational.

[0134] During the second time period, the GNSS sensor 20 is not operational (meaning it cannot receive signals from the satellite constellation 90), so it is necessary to estimate the trajectory of the pedestrian P using only the data from the auxiliary sensor suite 10' on the headset 110. Typically, a trajectory solution can be provided by performing a single integration of the data from the rate gyro 14' at each moment in the second time period to determine the device attitude, and performing a double integration of the data obtained from the accelerometer sensor 12' (while correcting for gravity) to obtain position data. However, as described above and as is well known in the art, such integration-based solutions based on consumer-grade inertial sensors are typically subject to large accumulated attitude and position errors after only a few seconds due to the numerical integration of measurements from noisy and generally poorly calibrated sensors. This is exacerbated when the device is not in a strapdown or gimbaled relationship with the body. As will be explained below, the present invention overcomes these problems.

[0135] Figure 1A navigation reference system N and sensor reference systems S and S' are also schematically illustrated. The navigation reference system is defined by an orthogonal coordinate system Nx, Ny, Nz and is typically an earth reference system, i.e. east-west, north-south, up-down. The sensor reference system S of the first platform is an orthogonal coordinate system Sx, Sy, Sz centered on the center of mass of the smartphone 100 (i.e., the platform on which the sensors are mounted). The sensor reference system S' of the second platform is an orthogonal coordinate system S'x, S'y, S'z centered on the center of mass of the headset 110. The trajectory of the pedestrian P during the second time period is determined in the navigation reference system N.

[0136] 2A shows relevant components of a smartphone 100, which includes a processor 1, a communication module 2, a memory 3, a screen 4, a local storage device 5 (non-volatile memory), and a battery 7. The communication module 2 includes components required for wireless communication, such as a receiver, a transmitter, an antenna, a local oscillator, and a signal processor.

[0137] The smartphone 100 also includes an auxiliary sensor suite 10, which herein includes an accelerometer 12, a gyroscope 14, a magnetometer 16, and a barometer 18. The accelerometer is configured to measure the acceleration of the smartphone; the gyroscope is configured to measure the rotation rate of the smartphone, the magnetometer is configured to measure the strength and direction of the local magnetic field, and thus measure the compass heading of the smartphone 100, and the barometer is configured to measure the atmospheric pressure and infer the altitude of the smartphone. The accelerometer, gyroscope, magnetometer, and barometer can be MEM devices, which are typically three-axis devices, wherein each orthogonal axis includes a separate sensor. Other auxiliary sensors may be included in such a sensor suite 10. The auxiliary sensor suite will be referred to as an inertial measurement unit (IMU) 10. The smartphone 100 also includes a light sensor 30.

[0138] The smartphone 100 also includes a GNSS sensor 20, such as (in any combination of) GPS, Galileo or GLONASS sensors, which serves as a primary positioning unit. In other embodiments, other primary positioning units may be used in place of or in combination with the GNSS sensor, examples of which have been discussed in the Summary of the Invention section.

[0139] Each of the communication module 2, memory 3, screen 4, local storage 5, battery 7, sensor suite 10, light sensor 30 and GNSS sensor 20 is in logical communication with a processor 1, which is also in logical communication with a tracking solution unit (TSU) 40, which is operable to obtain data from the GNSS unit 20 and auxiliary sensors and determine a tracking solution for the movement of the pedestrian. The TSU 40 includes a tracking and navigation module 44 and a machine learning algorithm (in this case a neural network) schematically shown at 46. The TSU 40 can be implemented in hardware, firmware, software, or a combination thereof.

[0140] like Figure 2b As shown, the headset 110 includes an auxiliary sensor suite (IMU) 10', in this case an accelerometer 12', a gyroscope 14', a magnetometer 16' and a barometer 18'. In the same manner as described with respect to the smartphone 100, the headset also includes a local storage device 5', a communication module 2' and a TSU 40. The headset 110 does not include a GNSS sensor or other main positioning unit.

[0141] exist Figure 2b , the headset 110 is shown as having its own TSU 40 thereon, wherein the tracking solution during the second time period is provided by its TSU. The headset 110 and the smartphone 100 communicate wirelessly, for example via the Internet or a local network or by other wireless protocols such as Bluetooth, so that data can be transferred between the smartphone and the headset. In other embodiments, the headset may not include a TSU, wherein the data obtained by the sensors on the headset 110 during the second time period is transmitted to the smartphone 100 or other device for remote analysis and providing a tracking solution.

[0142] Data obtained by the smartphone 100 when the GNSS sensor 20 is operating may be used to train the machine learning algorithm 46 so that during a second period of time when the GNSS sensor is unavailable, a tracking solution for the pedestrian P may be estimated using data obtained only from the auxiliary sensor mounted on the headset.

[0143] As will be described below, the training of the neural network is typically performed externally to the local device 100, 110. Figure 2a and 2b , the neural network schematically represented at 46 is the latest version of the neural network that has been downloaded to the smartphone 100 and the headset 110, and can therefore be used during the second time period when the GNSS sensor is unavailable.

[0144] Figure 32 is a flow chart outlining the steps of a preferred embodiment of a method 200 performed during a first time period during which the GNSS sensor 20 on the smartphone 100 is operational. This may be referred to as a training phase. In step 201, first data is obtained from the GNSS sensor 20 mounted on the smartphone, and second data is obtained from the auxiliary sensors 12', 14', 16', 18' mounted on the headset. Data is also obtained from the auxiliary sensors 12, 14, 16 and 18 of the IMU 10 of the smartphone. These first data and second data are obtained for multiple motion and positional contexts. For example, during the first time period during which the GNSS sensor 20 is operational, the pedestrian P may change the motion context from walking to running and back to walking again, or may intermittently turn their head to one side during motion, thereby changing the orientation of the headset relative to the direction of motion. Similarly, the positional context of the smartphone may change multiple times, perhaps from being held in the pedestrian's hand, to being placed in a side pocket of the pants and then being held to the ear during a mobile phone call.

[0145] During the training phase in the first time period, these data obtained by the sensors on the smartphone 100 and the headset 110 are typically sent via the respective communication modules 2, 2' to an external database (e.g., hosted by the cloud) for further processing by an external processor as described below. This advantageously reduces the storage space and processing power required by the local device 100, 110. However, it is envisaged that in alternative embodiments (described below) Figure 3 The steps can be performed locally on the smartphone and headset.

[0146] In step 202, an evolution of at least one training metric is determined based on the first data. The evolution of the training metric is at least partially (typically completely) determined by data obtained from the GNSS sensor. The training metric may be, for example, a series of position fixes of the smartphone 100 obtained by the GNSS sensor 20 during a first time period, wherein the evolution of these position fixes over time gives a trajectory of the smartphone (which gives a good approximation of the trajectory of the pedestrian). Here, it is assumed that the data obtained directly from the GNSS sensor is accurate and reliable during the first time period, and is therefore used to determine the trajectory of the pedestrian during the first time period.

[0147] However, in other embodiments, further analysis of the first data obtained during the first time period may be performed to obtain a more accurate determination of the trajectory of the pedestrian during the first time period. For example, the trajectory of the pedestrian during the first time period may be determined based on analysis of data obtained from the IMU 10 and the GNSS sensor of the smartphone. Such techniques will be described in further detail herein.

[0148] In step 203, training data is generated using the first data, the second data and the determined trajectory. Thus, the training data includes data associated with the trajectory of the pedestrian determined during the first time period obtained from the auxiliary sensors 12', 14', 16', 18' mounted on the headset, the trajectory of the pedestrian determined during the first time period being obtained across various motion environments and position environments.

[0149] The position fix obtained by the GNSS sensor during the first time period is obtained in the navigation reference frame N, and therefore the determined trajectory is also in the navigation reference frame N. However, these data need to be transformed into the headset sensor reference frame S' to be used as training data. Therefore, during the first time period, the yaw offset between the sensor reference frame S' and the navigation reference frame N must be accurately known or determined. Data from the accelerometer 12', gyroscope 14' and magnetometer 16' on the headset can be used to track the yaw offset between the headset sensor reference frame S' and the navigation reference frame N, and therefore accurately transform the direction of motion obtained from the GNSS sensor 20 from the navigation reference frame N to the sensor reference frame S' for training data.

[0150] In step 204, these training data are then used to train the neural network. During the first time period, the pedestrian P may walk in a direction aligned with the Nx axis (see Figure 1 ). During walking motion, there will typically be motion along the Ny axis (e.g., a pedestrian's vertical "bounce"), motion along the Nz axis (e.g., a pedestrian's movement from side to side), and movement along Nx. These motions will be represented by data obtained by auxiliary sensors on the headset 110 (e.g., accelerometer 12' and gyroscope 14') and are present in the training data. By training using the training data, the NN learns these three orthogonal patterns that exist for the walking motion environment, regardless of the sensor orientation. Similarly, if the pedestrian P suddenly runs, at least the vertical "bounce" along the Ny axis is typically more obvious than walking, so the NN can again learn the characteristic orthogonal patterns in the accelerometer data that exist for the running environment, independent of the sensor orientation. In addition, if sufficient training data spanning a range of walking and running rates is provided, the NN can learn the relationship between the characteristic orthogonal patterns and different pedestrian rates. As described above, such training of the NN is typically performed external to the local smartphone and headset device.

[0151] These characteristic patterns in the data obtained from the auxiliary sensor on the headset during the first time period are associated with the reliable trajectory obtained by the GNSS sensor 20 on the smartphone. Thus, the NN is able to learn characteristic patterns associated with the pedestrian's velocity in the sensor reference frame S' and his or her direction of motion.

[0152] In step 205, the latest version of the trained machine learning algorithm is periodically downloaded to the smartphone 100 and the headset 110, where the algorithm is downloaded in the Figure 2a and 2b 46 are schematically shown in FIG.

[0153] In an embodiment, the present invention may utilize a "generic" machine learning algorithm trained using data from multiple different users, with the advantage that the training is based on a large amount of training data. However, preferably, such a generic algorithm may be customized for each individual user (e.g., by using weights based on the source of the training data) so that each user's individual motion characteristics may be represented in the customized algorithm. In other embodiments, the present invention may provide a separate "customized" machine learning algorithm for each user based solely on that user's data.

[0154] In this embodiment, the first machine learning algorithm is a neural network (NN). The NN includes convolutional and recursive stages that are trained using the training data generated in step 203. During a second time period (described in more detail below), data obtained by the auxiliary sensors 12', 14', 16', 18' of the headset 110 is input to the trained NN, which then makes accurate predictions of at least one of the direction of motion, rate, speed, motion environment, and position environment based on the temporal patterns observed in the data. The data frames obtained from the auxiliary sensors during the second time period are passed to the trained NN, which is represented by a system of weighted coefficients associated with each auxiliary sensor and connected by a network of multiplication and addition operations. By applying these weighted sums to the input data frames obtained during the second time period, output predictions from the NN are generated.

[0155] Figure 4 is a flow chart outlining the steps of a preferred embodiment of the method 300 performed during a second time period during which the GNSS sensor 20 is not operational. However, it should be understood that the method 300 may be used during a time period during which the GNSS sensor is operational. In step 301, third data is obtained from auxiliary sensors mounted on the headset, namely from the accelerometer 12', the gyroscope 14', the magnetometer 16' and the barometer 18'. As already explained above, conventional attempts to determine the trajectory based solely on data from such sensors are unsuccessful, particularly in scenarios where the platform and the body are not in a strapdown or gimbaled relationship.

[0156] Optionally, at step 302, the motion context and position context during the second time period are determined. This is typically performed by the neural network 46 analyzing third data (typically data obtained from the accelerometer 12' and gyroscope 14') based on learned patterns in the third data.

[0157] Alternatively, these motion and position environments may be determined by using an appropriate lookup table. Such a lookup table includes a plurality of predetermined motion and position environments and associated auxiliary sensor data patterns (typically accelerometer and gyroscope data patterns) and may be stored in a local storage device. The third data obtained from the headset (typically from the accelerometer 12' and gyroscope 14') during the second time period may then be matched to the most likely motion and position environment based on the pattern in the obtained third data.

[0158] As another alternative, a separate machine learning algorithm may be used that has been trained on previously acquired training data. As yet another option, the positional environment and the motion environment may be determined by user input. For example, the pedestrian P may input the desired environment via a graphical user interface (GUI) presented to the pedestrian during the second time period. Typically, this will be a selection from a plurality of predetermined options.

[0159] Importantly, the motion context and position context during the second time period are determined solely based on data obtained from sensors mounted on the headset 110, without the need for external or auxiliary information (eg, CCTV cameras).

[0160] The third data is analyzed by the NN on a frame-by-frame basis. A frame (or "window") of the third data includes a plurality of time-ordered measurements obtained by at least one auxiliary sensor mounted on the headset. In the present embodiment, where the machine learning algorithm is in the form of a neural network, each of said frames of the third data has the same fixed time length. The length of each frame of the third data may be selected based on an optimal frame length for the neural network.

[0161] Optionally, the method may include a step 303 of determining a frame length based on an (optionally) determined motion environment. Typically, it is desirable that the direction of motion of the pedestrian does not change during the frame of the obtained third data, as this may result in difficulty in determining the direction of motion during that time. Therefore, preferably, each frame of the third data is selected so that it does not contain environmental changes. For example, in the case where the pedestrian is determined to have a walking or running motion environment, the time length of the frame of the third data may be selected to be shorter than the typical stride duration of the pedestrian, for example, the frame length may be of the order of 0.1 seconds. In contrast, in other embodiments, if the motion environment is determined to be the motion environment of a car driving along a straight road, the data frame may be selected to have a longer time length in order to reduce the required processing power.

[0162] Frames of the third data may be selected to overlap one another, wherein multiple metric estimates obtained for the overlapping frames are averaged to give a more reliable estimate overall (e.g.) of the pedestrian's stride cycle, compared to a single frame having a time length spanning the stride cycle.

[0163] It should be noted that when training a machine learning algorithm using training data ( Figure 3 In step 204), the same frame length is used to determine. In other words, for analyzing the third data during the second time period, when training the NN, the same frame length is used for the generated training data.

[0164] At step 304, the third data is analyzed using a neural network to estimate the first metric of interest. In this embodiment, based on the third data obtained from the auxiliary sensor on the headset, the trained NN outputs the velocity of the pedestrian P in the sensor reference frame S' for each frame of the third data input to the NN. This velocity of the pedestrian for each frame of the third data can be referred to as a motion model of the pedestrian during the second time period, which is then used to constrain the tracking solution, as described below.

[0165] At step 305, the estimated velocity solution from the NN is then passed to the tracking and navigation module 44. The tracking and navigation module 44 performs further analysis of the third data in order to estimate the trajectory of the pedestrian P during the second time period, which trajectory is constrained by the velocity prediction ("motion model") obtained from the NN. This analysis of the third data is typically performed using a Kalman filter, and the states tracked by the Kalman filter are typically orientation, velocity, and position, as described below.

[0166] The TSU obtains data from the auxiliary sensors 12', 14', 16', 18' on the headset 110 during the second time period. The data from these sensors is then used to estimate the orientation, position and velocity of the pedestrian P during the second time period using conventional inertial navigation techniques (e.g., integrating the accelerometer data while correcting for gravity to obtain velocity and position). These orientation, position and velocity estimates obtained using the auxiliary sensor measurements are referred to as IMU solutions. The third data obtained from the sensors of the IMU 10' is used to update the orientation, velocity and position states with each new accelerometer and gyroscope measurement.

[0167] As described above, the NN outputs a predicted velocity for each frame of the third data in the headset sensor reference frame S', and therefore these velocity predictions need to be transformed into the navigation reference frame N so that the trajectory of the pedestrian during the second time period can be determined in the navigation reference frame N. The tracking orientation of the IMU solution is used to determine the compass heading of the headset 110, and therefore the yaw offset between the sensor reference frame S' and the navigation reference frame N. The velocity predictions obtained from the NN are then transformed to the navigation reference frame N by the tracking and navigation module 44 using the determined yaw offset.

[0168] Once the velocity predictions from the NN have been transformed into the navigation reference frame N, these transformed velocity predictions are fused with the current IMU solution using a Kalman filter so that corrections are applied to all tracked states (i.e., orientation, velocity, and position). In this way, the IMU solution is constrained by the velocity output from the NN, resulting in an accurate tracking solution for the pedestrian during the second time period.

[0169] Thus, the output from the tracking and navigation module 44 is a series of 2D velocity, orientation and position estimates for each frame of the third data, from which the 2D trajectory of the pedestrian in the navigation reference frame during the second time period can be estimated.

[0170] Other states (e.g., sensor biases that may be estimated using techniques such as zero-speed correction (ZUPT)) may be tracked within the Kalman filter framework to further increase the accuracy and reliability of the pedestrian trajectory during the second time period. In some embodiments, if GNSS measurements become available during the second time period, these GNSS measurements may also be fused with the IMU solution within the Kalman filter. However, the present invention is designed to provide a reliable tracking solution without the need for such GNSS data.

[0171] In the exemplary embodiment described above, the output from the NN is a series of velocity estimates for each frame of the third data obtained. In an alternative embodiment, the NN can be used to estimate the direction of motion of each frame of the third data (in the headset reference frame S'). The data obtained from the IMU 10' on the headset can be used to estimate the rate of each frame of the third data using conventional techniques such as integration (i.e., without using the NN). Therefore, the combination of the rate estimate with the direction of motion estimate from the NN and the output from the tracking and navigation module 44 is a series of 2D velocity estimates for each frame of the third data, which can be transformed to the navigation reference frame N so that the trajectory of the pedestrian can be estimated. This alternative embodiment (in which the NN is used only to estimate the direction of motion) can advantageously use less processing power than when the NN is used to generate velocity estimates.

[0172] Further techniques for analyzing the third data in step 305 are outlined below.

[0173] Figure 5 is a diagram showing a pedestrian trajectory (i.e., a series of position estimates) obtained using the present invention (400) using the above configuration (i.e., when the pedestrian wears the headset 110 during a period when no primary positioning unit is operating). For comparison, a trajectory (410) using conventional "dead reckoning" (where the direction of motion estimate is obtained using compass heading and step detection) is also shown, as well as ground truth obtained using a Google Tango device (420). The x and y axes represent distance in the navigation reference frame.

[0174] Here, the pedestrian is walking in a room inside a building where GNSS data is not available, so that only data from the auxiliary sensor on the headset is used to obtain a trajectory solution. The training data is pre-acquired during a first period of time when the pedestrian wears the headset and also has a working GNSS sensor located in his pocket to provide movement direction and position measurements. The training data is obtained when the pedestrian is outdoors (such that the GNSS sensor has sufficient satellite visibility and is therefore working).

[0175] During the second time period in which the trajectory shown is obtained, a trajectory 400 is obtained using a NN trained using the training data obtained during the first time period. As can be seen, the estimated trajectory 400 is advantageously aligned with the ground truth and therefore proves to be extremely robust to changes in head orientation. In contrast, a trajectory 410 obtained using conventional techniques (in this case compass heading and step detection) is highly unreliable compared to the ground truth due to the naive inference that changes in head orientation are related to changes in direction of motion.

[0176] In this example, in addition to being used to determine the trajectory of the pedestrian during the second time period, the inertial sensors mounted on the headset can also be used to determine the orientation of the pedestrian's head. This can be done by analyzing data obtained from, for example, an accelerometer 12', a gyroscope 14', and a magnetometer 16' mounted on the headset. For example, in the case of location-based services or advertising, the orientation of the user's head relative to its direction of motion can be a useful metric.

[0177] In the above-described embodiments, the primary positioning unit is a GNSS unit. However, in other embodiments, the primary positioning unit used to generate training data during the first time period may be a visual odometry system configured to determine the position, orientation and / or velocity of the subject based on associated images obtained during the first time period (e.g., from CCTV images or using the Google Tango platform).

[0178] Figure 6 The evolution of the position of a pedestrian P determined between two instants T1 and T2 is schematically shown. Shaded boxes 101a, 101b, 101c, 101d, 101e, 101f represent time periods (“confidence segments”) in which the absolute position of the pedestrian has been determined with high confidence, for example due to the availability of high-quality GNSS data from a smartphone 100 carried by the pedestrian. These confidence segments are called “first time segments”, and the data obtained during these time segments can be used to train the machine learning algorithm in the manner described above. This can include determining the biases in the accelerometer 12, gyroscope 14 and magnetometer 16 sensors.

[0179] The line 450 between the confidence segments represents the estimated trajectory of the pedestrian (i.e., the evolution of his / her position) determined by the TSU 40 during the period of time when GNSS data is not available or is considered unreliable (the "second period of time"). A and T B The GNSS data is shown illustratively as defining a time period during which the GNSS data is deemed too unreliable for determining the absolute position of the device. Metrics that may be used to cause a rejection (or reduction in confidence) of the GNSS data and thereby cause a second time period may include, for example, fewer than a predetermined number of satellites being used, a signal strength of the GNSS data being below a predetermined threshold, etc.

[0180] During a second time period between confidence segments represented by shaded boxes, data from auxiliary sensors of the IMU 10' on the headset 110 is analyzed to estimate the trajectory of the pedestrian, where the trajectory solution is constrained by the output from the trained neural network in the manner described above.

[0181] Techniques for estimating the evolution of a metric of interest using a motion model

[0182] The present invention allows the accurate evolution of a metric of interest (typically a trajectory) to be determined. This may be during a first time period (e.g., used as training data) and / or during a second time period as a tracking solution. The present invention is particularly advantageous because a trained machine learning algorithm provides a motion model (or its parameters) by which a tracking solution can be constrained. This is in contrast to conventional techniques where the motion model may need to be user input or inferred from acquired data, without trained knowledge of prior "trips" represented in the trained machine learning algorithm.

[0183] The following description relates to techniques for estimating the evolution of a metric of interest using such a motion model and to analyzing data in the first or second time period. However, for clarity of description, for example to obtain trajectories for use as training data, we refer to the first time period.

[0184] Figure 7 Schematically illustrates a technique for obtaining a position solution for the smartphone 100 at time T1 during a first time period using both data obtained from the GNSS sensor 20 and data obtained from the IMU 10 mounted on the smartphone. Position solutions for multiple time periods T1, T2, ... Tn may be obtained in the same manner to determine a trajectory during the first time period.

[0185] The data obtained by the GNSS sensor 20 during the first time sub-period 2100 (referred to herein as "first sub-data") and the data obtained by the auxiliary sensor of the IMU 10 during the second time sub-period 2200 (referred to herein as "second sub-data") constrain the position solution at time T1 (represented at 3000). Here, the time sub-period 2100 is longer than the time sub-period 2200, and the time sub-period 2100 is typically on the order of 1-10 seconds. The time sub-period 2200 is typically on the order of ~1 second. In this example, both time sub-periods are shorter than the first time period.

[0186] The first sub-data and the second sub-data are provided to the TSU 40, which calculates the expected solution at the time T1. This is done by comparing the first sub-data and the second sub-data to each other and / or to a motion model of the subject during the first time period in order to obtain corrected first sub-data and / or second sub-data.

[0187] The motion model typically includes the context of the determined position of the smartphone relative to the subject and the context of the subject's motion. The motion model may also include at least one parameter that quantitatively represents the motion of the device. The at least one parameter quantitatively describes an aspect of the motion, such as the user's step length or velocity.

[0188] The motion model may be pre-selected by the user of the smartphone, for example by interacting with a suitable graphical user interface (GUI) presented to the user via the screen 4. For example, at the beginning of the first time period, the user may decide to go for a run, and select a motion model comprising a "running" motion environment.

[0189] Alternatively, the first and / or second sub-data may be analyzed by the TSU 40 to determine at least one of a position context and a motion context for use in generating a position solution at a particular moment in time. In particular, the second sub-data and data from the accelerometer and gyroscope sensors for determining a current motion and / or position context may be analyzed by the TSU 40. This is typically performed by matching patterns in the sensor data stream to specific motion and position contexts stored in a lookup table.

[0190] The data obtained from the accelerometer 12 can be used to determine the user's pace, and determine that the motion environment is "walking", and furthermore the user's step length is 0.8m. In addition to the data obtained from the gyroscope sensor 14 that can be used to determine the location environment, the light sensor 30 can detect minimal light or no light during the time period 2200, and therefore can infer that if the time is during daytime hours, the smartphone 100 is in an enclosed space such as a pocket or bag. Therefore, the second time sub-segment 2200 can be considered to provide an environment or "motion model" under which the data from the first time sub-segment 2100 is processed to provide a location solution.

[0191] The first and second sub-data may be compared to each other and / or to a motion model. Such analysis is performed by the tracking and navigation module 44 of the TSU 40 and may include self-consistency analysis (analyzing data acquired by the sensor over a corresponding time period) and cross-reference analysis, where data from one sensor is compared to data from another sensor and / or a motion model. The analysis performed by the tracking and navigation module may include analyzing the acquired data forward and / or backward in time or as a batch process.

[0192] For example, in the case where the motion model has a motion environment determined to be "walking", but when analyzing the first sub-data obtained from the GNSS sensor, a data event that does not fit this walking motion model (e.g., a sudden position shift of 50 m) is observed. This data event can be marked as unreliable, so the data event is used to provide corrected first sub-data, and the TSU 40 can provide a final position solution based on the first sub-data.

[0193] like Figure 7 As shown, a particular advantage of the present technique is that the first time subsegment 2100 and the second time subsegment 2200 extend into the future relative to time instant T1. It is not always necessary to determine the tracking solution immediately, and using data analyzed in a time window extending into the past and / or future relative to a particular time instant enables a more accurate and reliable tracking solution to be obtained. The time delay between the end of the time windows 2100, 2200 and time instant T1 may be 1 second or longer.

[0194] As will be appreciated, during the initial training of the machine learning algorithm, the motion model for the training data will be defined without using the output from the algorithm, as the algorithm is untrained at this stage. However, as the machine learning algorithm is trained over time, it can be used to provide motion model parameters during a further first period of time in order to further refine the training data.

[0195] For example, motion model parameters for a particular user and device / sensor combination may be automatically learned by the machine learning algorithm 46 and refined over multiple such trips spanning a wide range of motion and location environments. For example, it may be determined that after multiple trips, the reliable step length of a user of a smartphone is 0.75m rather than the initially determined 0.8m, thereby providing a more accurate navigation solution. The motion model and its various parameters may be stored in a local storage device 5 on the device, or (e.g., using a client-server system) via other addressable storage devices and indexed by location environment and / or motion environment. Subsequently, when a particular motion and / or location environment is determined, the stored motion model and its corresponding parameters may be automatically selected from the addressable storage device.

[0196] The motion models and associated parameters may further be indexed by device and / or user, so that a motion model appropriate for the device and its current user may generally be automatically selected.

[0197] The tracking and navigation module 44 of the TSU 40 provides position solutions at multiple times in order to determine the trajectory of the device during the first time period.

[0198] like Figure 7 As shown, by analyzing the "rolling" windows of the first and second sub-data, position solutions at multiple moments can be obtained. In other embodiments, the first and second time sub-segments 2100, 2200 can be substantially the same length of time as the first time period, wherein the trajectory solution is obtained by batch analysis of the entire data set, such as Figure 8 Schematically shown in FIG.

[0199] The analysis performed by the tracking and navigation module 44 may include processing the acquired data both forwards and backwards in time, which allows any asymmetries in them to be used in determining their reliability and accuracy. Fig. 9 The one-dimensional position data obtained by the GNSS sensor 20 is schematically shown. In this example, the time T A With time T B The position changes between time period A and time period B are caused by erroneous position data. When processing the data forward in time, the gradual change in position during time period A can first be viewed as a possible "allowed" motion of the device (the allowed motion is constrained, for example, by the current motion model and / or measurements from auxiliary sensors). In this case, the erroneous GNSS position data from time period A is used to generate a navigation solution. The rapid change in position during time period B falls outside the allowed motion range of the device, indicating a potential problem with the GNSS position data.

[0200] However, by only processing forward in time, it is unclear at time T B Is the data before or after wrong? In a naive system that only uses forward analysis in time, at time T B Some or all of the “accurate” position data thereafter (shown here as “C”) may be rejected because they are now inconsistent with the navigation solution that has been corrupted by the erroneous data from time period A. In extreme cases, the system may never be able to recover the true navigation solution. B to T AThe same data of ( ) is processed backwards in time, and almost discontinuous jumps in position during time period B are detected as being inconsistent with the allowed motion of the device, and thus the position data is ignored (or assigned a lower confidence) when combining the data to obtain a navigation solution. This continues until such time as the position data is deemed consistent with the current navigation solution and the "allowed" motion of the device from the motion model. For example, at Fig. 9 In time T B to T A All or most of the erroneous position data will correctly be ignored (or given a lower confidence) by the TSU 40 during processing backwards in time.

[0201] The analysis may include iteratively processing the acquired data forward and / or backward in time. For example, a confidence threshold applied to an initial pass of the data may cause all acquired data to be allowed to be used in the trajectory solution provided by the tracking and navigation module 44, although such data may include erroneous results, such as in Fig. 9 The moment T depicted in A and T B However, it is important not to lose data that could represent the actual motion performed by the device. In subsequent passes of data, data from a specific time period (e.g., T A and T B Those that are between 1 and 2 (between 1 and 2) may not meet the new confidence threshold determined after the first pass and are therefore ignored or corrected. The benefit of processing the data backwards in time rather than just using multiple passes forward in time is that the uncertainty in any derived parameter increases with time in the absence of measurements. Therefore, by processing the data on either side of the region in two directions and combining the estimates provided by the two passes via a weighted average or similar calculation, it is possible to obtain a more accurate estimate of the confidence threshold at a time such as T. A and T B The overall estimate of the parameters is better in the region between .

[0202] The confidence threshold may be initially set based on known statistical noise characteristics of the device sensor or information derived from other sensors. For example, the accelerometer 12 may provide acceleration information in order to set a threshold for expected changes in GNSS frequency measurements during dynamic motion. When the data is subsequently transmitted, the bias on the accelerometer may be more accurately known (e.g., a zero velocity update analysis may be performed during an initial analysis that reveals bias information). This means that the acceleration data is more accurate and reliable, and therefore the confidence threshold for expected frequency changes may be tighter (i.e., vary less). As a result, any errors in the frequency measurement that were allowed during the first pass of the data may now be ignored on the second pass due to the tighter (more confident) confidence threshold.

[0203] Return to reference Fig. 9 As mentioned above, when the data is first passed, the initial confidence threshold can be such that at time T A and T B GNSS data acquired between 1 and 2 is deemed consistent with the "allowed" motion of the device from the motion model. During the course of the first pass processing, a better estimate of the aiding sensor bias is determined, which can be used on subsequent passes of the data to provide more accurate aiding sensor data, and therefore provide tighter confidence thresholds for other sensor data. For example, in a subsequent pass, it may now be found that at time T A and T B The GNSS position data between is inconsistent with the more tightly constrained allowed motion of the device and, as a result, is filtered out (or assigned a lower confidence) from the final trajectory solution.

[0204] Confidence thresholds and analysis of correlations forward and backward in time can be applied to sensor data as described, and also to inferred data and metrics of interest, including platform state, pose, and inferred or latent variables such as sensor biases and calibration parameters.

[0205] Return to reference Figure 7 , the duration of the first and second time subsegments 2100, 2200 is selected so as to allow the tracking and navigation module 44 to optimally assess the reliability and accuracy of the data obtained from the device sensors. Furthermore, in some embodiments, the duration of the first and / or second time subsegments 2100, 2200 may be dynamically changed in response to the assessment of the module 44. For example, the second time subsegment 2200 may be extended due to erroneous data determined in the time subsegment 2100, such that the extended time subsegment allows for more reliable interpretation of the data (e.g., to automatically select a motion model).

[0206] like Figure 7 As schematically seen in FIG. 1 , the first and second time subsegments 2100, 2200 may extend into the future relative to time T1. In other words, when a navigation solution is determined at time T1, future data about T1 is used to constrain the solution. This provides a more accurate determination of the evolution of the metric of interest compared to an instantaneous solution, since the evolution of the metric can be determined relative to time T1. 1 The constraints are interpolated for future time periods, rather than simply extrapolated. In other words, relative to time T 1 Extending the second time period into the future allows for a more accurate "overall picture" of the environment or data from which the evolution of the metric of interest will be determined.

[0207] However, there are some situations where real-time information may be needed, such as navigation. Fig.10 In these cases, the time T1 It can be at the current time or within a small interval after the current time. The relative position of the second time sub-segment 2100 and the first time sub-segment 2200 is also shown, where both time periods are at time T 1 End, and relative to T 1 Only extends into the past.

[0208] Thus, the trajectory of the device can be provided to the user in near real time, where the solution at the current time is constrained only by past data. Advantageously, however, the position solution at previous times can be continuously updated as new data is obtained. For example, at time T 0 , the first and second time periods are relative to T 0 Extending into the future, so in T 0 At time T 0 Compared with the solution determined in a small interval after itself, it can be determined at T 0 In this way, the present invention provides a near real-time determination of the evolution of a metric of interest, as well as an improved determination of said evolution over time as more data is obtained.

[0209] Fig.11 6000 is a flowchart outlining the main steps of one embodiment of the present invention for determining the evolution of a metric of interest (e.g., a trajectory) using a motion model. In step 6010, a user of the smartphone 100 pre-selects a motion model, typically by interacting with a suitable GUI via the device's screen. As described above, the motion model may include a motion environment, a position environment, and at least one quantitative parameter describing the motion. In step 6010, the user may pre-select at least one of these components. For example, the user may simply select a motion model with a motion environment of "running."

[0210] At step 6020, the TSU 40 obtains data from the first time subsegment, and at step 6030 obtains data from the second time subsegment. Although these are set forth as separate steps in method 6000, it should be understood that the TSU 40 may obtain data from the first and second time subsegments substantially simultaneously. In other embodiments, data from the second time subsegment may be obtained before obtaining data from the first time subsegment.

[0211] At step 6040, the first and second sub-data are analyzed by the tracking and navigation module 44, as described above, and at step 6050, the module 44 determines the evolution of the metric of interest (e.g., trajectory) between the first and second moments in time. In the series of steps set forth in method 6000, the motion model is shown as being pre-selected prior to obtaining the data. However, it will be understood that the motion model may be selected by a user after obtaining the data, or may be determined based on analyzing the data itself (e.g., using a trained machine learning algorithm). Typically, step 6040 of analyzing the first and second sub-data includes comparing the first and second sub-data to each other and / or to the motion model.

[0212] Fig.12 7000 is a flowchart outlining the main steps of another embodiment of the present invention for determining the evolution of a metric during a situation where a motion model is automatically selected based on an analysis of first and second sub-data. In step 7010, the TSU 40 obtains first sub-data from a first time sub-segment and in step 7020 obtains second sub-data from a second time sub-segment. Fig.12 In the same manner as described, steps 7010 and 7020 may in some cases occur substantially simultaneously or in a reverse order.

[0213] In step 7030 , the first and second sub-data are analyzed by the tracking and navigation module 44 .

[0214] At step 7040, the motion and / or positional context of the device is determined based on the analysis performed at step 7030. This may be determined, for example, by a trained machine learning algorithm. In such an embodiment, the method may then proceed to step 7090 to determine a trajectory based on the analysis of the first and second sub-data and the determined motion and / or positional context.

[0215] In other embodiments, the method proceeds to decision step 7050, where it is determined whether motion model parameters corresponding to the current user, the device, and the location and / or motion environment determined in step 7040 are stored in an addressable storage device (e.g., local storage device 5). If such a motion model exists, the motion model is called (step 7080), and in step 7090, the navigation and tracking module 44 determines the evolution of the metric of interest between the first and second moments, where the evolution is constrained by the first data, the second data, and the motion model, the motion model including at least one parameter called from the local storage device. Quantitative motion model parameters may include, for example, an estimate of the user's step length. As described above, advantageously, such parameters may have been refined during a previous analysis of data obtained for the user by the device, for example using a trained machine learning algorithm.

[0216] If it is determined at step 7050 that there are no corresponding motion model parameters, the method moves to optional step 7060, where the determined motion and / or position environment is used to determine at least one parameter associated with the motion of the device. This may be performed using, for example, a trained machine learning algorithm. Optionally, this may be stored in an addressable storage device (step 7070), indexed by the determined motion and / or position environment and the identification device. Subsequently, at step 7090, module 44 determines the time T 1 and T 2 The evolution of a metric of interest between , wherein the evolution is constrained by the first data, the second data and the motion model.

[0217] Fig.13 8010 schematically illustrates how data from one or more sensors (eg, used as training data) may be used to determine a tracking solution or a navigation solution for the smartphone 100 described above, for example, during a first time period. 1 The position solution generated by TSU 40, where time T 1 At the current moment or within a small interval after the current moment. The position solution of the TSU is based on data from the auxiliary sensors of the IMU 10, which may have a sampling rate in the range of 100-1000 samples per second, so the time period of block 8010 may be on the order of 1-10 ms. In conventional solutions, this "instantaneous" data obtained from the IMU will simply be used to output a position solution at that moment in time, and therefore the evolution of the position solution will be subject to large error drift.

[0218] like Fig.13 As schematically shown, the present invention can utilize the time T from about TSU 1 Data obtained from past and future time periods in order to constrain the tracking solution at that time. Information from any of boxes 8010 to 8040 can be analyzed within and between boxes using various methods (e.g., using a Kalman filter) to optimally filter and combine all of the available data to produce an improved tracking solution. In this example, box 8020 labeled "Zero Speed ​​Correction" represents data obtained by the IMU 10 and which has been analyzed to determine whether the IMU 10 was stationary at the time during the time interval represented by box 8020. The period of zero speed detected can be used, for example, to reset the velocity measurement tracked by the TSU, or to determine a bias in the inertial sensors of the IMU 10, which can then be used to correct the error at time T. 1 Constrained TSU solution.

[0219] Box 8030 may represent data obtained from a GNSS sensor that may be used to provide position and velocity data to further constrain the TSU solution. Additionally, information from box 8020 may be used to filter out GNSS data that violates the confidently determined zero velocity condition in box 8030. The attitude data may be obtained from an accelerometer and / or a gyroscope and / or a magnetometer sensor.

[0220] The motion model schematically shown at box 8040 may have been pre-selected or automatically determined (e.g., by analyzing the raw IMU data using a trained machine learning algorithm or based on position, velocity, and posture data). The motion model may include one or more parameter models that describe certain aspects of the user's motion. Analysis may be performed on the sensor data contained in one or more of boxes 8010 to 8040 to extract values ​​that are then used as inputs to such a model. For example, cadence may be extracted from accelerometer data and then input to a function that models a pedestrian's step length, or raw accelerometer and gyroscope data may be passed to a trained neural network that is capable of predicting the speed and direction of motion. As described above, the motion model at box 8040 may be used to assist in filtering the input sensor data and constraining a navigation solution or a tracking solution.

[0221] Fig.13 Boxes 8020, 8030, and 8040 of different sizes (ie, time periods) are shown, but this is not necessary and is for illustration purposes only. However, advantageously, at time T 1 The data obtained and used to constrain the TSU solution have been obtained over a longer period of time and are relative to the time T 1 Extending into the past and into the future. The evolution of the metric of interest between a first time instant and a second time instant can be determined by combining positioning solutions (here positioning data determined from the TSUs) obtained at a plurality of such time instants.

[0222] As you will understand, Fig.13 The diagram is for illustrative purposes only, and the boxes may represent different sensor data flows and analyses. For example, during the second time period, there may be no available data from the primary positioning unit (eg, GNSS sensor).

[0223] The main example used in the embodiments is an example of data obtained from a first device (smartphone) and a second independent device (headphones), the second independent device being used to train a machine learning algorithm so that the data obtained from the second device can be used to provide a reliable estimate of a metric related to the motion of the subject. In another example, the vehicle has auxiliary sensors mounted on its chassis (second platform) that measure the acceleration and turning rate of the vehicle. The driver carries a smartphone (first platform) in his pocket with a built-in GNSS receiver that acts as the main positioning unit and sensors that measure acceleration, turning rate, etc. The data obtained from the sensors on the first platform during a first time period when the GNSS receiver is operating can be used to train a machine learning algorithm so that the sensors on the vehicle chassis can maintain a reliable trajectory of the vehicle during a second time period.

[0224] Other preferred aspects of the present invention are set out in the numbered clauses below. These aspects particularly relate to determining the evolution of a metric of interest during a first and / or second time period (e.g., a training metric during a first time period, or for performing analysis on third data obtained during a second time period).

[0225] [Numbered clause 1] A method of combining data from at least one sensor in order to determine the evolution of a metric of interest over time, wherein at least one sensor is configured to make measurements from which position or movement can be determined, the method comprising:

[0226] In a first time period, first data is obtained from a first sensor installed on a first platform;

[0227] In a second time period, obtaining second data from the first sensor and / or a second sensor installed on the first platform or the second platform;

[0228] determining an evolution of a metric of interest between a first moment in time and a second moment in time, wherein at least one of the first moment in time and the second moment in time is within at least one of the first time period and the second time period, and

[0229] Therein, the evolution of the metric of interest is constrained by at least one of the first data, the second data, and a motion model of the first platform and / or the second platform.

[0230] [Numbered Clause 2] The method according to Clause 1 also includes: analyzing the first data and / or the second data over at least a portion of the corresponding first time period and / or second time period to evaluate the reliability and / or accuracy of the first data and / or the second data.

[0231] [Numbered Clause 3] The method of clause 1 or 2, further comprising: analyzing the first data and / or the second data over at least a portion of the corresponding first time period and / or second time period to obtain corrected first data and / or corrected second data, wherein

[0232] The evolution of the metric of interest is constrained by at least one of the corrected first data, the corrected second data, and a motion model of the first platform and / or the second platform.

[0233] [Numbered Clause 4] A method according to clause 2 or 3, wherein the step of analyzing the first data and / or the second data includes: performing a self-consistency check.

[0234] [Numbered Clause 5] A method according to any one of clauses 2 to 4, wherein the step of analyzing the first data and / or the second data comprises comparing the data with each other and / or with a motion model.

[0235] [Numbered Clause 6] A method according to any of clauses 2 to 5, wherein the duration and / or amount of overlap of the first time period and the second time period relative to each other is selected to allow optimal analysis of the first data and / or the second data.

[0236] [Numbered Clause 7] A method according to Clause 6, wherein the duration of a time period and its overlap relative to (one or more) other time periods and the first moment and the second moment are dynamically adjusted in response to changes in the reliability and / or accuracy of associated sensor data.

[0237] [Numbered Clause 8] A method according to any of the preceding clauses, wherein at least one of the first data and the second data is analyzed backward in time.

[0238] [Numbered Clause 9] A method according to any of the preceding clauses, wherein the first data and / or the second data are analyzed iteratively forward and / or backward in time.

[0239] [Numbered Clause 10] A method according to Clause 9, wherein a first analysis of the first data and / or the second data is based on at least one confidence threshold, and at least one confidence threshold is modified after the first analysis; wherein a subsequent analysis of the first data and / or the second data is based on the modified confidence threshold.

[0240] [Numbered Clause 11] A method according to any of the preceding clauses, wherein the motion model includes at least one of a motion environment component and a location environment component, the motion environment component includes the motion environment of the first platform and / or the second platform, and the location environment component includes the location environment of the first platform and / or the second platform.

[0241] [Numbered Clause 12] A method according to any of the preceding clauses, wherein the motion model comprises at least one parameter that quantitatively describes an aspect of the motion, and / or at least one function that can be used to determine a parameter that quantitatively describes an aspect of the motion.

[0242] [Numbered Clause 13] A method according to clause 12, wherein at least one parameter is one of: pedestrian step length, pedestrian speed, pedestrian height, pedestrian leg length, stair step height, stair horizontal distance or compass heading to motion direction offset.

[0243] [Numbered Clause 14] A method according to clause 12 or 13, wherein at least one function is a function that determines a pedestrian's step length or rate.

[0244] [Numbered Clause 15] A method according to any of the preceding clauses, wherein at least one component of the motion model is manually defined by a user.

[0245] [Numbered Clause 16] A method according to any of the preceding clauses, wherein at least one component of the motion model is automatically determined based on an analysis of the first data and / or the second data.

[0246] [Numbered Clause 17] A method according to any of the preceding clauses, wherein, during determination of the evolution of at least one previous metric of interest, at least one component of the motion model is automatically determined based on analysis of data obtained from the first sensor and / or the second sensor.

[0247] [Numbered Clause 18] A method according to any of clauses 11 to 17, wherein at least one parameter and / or function of a given motion model is stored in an addressable storage device, indexed by at least one of the motion environment, the location environment, or the ID of the first platform and / or the second platform.

[0248] [Numbered Clause 19] A method according to any of the preceding clauses, wherein the motion model of the first platform and / or the second platform is a pre-selected motion model.

[0249] [Numbered Clause 20] A method according to any of the preceding clauses, wherein the motion model is an automatically selected motion model.

[0250] [Numbered Clause 21] According to the method of Clause 20, when subordinate to Clause 11, the motion model is automatically selected by analyzing data obtained by the first sensor and / or the second sensor during the first time period and / or the second time period and / or during the time period between the first moment and the second moment to determine at least one of the motion environment and the position environment between the first moment and the second moment.

[0251] [Numbered Clause 22] The method according to Clause 21 also includes: using at least one of the IDs of the first platform and / or the second platform, the determined motion environment, and the determined location environment to call at least one motion model parameter and / or function from an addressable storage device, and at least one motion model parameter and / or function is indexed using at least one of the IDs of the first platform and / or the second platform, the motion environment, and the location environment.

[0252] [Numbered Clause 23] The method according to Clause 20 or 21 also includes: determining at least one motion model parameter and / or function of the motion model, and storing the at least one motion model parameter and / or function in an addressable storage device, wherein the motion model parameter and / or function is indexed using at least one of the IDs of the first platform and / or the second platform, the motion environment, and the location environment.

[0253] [Numbered Clause 24] A method according to any of the preceding clauses, wherein constraining the evolution of the metric of interest comprises: determining a measurement bias of the first sensor and / or the second sensor and correcting the bias.

[0254] [Numbered Clause 25] A method according to any of the preceding clauses, wherein constraining the evolution of the metric of interest comprises determining at least one data point of the first data and / or the second data that does not correspond to the motion model, and correcting the at least one data point.

[0255] [Numbered Clause 26] A method according to any of the preceding clauses, wherein the second data is obtained from a second sensor mounted on the first platform.

[0256] [Numbered Clause 27] A method according to any one of clauses 1 to 25, wherein the second data is obtained from a second sensor mounted on a second platform different from the first platform.

[0257] [Numbered Clause 28] The method of clause 27, wherein the first platform and the second platform have at least one common motion component.

[0258] [Numbered Clause 29] A method according to clause 27 or 28, wherein the first platform and the second platform are mounted on the same subject of interest.

[0259] [Numbered Clause 30] A method according to any of the preceding clauses, wherein the first time period and the second time period at least partially overlap.

[0260] [Numbered Clause 31] A method according to any one of clauses 1 to 29, wherein the first time period and the second time period do not overlap.

[0261] [Numbered Clause 32] A method according to any of the preceding clauses, wherein at least one of the first time period and the second time period extends into the future relative to the second moment.

[0262] [Numbered Clause 33] A method according to any one of clauses 1 to 31, wherein the first time period and / or the second time period does not substantially extend into the future relative to the second moment in time.

[0263] [Numbered Clause 34] A method according to any of the preceding clauses, wherein the evolution of the metric of interest is calculated in near real time.

[0264] [Numbered Clause 35] A method according to any of clauses 1 to 33, wherein the metrics of interest are computed in a batch manner.

[0265] [Numbered Clause 36] A method according to any of the preceding clauses, wherein the second time period is longer than the first time period.

[0266] [Numbered Clause 37] A method according to any of the preceding clauses, wherein the second data is obtained from a second sensor different from the first sensor, and the first sensor has a faster data sampling rate than the second sensor.

[0267] [Numbered Clause 38] A method according to any of the preceding clauses, wherein the first sensor is an inertial measurement unit.

[0268] [Numbered Clause 39] A method according to any of the preceding clauses, wherein the first sensor is part of an inertial navigation system.

[0269] [Numbered Clause 40] A method according to any of the preceding clauses, wherein the first sensor includes at least one of: an accelerometer, a gyroscope, a magnetometer, a barometer, a GNSS unit, a radio frequency receiver, a pedometer, a camera, a light sensor, a pressure sensor, a strain sensor, a proximity sensor, a radar, and a lidar.

[0270] [Numbered Clause 41] A method according to any of the preceding clauses, wherein the second sensor includes at least one of: an accelerometer, a gyroscope, a magnetometer, a barometer, a GNSS unit, a radio frequency receiver, a pedometer, a camera, a light sensor, a pressure sensor, a strain sensor, a proximity sensor, a radar, and a lidar.

[0271] [Numbered Clause 42] The method according to any of the preceding clauses also includes: obtaining third data from at least one of the first sensor, the second sensor and the third sensor in a third time period; wherein the determination of the evolution of the metric of interest is also constrained by the third data.

[0272] [Numbered Clause 43] A method according to any of the preceding clauses, wherein the metric of interest includes at least one of: position, range, rate, speed, trajectory, altitude, compass heading, pace, step length, distance traveled, motion environment, location environment, output power, calorie count, sensor bias, sensor scale factor, and sensor alignment error.

[0273] [Numbered Clause 44] A method according to any of the preceding clauses, wherein the first platform is mounted on a subject of interest.

[0274] [Numbered Clause 45] A computer-readable medium comprising executable instructions that, when executed by a computer, cause the computer to perform the method of any of the preceding clauses.

[0275] [Numbered Clause 46] A system comprising:

[0276] at least one sensor configured to perform measurements from which a position or movement can be determined in order to determine the evolution of a metric of interest, and

[0277] A processor adapted to perform the following steps:

[0278] In a first time period, first data is obtained from a first sensor installed on a first platform;

[0279] In a second time period, obtaining second data from the first sensor and / or the second sensor installed on the first platform or the second platform;

[0280] determining an evolution of a metric of interest between a first moment in time and a second moment in time, wherein at least one of the first moment in time and the second moment in time is within at least one of a first time period and a second time period;

[0281] The evolution of the metric of interest is constrained by at least one of the first data, the second data, and a motion model of the first platform and / or the second platform.

[0282] [Numbered Clause 47] A system according to clause 46, wherein the first sensor is an inertial measurement unit.

[0283] [Numbered Clause 48] A system according to clause 46 or 47, wherein the first sensor is part of an inertial navigation system.

[0284] [Numbered Clause 49] A system according to any of clauses 46 to 48, wherein the first sensor includes at least one of: an accelerometer, a gyroscope, a magnetometer, a barometer, a GNSS unit, a radio frequency receiver, a pedometer, a camera, a light sensor, a pressure sensor, a strain sensor, a proximity sensor, a radar, and a lidar.

[0285] [Numbered Clause 50] A system according to any of clauses 46 to 49, wherein the second sensor includes at least one of: an accelerometer, a gyroscope, a magnetometer, a barometer, a GNSS unit, a radio frequency receiver, a pedometer, a camera, a light sensor, a pressure sensor, a strain sensor, a proximity sensor, a radar, and a lidar.

[0286] [Numbered Clause 51] A system according to any of clauses 46 to 50, wherein the metric of interest includes at least one of: position, range, rate, speed, trajectory, altitude, compass heading, pace, step length, distance traveled, motion environment, location environment, output power, calorie count, sensor bias, sensor scale factor, and sensor alignment error.

[0287] [Numbered Clause 52] A system according to any of clauses 46 to 51, comprising a second sensor mounted on the first platform.

[0288] [Numbered Clause 53] A system according to any of clauses 46 to 51, comprising a second sensor mounted on a second platform different from the first platform.

[0289] [Numbered Clause 54] A system according to clause 53, wherein the first platform and the second platform have at least one common motion component.

[0290] [Numbered Clause 55] A system according to clause 53 or 54, wherein the first platform and the second platform are mounted on the same subject of interest.

Claims

1. A computer-implemented method performed in a tracking system for tracking motion of a subject over time, the method comprising: (a) obtaining first data related to the movement of the subject from at least one master positioning unit during a first time period, wherein the at least one master positioning unit is mounted on a first platform carried by the subject, or wherein the at least one master positioning unit is separate from the subject, the master positioning unit being operative during the first time period, the at least one master positioning unit being configured to directly provide position data and navigation data; (b) obtaining, during the first time period, second data from one or more auxiliary sensors, the one or more auxiliary sensors being configured to make measurements from which position or movement can be determined, whereby the second data comprises measurements from which position or movement can be determined, the one or more auxiliary sensors being mounted on one or more second platforms carried on the body, wherein the first platform and the one or more second platforms are capable of moving independently of each other; (c) generating first training data comprising the first data and the second data, the first training data comprising orientation information of the first platform and the one or more second platforms, and at least one training metric related to the motion of the subject during the first time period, wherein the training metric is determined using at least the first data obtained from the at least one master positioning unit; (d) obtaining, during a second time period, third data from the one or more auxiliary sensors, the third data comprising measurements from which position or movement can be determined; and (e) analyzing the third data using a first algorithm trained using the first training data to estimate at least one first metric related to the movement of the subject during the second time period, the third data comprising a relative orientation between the first platform and the one or more second platforms during the period in which the first data and the second data were obtained.

2. The method according to claim 1, wherein: The first data is obtained from at least one main positioning unit mounted on the first platform and from at least one main positioning unit separated from the main body.

3. The method according to claim 1 or 2, wherein: The first data and the second data are obtained for a plurality of motion environments of the subject, wherein the motion environment of the subject during the second time period corresponds to the motion environment during the first time period.

4. The method according to claim 1 or 2, wherein: The first data and the second data are obtained for a plurality of position environments of the one or more second platforms relative to the subject, wherein the position environment of the one or more second platforms relative to the subject during the second time period corresponds to the position environment during the first time period.

5. The method according to claim 1, wherein: The at least one first metric estimated in step (e) is at least one of: a direction of motion, a velocity, a speed, a motion context of the subject, and a position context of the one or more second platforms relative to the subject.

6. The method according to any one of claims 1, 2 and 5, wherein: The first algorithm comprises a neural network.

7. The method according to any one of claims 1, 2 and 5, further comprising: The third data is analyzed to estimate an evolution of at least one second metric related to the motion of the subject during the second time period, wherein the evolution of the at least one second metric is constrained by the at least one first metric estimated using the first algorithm.

8. The method according to claim 7, wherein: The third data is analyzed to estimate a trajectory of the subject during the second time period.

9. The method according to claim 7, wherein: Analyzing the third data comprises comparing the third data with the at least one first metric to obtain corrected third data, wherein the estimation of the evolution of the second metric is based on the corrected third data.

10. The method according to claim 9, wherein: The obtaining of the corrected third data comprises determining a measurement bias of the one or more auxiliary sensors, and correcting the measurement bias to obtain the corrected third data.

11. The method according to any one of claims 8 to 10, wherein: An estimate of the evolution of the second metric is constrained by the at least one first metric using a Kalman filter.

12. The method according to any one of claims 1, 2, 5 and 8-10, wherein: In step (e), the third data is arranged as a plurality of frames, each frame comprising a plurality of measurements from the one or more auxiliary sensors in time sequence, wherein the first algorithm is used to provide an estimate of at least one first metric for each of the frames.

13. The method according to claim 12, wherein: In step (c), the second data is arranged into a plurality of frames, each frame comprising a plurality of measurements from the at least one auxiliary sensor in time sequence, wherein: The time length of the frame of the second data and the time length of the frame of the third data are the same.

14. The method according to claim 13, wherein: The plurality of frames have a time length based on at least one of a determined position context and a determined motion context during the respective first and second time periods.

15. The method according to claim 1, further comprising: (a1) determining a training position environment of the one or more second platforms relative to the subject during the first time period, wherein the first training data includes the training position environment.

16. The method according to claim 15, further comprising the steps of: (a2) determining a training movement environment of the subject during the first time period, wherein the first training data includes the training movement environment.

17. The method according to claim 16, wherein: At least one of the training position environment and the training movement environment is determined by analyzing at least one of the first data and the second data or by user input.

18. The method according to any one of claims 1, 2, 5, 8-10 and 15-16, wherein The first algorithm is selected from a set of predetermined algorithms for estimating the at least one first metric, wherein the selection is based on the at least one first metric to be determined.

19. The method of claim 1, further comprising: An evolution of the training metric associated with the subject's movement during the first time period is determined, wherein the first training data includes at least one of the evolution of the training metric and a trajectory of the subject during the first time period.

20. The method according to claim 19, wherein: The evolution of the training metric is determined based on the first data and the second data obtained during the first time period.

21. The method according to claim 19 or 20, further comprising: Fourth data is obtained from at least one further auxiliary sensor mounted on the first platform carried on the body during the first time period, wherein the first data and at least one of the second data and the fourth data are used to determine an evolution of the training metric.

22. The method according to claim 19, wherein: The step of determining the evolution of the training metric comprises: In a first time sub-segment within the first time period, obtaining first sub-data from the at least one main positioning unit; obtaining second sub-data from the one or more auxiliary sensors and / or at least one further auxiliary sensor mounted on the first platform in a second time sub-period within the first time period; comparing the first sub-data and the second sub-data with each other and / or with a motion model of the subject during a first time period to obtain corrected first sub-data and / or corrected second sub-data; and An evolution of the training metric related to the movement of the subject during a first time period is determined based on the corrected first sub-data and / or the corrected second sub-data.

23. The method according to claim 22, wherein: Comparing the first sub-data and / or the second sub-data with the motion model comprises: performing a self-consistency check.

24. The method according to claim 22, wherein: Comparing the first sub-data with the second sub-data with each other and / or with the movement model comprises determining measurement deviations of at least one auxiliary sensor and / or the at least one main positioning unit and correcting the deviations to obtain corrected sub-data.

25. The method according to any one of claims 22 to 24, wherein: The duration and / or amount of overlap of the first and second time sub-periods is selected based on a reliability and / or accuracy analysis of data obtained from the one or more auxiliary sensors and at least one primary positioning unit during the first time period.

26. The method of claim 22, wherein: At least one of the first partial data and the second partial data is analyzed backward in time in order to obtain the corrected first partial data and / or the corrected second partial data.

27. The method of claim 22, wherein: At least one of the first sub-data and the second sub-data is analyzed iteratively forward and backward in time in order to obtain the corrected first sub-data and / or the corrected second sub-data.

28. The method of claim 22, wherein: The motion model includes at least one of: a positional context of the one or more second platforms relative to the subject during the first time period, and a motion context of the subject during the first time period.

29. The method according to any one of claims 1, 2, 5, 8-10, 13-17, 19-20, 22-24 and 26-28, further comprising the steps of: (f) obtaining, during a third time period, fifth data related to the movement of the subject from the at least one master positioning unit; (g) obtaining sixth data from the one or more auxiliary sensors during the third time period; (h) determining, based at least in part on the fifth data, an evolution of a second training metric associated with the subject's movement during the third time period; (i) analyzing the sixth data to estimate an evolution of the second training metric during the third time period, wherein the evolution of the estimated second training metric is constrained by at least one first metric, the at least one first metric is estimated using the first algorithm, the first algorithm is trained using the first training data; (j) comparing the determined evolution of the second training metric with the estimated evolution of the second training metric, and if the difference between the determined evolution of the second training metric and the estimated evolution of the second training metric is greater than a predetermined threshold, updating the first training data using second training data, wherein the second training data includes the fifth data and the sixth data.

30. The method according to claim 7, wherein: Estimating the evolution of the at least one second metric comprises: In a third time subsegment within the second time period, obtaining third sub-data from a first auxiliary sensor mounted on the second platform; In a fourth time subsegment within the second time period, fourth sub-data are obtained from the first auxiliary sensor and / or another auxiliary sensor mounted on the second platform or another second platform carried by the main body; comparing the third sub-data and the fourth sub-data with each other and / or with the estimated first metric to obtain corrected third sub-data and / or corrected fourth sub-data; and An evolution of the second metric during the second time period is estimated based on the corrected third sub-data and / or the corrected fourth sub-data.

31. The method according to claim 30, wherein: Comparing the third sub-data and the fourth sub-data with the first metric includes performing a self-consistency check.

32. The method according to claim 30 or 31, wherein: At least one of the third partial data and the fourth partial data is analyzed backward in time in order to obtain the corrected third partial data and / or the corrected fourth partial data.

33. The method according to claim 30 or 31, wherein: At least one of the third sub-data and the fourth sub-data is analyzed iteratively forward and backward in time in order to obtain the corrected third sub-data and / or the corrected fourth sub-data.

34. A tracking system for tracking the motion of a subject over time, the system comprising: at least one master positioning unit, the at least one master positioning unit being mounted on a first platform capable of being carried on the subject, or wherein the master positioning unit is separate from the subject, wherein the at least one master positioning unit is configured to directly provide position data and navigation data; one or more auxiliary sensors configured to make measurements from which position or movement can be determined, the one or more auxiliary sensors being mounted on one or more second platforms capable of being carried on the main body; and A processor adapted to perform the following steps: (a) obtaining first data from the at least one main positioning unit during a first time period in which the at least one main positioning unit operates, wherein during the first time period, the at least one main positioning unit is mounted on the first platform carried by the main body, or the at least one main positioning unit is separated from the main body; (b) obtaining, during the first time period, second data from the one or more auxiliary sensors, the second data comprising measurements from which position or movement can be determined, the one or more auxiliary sensors being mounted on one or more second platforms carried on the body, wherein the first platform and the one or more second platforms are capable of moving independently of each other; (c) generating first training data comprising the first data and the second data, the first training data comprising orientation information of the first platform and the one or more second platforms, and at least one training metric related to the motion of the subject during the first time period, wherein the training metric is determined using at least the first data obtained from the at least one master positioning unit; (d) obtaining third data from the one or more auxiliary sensors during a second period of time while the one or more second platforms are carried on the body, the third data comprising measurements from which position or movement can be determined; and (e) analyzing the third data using a first algorithm trained using the first training data to estimate at least one first metric related to the movement of the subject during the second time period, the third data comprising a relative orientation between the first platform and the one or more second platforms during the period in which the first data and the second data were obtained.

35. The system of claim 34, further comprising at least one additional auxiliary sensor mounted on the first platform.

36. A system according to any one of claims 34 to 35, wherein: The processor is further adapted to perform the following steps: analyzing the third data to estimate the evolution of at least one second metric related to the movement of the subject during the second time period, wherein the evolution of the at least one second metric is constrained by the at least one first metric estimated using the first algorithm.

37. The system of claim 34, wherein: The processor is adapted to perform the method according to any one of claims 1, 2, 5, 8-10, 13-17, 19-20, 22-24, 26-28 and 30-31.

38. The system according to any one of claims 34-35, wherein: The one or more auxiliary sensors include at least one of: an accelerometer, a gyroscope, a magnetometer, a barometer, a pedometer, a light sensor, a pressure sensor, a strain sensor, a proximity sensor, and a camera.

39. The system according to any one of claims 34-35, wherein: The at least one main positioning unit comprises at least one of: a GNSS unit, a camera, a radar and a lidar.

40. The system according to any one of claims 34-35, wherein: The one or more auxiliary sensors are part of an inertial navigation system.

Images