Pedestrian navigation heading constraint method and apparatus, electronic device, storage medium and program product

Abstract

The present disclosure relates to a pedestrian navigation heading constraint method, device, electronic equipment, storage medium and program product. The method comprises: obtaining inertial data of a target pedestrian; based on the inertial data, extracting the heading of the target pedestrian at the current time; in response to the heading of the target pedestrian within a first sliding window satisfying a first preset straight line condition, based on the difference information between the heading at the current time and the heading of the target pedestrian within the first sliding window, and / or the error distribution of the heading of the target pedestrian within a second sliding window, performing heading constraint processing on the target pedestrian at the current time, wherein the right boundaries of the first sliding window and the second sliding window are both the current time, and the length of the second sliding window is greater than the length of the first sliding window.

Description

Technical Field

[0001] This disclosure relates to the field of indoor pedestrian navigation technology, and in particular to a pedestrian navigation heading constraint method, a pedestrian navigation heading constraint device, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] Dead reckoning (DR) is the process of estimating a future location using a known starting position and estimated speed and direction of movement over time, without any external references. Extending this concept to human motion models, calculating stride length and stride frequency based on the patterns and periodic characteristics of pedestrian movement constitutes pedestrian dead reckoning (PDR).

[0003] Pedestrian dead reckoning uses inertial measurement units (gyroscopes and accelerometers) to estimate a pedestrian's heading and position, offering advantages such as autonomy, real-time performance, and low power consumption. Given initial information, the pedestrian's attitude can be updated by measuring the angular velocity or angular increment from the gyroscope, obtaining the target's direction of motion in the navigation coordinate system. Then, the accelerometer data is used to estimate the pedestrian's step length and number of steps, or by directly integrating the accelerometer data, the pedestrian's velocity and position can be obtained. This is the basic principle of indoor pedestrian navigation.

[0004] Pedestrian dead reckoning requires no prior information such as maps, nor additional base station deployments or external signals. Its high degree of autonomy and continuity has attracted widespread attention from scholars. Regarding the specific implementation of dead reckoning, scholars have proposed the following two methods:

[0005] The first method uses the traditional strapdown inertial navigation system update algorithm, which mainly consists of three parts: attitude update, velocity update, and position update. This method is simple and fast, and can achieve high-precision dead reckoning by applying methods such as zero-velocity correction. It also does not require complex modeling of pedestrian stride length and stride frequency models. However, its drawback is that it does not make good use of human motion models.

[0006] The second method involves processing and analyzing accelerometer data to detect the pedestrian's stride length and stride frequency. Methods include adaptive stride length estimation, neural network stride length estimation, autocorrelation frequency measurement, zero-point crossover frequency measurement, and peak frequency measurement. Simultaneously, attitude updates are performed based on gyroscope data to obtain the pedestrian's direction of motion. This method offers relatively high positioning accuracy and slow error divergence, but the model is complex and its robustness across different pedestrians is low.

[0007] It is evident that the accuracy of the pedestrian dead reckoning system gradually decreases over time due to the continuous accumulation of errors in the inertial navigation system during integration. In particular, errors in attitude updates can cause a significant drop in the positioning accuracy of the entire navigation system if there is a large deviation in attitude. Summary of the Invention

[0008] This disclosure provides a pedestrian navigation heading constraint technical solution.

[0009] According to one aspect of this disclosure, a pedestrian navigation heading constraint method is provided, comprising:

[0010] Obtain the inertial data of the target pedestrian;

[0011] Based on the inertial data, the heading of the target pedestrian at the current moment is extracted;

[0012] In response to the target pedestrian's heading within the first sliding window satisfying a first preset straight-ahead condition, based on the difference information between the current heading and the target pedestrian's heading within the first sliding window, and / or the error distribution of the target pedestrian's heading within the second sliding window, a heading constraint process is applied to the target pedestrian at the current moment, wherein the right boundary of both the first and second sliding windows is the current moment, and the length of the second sliding window is greater than the length of the first sliding window.

[0013] In one possible implementation, obtaining the inertial data of the target pedestrian includes:

[0014] The inertial data of the target pedestrian is obtained by using an inertial measurement unit installed on the foot of the target pedestrian.

[0015] In one possible implementation, extracting the target pedestrian's heading at the current moment based on the inertial data includes:

[0016] Based on the inertial data, the moment when the target pedestrian's foot fully touches the ground is extracted;

[0017] At the moment when the foot is fully on the ground, the heading of the target pedestrian is extracted based on the inertial data to obtain the heading of the target pedestrian at the current moment.

[0018] In one possible implementation, extracting the moment when the target pedestrian's foot fully touches the ground based on the inertial data includes:

[0019] For any given moment, extract the amplitude of the inertial data at that moment, and calculate the standard deviation of the inertial data within the third sliding window, wherein the right boundary of the third sliding window is the given moment;

[0020] In response to the amplitude being less than an amplitude threshold and the standard deviation being less than a standard deviation threshold, the moment is determined to be the moment when the target pedestrian's foot is fully on the ground.

[0021] In one possible implementation,

[0022] The amplitude of the inertial data includes: specific force amplitude and angular velocity amplitude;

[0023] The standard deviation of the inertial data includes: the standard deviation of specific force and the standard deviation of angular velocity;

[0024] The amplitude thresholds include: specific force amplitude threshold and angular velocity amplitude threshold;

[0025] The standard deviation thresholds include: specific force standard deviation threshold and angular velocity standard deviation threshold;

[0026] The step of determining the moment when the target pedestrian's foot is fully on the ground in response to the amplitude being less than the amplitude threshold and the standard deviation being less than the standard deviation threshold includes: determining the moment when the target pedestrian's foot is fully on the ground in response to the specific force amplitude being less than the specific force amplitude threshold, the angular velocity amplitude being less than the angular velocity amplitude threshold, the specific force standard deviation being less than the specific force standard deviation threshold, and the angular velocity standard deviation being less than the angular velocity standard deviation threshold.

[0027] In one possible implementation, extracting the target pedestrian's heading at the current moment based on the inertial data includes:

[0028] Based on the inertial data, the b-frame is updated using the equivalent rotation vector method to obtain the quaternion of the b-frame attitude transformation;

[0029] Based on the inertial data, the n-system is updated using the equivalent rotation vector method to obtain the quaternion of the n-system attitude transformation;

[0030] The quaternion of the current posture of the target pedestrian is calculated based on the quaternion of the b-series posture transformation and the quaternion of the n-series posture transformation.

[0031] Based on the quaternion of the current posture, the heading of the target pedestrian at the current moment is obtained.

[0032] In one possible implementation, the first preset straight-ahead condition includes:

[0033] The standard deviation of the heading is less than the standard deviation threshold of the heading.

[0034] In one possible implementation, the method further includes:

[0035] If the heading of the target pedestrian within the first sliding window does not meet the first preset straight-ahead condition, it is determined that the heading constraint is invalid at the current moment.

[0036] In one possible implementation, the heading constraint processing of the target pedestrian at the current moment, based on the difference information between the heading at the current moment and the heading of the target pedestrian within the first sliding window, and / or the error distribution of the heading of the target pedestrian within the second sliding window, includes:

[0037] In response to the fact that the difference between the heading at the current moment and the heading of the target pedestrian within the first sliding window satisfies the second preset straight-ahead condition, a heading constraint is imposed on the target pedestrian at the current moment;

[0038] In response to the fact that the error distribution of the target pedestrian's heading within the second sliding window does not conform to a Gaussian distribution, fault repair is performed on the heading constraints already applied within the second sliding window.

[0039] In one possible implementation, the difference between the current heading and the heading of the target pedestrian within the first sliding window satisfies a second preset straight-ahead condition, including:

[0040] The absolute value of the difference between the current heading and the average heading of the target pedestrian within the first sliding window is less than the heading threshold.

[0041] In one possible implementation, applying a heading constraint to the target pedestrian at the current moment includes:

[0042] At the current moment, a heading constraint is applied to the target pedestrian using a first Kalman filter.

[0043] In one possible implementation, the fault repair of the heading constraints already applied within the second sliding window includes:

[0044] The second Kalman filter is used to repair the heading constraints that have been applied within the second sliding window.

[0045] In one possible implementation, the heading constraint processing of the target pedestrian at the current moment based on the difference information between the heading at the current moment and the heading of the target pedestrian within the first sliding window, and / or the error distribution of the heading of the target pedestrian within the second sliding window, further includes:

[0046] If the difference between the heading at the current moment and the heading of the target pedestrian within the first sliding window does not meet the second preset straight-ahead condition, it is determined that the heading constraint is invalid at the current moment.

[0047] According to one aspect of this disclosure, a pedestrian navigation heading constraint device is provided, comprising:

[0048] The acquisition module is used to obtain the inertial data of the target pedestrian;

[0049] An extraction module is used to extract the heading of the target pedestrian at the current moment based on the inertial data;

[0050] A heading constraint processing module is used to perform heading constraint processing on the target pedestrian at the current moment in response to the target pedestrian's heading in the first sliding window satisfying a first preset straight-ahead condition, based on the difference information between the heading at the current moment and the target pedestrian's heading in the first sliding window, and / or the error distribution of the target pedestrian's heading in the second sliding window. The right boundaries of the first sliding window and the second sliding window are both at the current moment, and the length of the second sliding window is greater than the length of the first sliding window.

[0051] In one possible implementation, the obtaining module is used for:

[0052] The inertial data of the target pedestrian is obtained by using an inertial measurement unit installed on the foot of the target pedestrian.

[0053] In one possible implementation, the extraction module is used for:

[0054] Based on the inertial data, the moment when the target pedestrian's foot fully touches the ground is extracted;

[0055] At the moment when the foot is fully on the ground, the heading of the target pedestrian is extracted based on the inertial data to obtain the heading of the target pedestrian at the current moment.

[0056] In one possible implementation, the extraction module is used for:

[0057] For any given moment, extract the amplitude of the inertial data at that moment, and calculate the standard deviation of the inertial data within the third sliding window, wherein the right boundary of the third sliding window is the given moment;

[0058] In response to the amplitude being less than an amplitude threshold and the standard deviation being less than a standard deviation threshold, the moment is determined to be the moment when the target pedestrian's foot is fully on the ground.

[0059] In one possible implementation,

[0060] The amplitude of the inertial data includes: specific force amplitude and angular velocity amplitude;

[0061] The standard deviation of the inertial data includes: the standard deviation of specific force and the standard deviation of angular velocity;

[0062] The amplitude thresholds include: specific force amplitude threshold and angular velocity amplitude threshold;

[0063] The standard deviation thresholds include: specific force standard deviation threshold and angular velocity standard deviation threshold;

[0064] The extraction module is used to: determine the moment when the target pedestrian's foot is fully on the ground in response to the following: the force amplitude is less than the force amplitude threshold, the angular velocity amplitude is less than the angular velocity amplitude threshold, the force standard deviation is less than the force standard deviation threshold, and the angular velocity standard deviation is less than the angular velocity standard deviation threshold.

[0065] In one possible implementation, the extraction module is used for:

[0066] Based on the inertial data, the b-frame is updated using the equivalent rotation vector method to obtain the quaternion of the b-frame attitude transformation;

[0067] Based on the inertial data, the n-system is updated using the equivalent rotation vector method to obtain the quaternion of the n-system attitude transformation;

[0068] The quaternion of the current posture of the target pedestrian is calculated based on the quaternion of the b-series posture transformation and the quaternion of the n-series posture transformation.

[0069] Based on the quaternion of the current posture, the heading of the target pedestrian at the current moment is obtained.

[0070] In one possible implementation, the first preset straight-ahead condition includes:

[0071] The standard deviation of the heading is less than the standard deviation threshold of the heading.

[0072] In one possible implementation, the device further includes:

[0073] The failure determination module is used to determine that the heading constraint is invalid at the current moment in response to the target pedestrian's heading within the first sliding window not meeting the first preset straight-ahead condition.

[0074] In one possible implementation, the heading constraint processing module is used to:

[0075] In response to the fact that the difference between the heading at the current moment and the heading of the target pedestrian within the first sliding window satisfies the second preset straight-ahead condition, a heading constraint is imposed on the target pedestrian at the current moment;

[0076] In response to the fact that the error distribution of the target pedestrian's heading within the second sliding window does not conform to a Gaussian distribution, fault repair is performed on the heading constraints already applied within the second sliding window.

[0077] In one possible implementation, the difference between the current heading and the heading of the target pedestrian within the first sliding window satisfies a second preset straight-ahead condition, including:

[0078] The absolute value of the difference between the current heading and the average heading of the target pedestrian within the first sliding window is less than the heading threshold.

[0079] In one possible implementation, the heading constraint processing module is used to:

[0080] At the current moment, a heading constraint is applied to the target pedestrian using a first Kalman filter.

[0081] In one possible implementation, the heading constraint processing module is used to:

[0082] The second Kalman filter is used to repair the heading constraints that have been applied within the second sliding window.

[0083] In one possible implementation, the heading constraint processing module is used to:

[0084] If the difference between the heading at the current moment and the heading of the target pedestrian within the first sliding window does not meet the second preset straight-ahead condition, it is determined that the heading constraint is invalid at the current moment.

[0085] According to one aspect of this disclosure, an electronic device is provided, comprising: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the executable instructions stored in the memory to perform the method described above.

[0086] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the above-described method.

[0087] According to one aspect of this disclosure, a computer program product is provided, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in an electronic device, a processor in the electronic device performs the above-described method.

[0088] In this embodiment, by obtaining the inertial data of the target pedestrian, the heading of the target pedestrian at the current moment is extracted based on the inertial data. In response to the target pedestrian's heading within a first sliding window satisfying a first preset straight-ahead condition, the heading is constrained at the current moment based on the difference between the current heading and the target pedestrian's heading within the first sliding window, and / or the error distribution of the target pedestrian's heading within a second sliding window. The right boundary of both the first and second sliding windows is the current moment, and the length of the second sliding window is greater than the length of the first sliding window. Therefore, prior information such as maps is not required. The system verifies whether the pedestrian is walking straight by statistically analyzing the error distribution of the heading during multiple steps (i.e., the heading within the larger second sliding window). If the pedestrian is walking straight, the heading is constrained each time the pedestrian's foot touches the ground. This embodiment can constrain the heading of a pedestrian navigation system in scenarios such as long indoor corridors, solving the problem of dependence on prior building information.

[0089] Heading error is the main error in pedestrian navigation. By adopting the pedestrian navigation heading constraint method provided in this embodiment, the accuracy of pedestrian dead reckoning based on low-cost inertial measurement units can be improved, and it can be applied to smart devices such as mobile phones and smart bracelets.

[0090] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0091] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0092] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0093] Figure 1 A flowchart illustrating the pedestrian navigation heading constraint method provided in an embodiment of this disclosure is shown.

[0094] Figure 2 This illustrates the several states that a pedestrian's feet experience during movement.

[0095] Figure 3 This diagram illustrates the gait extraction method in the pedestrian navigation heading constraint method provided in this embodiment of the present disclosure.

[0096] Figure 4 A schematic diagram of the pedestrian navigation heading constraint method provided in an embodiment of this disclosure is shown.

[0097] Figure 5 This diagram illustrates a scenario involving pedestrians on a long-radius turning section of a road.

[0098] Figure 6 This diagram illustrates the heading constraint verification in the pedestrian navigation heading constraint method provided in this embodiment of the present disclosure.

[0099] Figure 7 This diagram illustrates a second sliding window for heading constraint verification in the pedestrian navigation heading constraint method provided in an embodiment of this disclosure.

[0100] Figure 8 A block diagram of a pedestrian navigation heading constraint device provided in an embodiment of this disclosure is shown.

[0101] Figure 9 A block diagram of an electronic device 1900 provided in an embodiment of this disclosure is shown. Detailed Implementation

[0102] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0103] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0104] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0105] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0106] In summary, the relevant technologies have the following drawbacks:

[0107] First, the positioning accuracy of indoor pedestrian navigation is closely related to hardware performance. Low-cost indoor pedestrian navigation systems can often only maintain high accuracy for a short period of time.

[0108] Secondly, in indoor scenarios, the spatial structure is characterized by a long and narrow area with structures on both sides that strictly restrict pedestrian movement. In these situations, pedestrians exhibit very distinct movement characteristics, habitually walking along the center line of the road. However, current indoor pedestrian navigation algorithms rarely consider the indoor environment or constrain the errors of the navigation system.

[0109] Third, some scholars use the Manhattan model of buildings to constrain pedestrian navigation, but this requires prior information about the buildings, which is not conducive to its widespread application in practice.

[0110] Fourth, the heading constraint methods in related technologies are not well-suited for long-radius curves and are prone to causing incorrect heading constraints.

[0111] To address technical problems similar to those described above, this disclosure provides a pedestrian navigation heading constraint method. By obtaining the inertial data of a target pedestrian, the method extracts the pedestrian's heading at the current moment based on the inertial data. Responding to the pedestrian's heading within a first sliding window satisfying a first preset straight-ahead condition, and based on the difference between the current heading and the pedestrian's heading within the first sliding window, and / or the error distribution of the pedestrian's heading within a second sliding window, heading constraint processing is applied to the pedestrian at the current moment. The right boundaries of both the first and second sliding windows are at the current moment, and the length of the second sliding window is greater than the length of the first sliding window. Therefore, prior information such as maps is not required. The method verifies whether the pedestrian is walking straight by statistically analyzing the error distribution of the pedestrian's heading during multiple steps (i.e., the heading within the larger second sliding window). If the pedestrian is walking straight, the heading is constrained each time the pedestrian's foot touches the ground. This disclosure can constrain the heading of a pedestrian navigation system in scenarios such as long indoor corridors, solving the problem of dependence on prior building information.

[0112] Heading error is the main error in pedestrian navigation. By adopting the pedestrian navigation heading constraint method provided in this embodiment, the accuracy of pedestrian dead reckoning based on low-cost inertial measurement units can be improved, and it can be applied to smart devices such as mobile phones and smart bracelets.

[0113] The pedestrian navigation heading constraint method provided in this disclosure will be described in detail below with reference to the accompanying drawings.

[0114] Figure 1A flowchart illustrating a pedestrian navigation heading constraint method provided in an embodiment of this disclosure is shown. In one possible implementation, the entity executing the pedestrian navigation heading constraint method can be a pedestrian navigation heading constraint device. For example, the pedestrian navigation heading constraint method can be executed by a terminal device, a server, or other electronic equipment. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, vehicle-mounted device, or wearable device, etc. In some possible implementations, the pedestrian navigation heading constraint method can be implemented by a processor calling computer-readable instructions stored in memory. Figure 1 As shown, the pedestrian navigation heading constraint method includes steps S11 to S13.

[0115] In step S11, the inertial data of the target pedestrian is obtained.

[0116] In step S12, the heading of the target pedestrian at the current moment is extracted based on the inertial data.

[0117] In step S13, in response to the target pedestrian's heading within the first sliding window satisfying the first preset straight-ahead condition, based on the difference information between the current heading and the target pedestrian's heading within the first sliding window, and / or the error distribution of the target pedestrian's heading within the second sliding window, a heading constraint process is performed on the target pedestrian at the current time, wherein the right boundary of both the first and second sliding windows is the current time, and the length of the second sliding window is greater than the length of the first sliding window.

[0118] In this embodiment of the disclosure, the target pedestrian can represent the object of interest of the pedestrian navigation system. In the fields of pedestrian dead reckoning and indoor pedestrian navigation, it is necessary to measure and analyze the movement of the target pedestrian to provide accurate navigation information. The target pedestrian can be any individual using the pedestrian navigation system, such as consumers requiring navigation services in large shopping malls, airports, exhibitions, or other indoor environments, or pedestrians requiring precise navigation in outdoor environments with poor GPS (Global Positioning System) signals, such as cities or canyons.

[0119] In this embodiment of the disclosure, inertial data can represent data collected by an inertial measurement unit (IMU). Inertial data can include acceleration, angular velocity, etc., and can be used to calculate information such as the target pedestrian's stride length, stride frequency, attitude, and heading.

[0120] In one possible implementation, obtaining the inertial data of the target pedestrian includes acquiring the inertial data of the target pedestrian by means of an inertial measurement unit installed on the foot of the target pedestrian. In this implementation, by acquiring the inertial data of the target pedestrian by installing an inertial measurement unit on the foot of the target pedestrian, more accurate information on the moment of foot contact (i.e., the moment when the target pedestrian's foot is fully on the ground) and gait can be obtained, thereby improving the accuracy of indoor navigation.

[0121] In this embodiment of the disclosure, the heading of the target pedestrian can refer to the direction of movement of the target pedestrian in the navigation coordinate system.

[0122] In one possible implementation, extracting the target pedestrian's heading at the current moment based on the inertial data includes: extracting the moment when the target pedestrian's foot is fully on the ground based on the inertial data; and extracting the target pedestrian's heading based on the inertial data at the moment when the foot is fully on the ground to obtain the target pedestrian's heading at the current moment.

[0123] In this implementation, the foot movement of the target pedestrian can be monitored by analyzing the inertial data collected by the inertial measurement unit installed on the foot of the target pedestrian. Figure 2 This illustrates the several states a pedestrian's foot experiences during movement: heel strike, toes strike, foot fully on the ground, heel off the ground, and toes off the ground (i.e.,...). Figure 2 (The "foot moving in the air" in the text). When the foot is fully on the ground, it is in a relatively stationary state, and the heading is more accurate at this moment. Therefore, in this implementation method, the gait when the foot is fully on the ground can be used as the target for extraction.

[0124] In this implementation, the moment when the target pedestrian's foot is fully on the ground is extracted based on the inertial data, and the target pedestrian's heading is extracted based on the inertial data at the moment when the foot is fully on the ground, thus obtaining the target pedestrian's heading at the current moment. Therefore, by extracting the target pedestrian's heading at the current moment when the foot is fully on the ground, the heading estimation error caused by foot movement can be reduced, thereby improving the accuracy of pedestrian positioning.

[0125] In one possible implementation, extracting the moment when the target pedestrian's foot is fully on the ground based on the inertial data includes: for any given moment, extracting the amplitude of the inertial data at that moment, and calculating the standard deviation of the inertial data within a third sliding window, wherein the right boundary of the third sliding window is the moment; and determining that the moment is the moment when the amplitude is less than an amplitude threshold and the standard deviation is less than a standard deviation threshold in response to the moment when the target pedestrian's foot is fully on the ground.

[0126] In this implementation, for any given moment, the moment can be determined not to be the moment when the target pedestrian's foot is fully on the ground, in response to the moment when the amplitude of the inertial data is greater than or equal to an amplitude threshold, or the moment when the standard deviation of the inertial data within the third sliding window is greater than or equal to a standard deviation threshold.

[0127] Accurate gait extraction is crucial for pedestrian navigation systems. Only by precisely capturing the instant when a pedestrian's foot fully touches the ground can stride frequency be estimated, zero-velocity corrections made, and heading constraints imposed. In this implementation, for a pedestrian navigation system mounted on the foot, judging the amplitude of inertial data and the standard deviation of inertial data within the third sliding window can suppress noise and errors to a certain extent, thereby improving the accuracy of the extracted moment when the foot fully touches the ground.

[0128] In one possible implementation, the amplitude of the inertial data includes: specific force amplitude and angular velocity amplitude; the standard deviation of the inertial data includes: specific force standard deviation and angular velocity standard deviation; the amplitude threshold includes: specific force amplitude threshold and angular velocity amplitude threshold; the standard deviation threshold includes: specific force standard deviation threshold and angular velocity standard deviation threshold; the step of determining the moment when the target pedestrian's foot is fully on the ground in response to the amplitude being less than the amplitude threshold and the standard deviation being less than the standard deviation threshold includes: determining the moment when the target pedestrian's foot is fully on the ground in response to the specific force amplitude being less than the specific force amplitude threshold, the angular velocity amplitude being less than the angular velocity amplitude threshold, the specific force standard deviation being less than the specific force standard deviation threshold, and the angular velocity standard deviation being less than the angular velocity standard deviation threshold.

[0129] For an inertial measurement unit mounted on the foot, its six-axis output can be written as:

[0130] o = [f x ,f y ,f z ,ω x ,ω y ,ω z ] T

[0131] Among them, f i The specific force output that can represent the i-axis, ω i It can represent the angular velocity output along the i-axis.

[0132] In this implementation, the amplitude of the angular velocity and specific force measured by the inertial measurement unit and the standard deviation within the third sliding window can be taken into account, thereby accurately capturing the moment when the target pedestrian's foot is fully on the ground.

[0133] Figure 3This diagram illustrates a gait extraction method within the pedestrian navigation heading constraint method provided in this embodiment. In this implementation, the system first pauses for approximately 10 seconds before starting operation, taking the average output value during this period as a constant zero bias and subtracting it from subsequent data. Then, the three-axis specific force vector and amplitude, and the three-axis angular velocity vector and amplitude output at each moment are calculated. Next, a third sliding window of length n is set, and the standard deviation of angular velocity and specific force within each third sliding window are calculated. Since the foot is relatively stationary at the moment of full contact with the ground, the three-axis specific force vector and amplitude, three-axis angular velocity vector and amplitude, angular velocity standard deviation, and specific force standard deviation output by the inertial measurement unit should all be approximately 0. Therefore, in this implementation, appropriate thresholds for specific force amplitude f1, angular velocity amplitude ω1, specific force standard deviation f2, and angular velocity standard deviation ω2 can be set to extract the gait of the target pedestrian. If the result is less than the corresponding threshold (i.e., the specific force amplitude is less than the specific force amplitude threshold, the angular velocity amplitude is less than the angular velocity amplitude threshold, the specific force standard deviation is less than the specific force standard deviation threshold, and the angular velocity standard deviation is less than the angular velocity standard deviation threshold), then it indicates that the detection has been passed, meaning it can be determined that the target pedestrian's foot is completely on the ground. The following four expressions are used:

[0134]

[0135] in It can represent the vector sum of the measured values ​​of specific force. The vector sum of the measured values ​​that can represent angular velocity, σ f σ can represent the standard deviation of the force measurement. ω It can represent the standard deviation of angular velocity measurement.

[0136] like Figure 3 As shown, if the measurement results at a certain moment simultaneously pass the specific force amplitude detection, angular velocity amplitude detection, specific force sliding standard deviation detection, and angular velocity sliding standard deviation detection, it can be determined that the person is in a fully grounded state; otherwise, it can be determined that the foot is not fully grounded. That is, for any given moment, if the specific force amplitude is greater than or equal to the specific force amplitude threshold, it can be determined that this moment is not when the target pedestrian's foot is fully grounded; for any given moment, if the angular velocity amplitude is greater than or equal to the angular velocity amplitude threshold, it can be determined that this moment is not when the target pedestrian's foot is fully grounded; for any given moment, if the specific force standard deviation is greater than or equal to the specific force standard deviation threshold, it can be determined that this moment is not when the target pedestrian's foot is fully grounded; for any given moment, if the angular velocity standard deviation is greater than or equal to the angular velocity standard deviation threshold, it can be determined that this moment is not when the target pedestrian's foot is fully grounded.

[0137] In this embodiment of the disclosure, the heading of the target pedestrian at the current moment can be extracted based on the inertial data.

[0138] In one possible implementation, the target pedestrian's attitude can be updated based on the inertial data, and the target pedestrian's heading can be determined based on the updated attitude information.

[0139] For example, attitude update may include: using gyroscope data to determine angular velocity and then calculating the rotational change of attitude; using accelerometer data to correct attitude errors caused by gravity; and fusing the above data to obtain a more accurate attitude estimate.

[0140] As an example of this implementation, the attitude of a target pedestrian can be recursively obtained using a strapdown inertial navigation (SINS) attitude update algorithm. Strapdown inertial navigation is a navigation technique that uses inertial measurement units (IMUs). It does not rely on external references but determines its current position and attitude by calculating continuous motion from an initial position. For example, a strapdown inertial navigation attitude update algorithm may include: updating the heading using angular velocity measured by a gyroscope; updating velocity and position using acceleration measured by an accelerometer; and integrating the updated data using a mathematical model (such as a quaternion or rotation matrix) to obtain the current attitude estimate.

[0141] In one possible implementation, extracting the target pedestrian's heading at the current moment based on the inertial data includes: updating the b-frame using the equivalent rotation vector method based on the inertial data to obtain the quaternion of the b-frame attitude transformation; updating the n-frame using the equivalent rotation vector method based on the inertial data to obtain the quaternion of the n-frame attitude transformation; calculating the quaternion of the target pedestrian's current attitude based on the quaternion of the b-frame attitude transformation and the quaternion of the n-frame attitude transformation; and obtaining the target pedestrian's heading at the current moment based on the quaternion of the current attitude.

[0142] In this implementation, the first step can be to update the b-frame based on the inertial data using the equivalent rotation vector method, thereby obtaining the quaternion of the b-frame attitude transformation. for

[0143]

[0144] in It can represent the corresponding equivalent rotation vector. It can represent the magnitude of the equivalent rotation vector. If the bisample algorithm is used, then... Where Δθ k For the current angular increment output, Δθ k-1 The output is the angular increment of the previous epoch.

[0145] The second step involves updating the n-frame based on the inertial data using the equivalent rotation vector method, thereby obtaining the quaternion of the n-frame attitude transformation. for

[0146]

[0147] Where ξ k Let represent the equivalent rotation vector corresponding to the n-system transformation, then we have

[0148]

[0149] in This represents the rotation of the n-system caused by the Earth's rotation (the Earth's rotational angular velocity). This represents the rotation of the n-system caused by the curvature of the Earth's surface as the inertial navigation system moves near the Earth's surface. v represents the latitude (in radians). E and v N These are the eastward and northward speeds, R. M and R N These are the radii of the meridian circle and the radii of the east-west circle, respectively.

[0150] In inertial navigation and geophysics, the B-frame and N-frame are two commonly used coordinate systems for describing and calculating the motion of objects on Earth. The B-frame is a coordinate system fixed to the carrier (such as an aircraft, ship, or pedestrian) and moves and rotates with it. In strapdown inertial navigation systems, sensors (such as gyroscopes and accelerometers) are fixed to the carrier, and the measured data is relative to the B-frame. This data needs to be transformed to a geographic or navigation coordinate system for position and velocity calculations. The N-frame is a geographic coordinate system, typically fixed to the Earth and defined by latitude, longitude, and altitude. In navigation calculations, the N-frame can be used to define position and velocity. For example, in the Earth-Centered Inertial Coordinate System (ECI) or the local horizontal coordinate system, position and velocity are typically represented by the north, east, and down coordinates of the N-frame.

[0151] The third step is to calculate the quaternion of the target pedestrian's current pose (i.e., the quaternion of the target pedestrian's pose at the current moment), then we have

[0152]

[0153] in To represent quaternion multiplication, The quaternion representing the posture at the previous moment, This represents the attitude quaternion obtained recursively at the current moment.

[0154] As an example of this implementation, after calculating the quaternion of the target pedestrian's current pose, the updated pose quaternion can be normalized, resulting in:

[0155]

[0156] Therefore, by using the principle of quaternion to Euler angle conversion, the heading of the target pedestrian at the current moment can be obtained based on the quaternion of the current posture. Similarly, the heading of the target pedestrian when each foot is fully on the ground can be obtained.

[0157] In this implementation, the b-frame is updated using the equivalent rotation vector method based on the inertial data to obtain the quaternion of the b-frame attitude transformation. The n-frame is then updated using the equivalent rotation vector method based on the inertial data to obtain the quaternion of the n-frame attitude transformation. Based on the quaternions of the b-frame and n-frame attitude transformations, the quaternion of the current attitude of the target pedestrian is calculated. Finally, based on the quaternion of the current attitude, the heading of the target pedestrian at the current moment is obtained. This effectively avoids the gimbal lock problem, improves numerical stability and computational efficiency, and achieves more accurate, continuous, and stable pedestrian heading estimation, enhancing the robustness and positioning accuracy of the indoor navigation system in real-time response and complex dynamic environments.

[0158] In this embodiment, a first sliding window and a second sliding window can be set, wherein the right boundary of both the first and second sliding windows is the current time, and the size of the second sliding window is larger than the size of the first sliding window. In one example, the length of the first sliding window can be denoted as W, and the length of the second sliding window can be denoted as Q, where W < Q. The first sliding window can be used to initially determine whether the target pedestrian is maintaining a straight-ahead state, and the second sliding window can be used to verify whether the target pedestrian is maintaining a straight-ahead state.

[0159] In one possible implementation, the first preset straight-ahead condition includes: the standard deviation of the heading is less than the standard deviation threshold of the heading.

[0160] In this implementation, in response to the fact that the standard deviation of the target pedestrian's heading within the first sliding window is less than the heading standard deviation threshold, the target pedestrian can be subject to heading constraint processing at the current moment based on the difference information between the heading at the current moment and the heading of the target pedestrian within the first sliding window, and / or the error distribution of the heading of the target pedestrian within the second sliding window.

[0161] In another possible implementation, the first preset straight-ahead condition includes: the variance of the heading is less than the heading variance threshold.

[0162] In this implementation, in response to the target pedestrian's heading variance being less than a heading variance threshold within a first sliding window, the target pedestrian can be subject to heading constraint processing at the current moment based on the difference between the current heading and the target pedestrian's heading within the first sliding window, and / or the error distribution of the target pedestrian's heading within a second sliding window.

[0163] In one possible implementation, the method further includes: in response to the target pedestrian's heading within the first sliding window not meeting the first preset straight-ahead condition, determining that the heading constraint is invalid at the current moment.

[0164] As an example of this implementation, in response to the target pedestrian's heading standard deviation being greater than or equal to the heading standard deviation threshold within the first sliding window, it can be determined that the heading constraint is invalid at the current moment, that is, no heading constraint is performed at the current moment.

[0165] In this embodiment of the disclosure, in response to the target pedestrian's heading within a first sliding window satisfying a first preset straight-ahead condition, the target pedestrian's heading is constrained at the current moment based on the difference between the current heading and the target pedestrian's heading within the first sliding window, and / or the error distribution of the target pedestrian's heading within a second sliding window. Specifically, within the second sliding window, the target pedestrian's heading can be integrated based on the pedestrian's inertial data, and the error distribution of the target pedestrian's heading can be statistically analyzed.

[0166] The Gaussian distribution (i.e., the normal distribution) exhibits a clear central tendency, meaning that data fluctuates around the mean, and the probability of occurrence gradually decreases as the distance from the mean increases. In pedestrian navigation, if the target pedestrian is moving in a straight line, their heading change should be relatively stable, and the error (i.e., the deviation between the actual heading and the expected heading) should randomly fluctuate around zero error, which conforms to the characteristics of the Gaussian distribution.

[0167] In this embodiment of the disclosure, an error distribution can be obtained by statistically analyzing the heading error of the target pedestrian over a period of time. Statistical tests (such as the Shapiro-Wilk test) can be used to evaluate whether this distribution conforms to a Gaussian distribution. If the test results show that the error distribution does not differ significantly from the Gaussian distribution, i.e., the null hypothesis is accepted, then the pedestrian can be considered to be traveling in a straight line. If the heading error distribution does not conform to a Gaussian distribution, it means that the error is not randomly fluctuating, but may have a systematic bias or non-random pattern. This usually indicates that the pedestrian is not traveling in a straight line, but is turning or making other non-linear movements. In this case, the error may systematically increase over time, reflecting a continuous change in the pedestrian's heading rather than random noise.

[0168] In one possible implementation, in response to the target pedestrian's heading within the first sliding window satisfying a first preset straight-ahead condition, a heading constraint can be applied to the target pedestrian at the current moment based on the difference between the current heading and the target pedestrian's heading within the first sliding window. For example, in response to the difference between the current heading and the target pedestrian's heading within the first sliding window not satisfying a second preset straight-ahead condition, it can be determined that the target pedestrian is not in a straight-ahead state, and the heading constraint is deemed invalid at the current moment.

[0169] In another possible implementation, in response to the target pedestrian's heading within the first sliding window satisfying the first preset straight-ahead condition, the target pedestrian's heading can be constrained at the current moment based on the difference between the current heading and the target pedestrian's heading within the first sliding window, as well as the error distribution of the target pedestrian's heading within the second sliding window.

[0170] As an example of this implementation, in response to the difference between the current heading and the heading of the target pedestrian in the first sliding window satisfying the second preset straight-ahead condition, and the error distribution of the heading of the target pedestrian in the second sliding window conforming to a Gaussian distribution, it can be determined that the target pedestrian is in a straight-ahead state, and a heading constraint can be applied to the target pedestrian at the current time.

[0171] As another example of this implementation, in response to the difference between the current heading and the heading of the target pedestrian in the first sliding window satisfying the second preset straight-ahead condition, and the error distribution of the heading of the target pedestrian in the second sliding window not conforming to the Gaussian distribution, it can be determined that the target pedestrian is not in a straight-ahead state, the heading constraint at the current time is determined to be invalid, and the heading constraint already applied in the second sliding window is repaired.

[0172] In another possible implementation, in response to the target pedestrian's heading within the first sliding window satisfying a first preset straight-ahead condition, a heading constraint can be applied to the target pedestrian at the current moment based on the error distribution of the target pedestrian's heading within the second sliding window. For example, in response to the target pedestrian's heading error distribution within the second sliding window not conforming to a Gaussian distribution, it can be determined that the target pedestrian is not in a straight-ahead state, the heading constraint is determined to be invalid at the current moment, and the heading constraints already applied within the second sliding window can be repaired.

[0173] In this embodiment of the disclosure, if the target pedestrian's heading within the first sliding window meets a first preset straight-ahead condition, and the difference between the current heading and the target pedestrian's heading within the first sliding window meets a second preset straight-ahead condition, the error distribution of the target pedestrian's heading within the second sliding window can be compared with a Gaussian distribution to verify whether the target pedestrian is moving straight. If the error distribution of the target pedestrian's heading within the second sliding window conforms to a Gaussian distribution, it usually means that the error is random, and the target pedestrian is likely moving straight; if the error distribution of the target pedestrian's heading within the second sliding window does not conform to a Gaussian distribution, it may mean that the target pedestrian is turning or making other non-linear movements.

[0174] In this embodiment of the disclosure, the characteristics of Gaussian distribution can be used to more accurately identify the straight-ahead state of pedestrians, thereby applying heading constraints in the straight-ahead state, reducing accumulated errors, and improving positioning accuracy.

[0175] In one possible implementation, the heading constraint processing of the target pedestrian at the current moment based on the difference information between the heading at the current moment and the heading of the target pedestrian in the first sliding window, and / or the error distribution of the heading of the target pedestrian in the second sliding window, includes: applying a heading constraint to the target pedestrian at the current moment in response to the difference information between the heading at the current moment and the heading of the target pedestrian in the first sliding window satisfying a second preset straight-ahead condition; and performing fault repair on the heading constraints already applied in the second sliding window in response to the error distribution of the heading of the target pedestrian in the second sliding window not conforming to a Gaussian distribution.

[0176] In this implementation, if the target pedestrian's heading within the first sliding window meets the first preset straight-going condition, and the difference between the current heading and the target pedestrian's heading within the first sliding window meets the second preset straight-going condition, then it can be preliminarily determined that the target pedestrian is in a straight-going state, and a heading constraint can be applied to the target pedestrian at the current moment.

[0177] As an example of this implementation, the difference between the current heading and the heading of the target pedestrian within the first sliding window satisfies a second preset straight-ahead condition, including: the absolute value of the difference between the current heading and the average heading of the target pedestrian within the first sliding window is less than a heading threshold.

[0178] As another example of this implementation, the difference between the current heading and the heading of the target pedestrian within the first sliding window satisfies a second preset straight-ahead condition, including: the absolute value of the difference between the current heading and the weighted average of the headings of the target pedestrian within the first sliding window is less than a heading threshold.

[0179] As another example of this implementation, the difference between the current heading and the heading of the target pedestrian within the first sliding window satisfies a second preset straight-ahead condition, including: the absolute value of the difference between the current heading and the median of the heading of the target pedestrian within the first sliding window is less than a heading threshold.

[0180] As an example of this implementation, applying a heading constraint to the target pedestrian at the current time includes: applying a heading constraint to the target pedestrian through a first Kalman filter at the current time.

[0181] The heading constraint implementation of the first Kalman filter is described below. In this example, the state model of the pedestrian navigation system can be given as follows:

[0182]

[0183] Where X k Let k be the state of the pedestrian at time k. These include three-dimensional attitude and three-dimensional velocity, respectively. H is the model's measurement matrix, and its specific expression is as follows:

[0184]

[0185] In this example, the system can be updated using Kalman filtering when the target pedestrian's foot is fully on the ground, i.e., a heading constraint is applied, where the applied observation is...

[0186] Figure 4 A schematic diagram illustrating the pedestrian navigation heading constraint method provided in an embodiment of this disclosure is shown. Figure 4 As shown, the attitude of a target pedestrian can be updated using strapdown inertial navigation. Based on the pedestrian's inertial data, the moment when the pedestrian's foot fully touches the ground can be extracted, and at that moment, the pedestrian's heading can be extracted based on the inertial data. Thus, the heading at each moment when the pedestrian's foot fully touches the ground can be recorded over a period of time. An appropriate length W of the first sliding window and a heading standard deviation threshold Y can be set. The heading obtained from the attitude quaternion can be judged. If the standard deviation of the pedestrian's heading within the first sliding window is less than the heading standard deviation threshold Y, the pedestrian can be considered to be in a straight-line state within the first sliding window, and the mean of the heading within the first sliding window can be calculated as the heading Y for subsequent constraints. res Otherwise, it can be assumed that the target pedestrian did not move straight within the first sliding window, and the window continues to slide to find the target pedestrian's heading when moving straight. Once the target pedestrian is in a straight-moving state, the heading Y when the next foot fully touches the ground can be obtained using the strapdown inertial navigation attitude update algorithm.now If Y now and Y res The absolute value of the difference | Y now -Y res If the distance is less than the heading threshold T, the target pedestrian can be considered to still be going straight and can enter the first Kalman filter for heading constraint; otherwise, the target pedestrian can be considered to no longer be going straight.

[0187] In this implementation, in response to the fact that the error distribution of the target pedestrian's heading within the second sliding window does not conform to a Gaussian distribution, fault repair can be performed on the heading constraints already applied within the second sliding window.

[0188] Figure 5 This diagram illustrates a scenario involving pedestrians on a long-radius turning section. It takes into account that some buildings have long-radius turning sections, such as... Figure 5 As shown, in this case, due to the small turning angle, the heading constraint algorithm may cause incorrect heading constraints. Therefore, in this implementation, heading constraint verification can be performed based on the heading error distribution within the second sliding window.

[0189] Figure 6 This diagram illustrates the heading constraint verification process in the pedestrian navigation heading constraint method provided in this embodiment of the present disclosure. Figure 6 As shown, a suitable length Q of the second sliding window can be set. In this implementation, the heading of the target pedestrian within the first sliding window satisfies a first preset straight-ahead condition, and the heading Y at the current moment... now The average heading Y of the target pedestrian within the first sliding window res The absolute value of the difference | Y now -Y res When the heading error is less than the heading threshold T, the distribution of heading error within the second sliding window can be statistically analyzed and a Shapiro-Wilk test can be performed. The Shapiro-Wilk test is a method used to test whether random sample data conforms to a Gaussian distribution. In this implementation, x1 can be set... <x2<x3<…<x n Let n be a set of sequentially arranged samples. The null hypothesis H0 indicates that the sample data is not significantly different from the Gaussian distribution (i.e., the sample data conforms to the Gaussian distribution), and the alternative hypothesis H1 indicates that the sample data is significantly different from the Gaussian distribution (i.e., the sample data does not conform to the Gaussian distribution).

[0190] The statistic W used in the Shapiro-Wilk test can be defined as follows:

[0191]

[0192] Where x (i)Let represent the i-th order statistic, which is the i-th smallest number in the sample; This represents the average value of the sample.

[0193] constants (a1,…,a) n It can be obtained through the following formula:

[0194]

[0195] Where m = (m1, ..., m) n ) T ,m1,…,m n V is the expected value of ordered, independent, and identically distributed statistics sampled from a standard Gaussian distributed random variable, while V is the covariance of these ordered statistics.

[0196] The more similar the distribution of the sample data is to the Gaussian distribution, the more the statistic W will tend to a constant 1; otherwise, it will tend to a constant 0. Next, the p-value obtained from the test can be compared with the significance level α. If the p-value is less than the significance level α, H0 can be rejected, indicating a significant difference between the sample data and the Gaussian distribution, and the heading constraint is invalid. Otherwise, H0 is accepted, indicating that the sample data conforms to a Gaussian distribution, and the heading constraint is successful (i.e., the heading constraint is valid).

[0197] By verifying the heading constraint, long-radius turning sections can be accurately identified, effectively preventing erroneous constraints in the heading constraint algorithm. It has advantages such as high robustness and simple design.

[0198] As an example of this implementation, the fault repair of the heading constraints already applied within the second sliding window includes: performing fault repair on the heading constraints already applied within the second sliding window through a second Kalman filter.

[0199] Figure 7This diagram illustrates a second sliding window for heading constraint verification in the pedestrian navigation heading constraint method provided in this embodiment. Since heading constraint verification considers a longer time period, the second sliding window can be relatively large, exhibiting a lag during detection. In the heading constraint algorithm, if the sample data within the second sliding window passes the Shapiro-Wilk test (i.e., satisfies a Gaussian distribution), the time of the last data point within that second sliding window and the heading constraint at that time can be recorded. If, in the next sliding window, the sample data of the second sliding window is deemed not to satisfy a Gaussian distribution, the incorrect heading constraint between times t-1 can be corrected based on the recorded time of the last data point in the previous window (time t-1) and the heading constraint. In this example, the system state can be rolled back to time t-1 using a second Kalman filter, and subsequently updated using strapdown inertial navigation until the system state again meets the conditions of the heading constraint algorithm and the heading constraint verification algorithm.

[0200] As an example of this implementation, the heading constraint processing of the target pedestrian at the current moment based on the difference information between the heading at the current moment and the heading of the target pedestrian in the first sliding window, and / or the error distribution of the heading of the target pedestrian in the second sliding window, further includes: in response to the fact that the difference information between the heading at the current moment and the heading of the target pedestrian in the first sliding window does not meet the second preset straight-ahead condition, determining that the heading constraint is invalid at the current moment.

[0201] In this example, if the difference between the current heading and the heading of the target pedestrian within the first sliding window does not meet the second preset straight-ahead condition, it can be determined that the target pedestrian is not in a straight-ahead state, and the heading constraint is determined to be invalid at the current time.

[0202] In this embodiment of the disclosure, if the target pedestrian is not in a straight-line state, for example, if the pedestrian is turning or moving on a non-straight path, it can be determined that the heading constraint is invalid, that is, the heading constraint is not performed. This allows for flexible response to changes in the target pedestrian's movement state and maintains high navigation accuracy.

[0203] The pedestrian navigation heading constraint method provided in this disclosure is illustrated below through two specific application scenarios.

[0204] Application Scenario 1:

[0205] In this application scenario, an inertial measurement unit installed on the foot of the target pedestrian can be used to obtain the pedestrian's inertial data. Based on this inertial data, the moment when the pedestrian's foot fully touches the ground can be extracted. Furthermore, based on the inertial data at that moment, the pedestrian's current heading can be obtained.

[0206] Next, we can perform a three-level judgment:

[0207] First-level judgment: Determine whether the standard deviation of the target pedestrian's heading within the first sliding window is less than the heading standard deviation threshold. If so, proceed to the second-level judgment; otherwise, it can be determined that the target pedestrian is not in a straight-ahead state, i.e., the heading constraint is invalid at the current moment.

[0208] The second level of judgment determines whether the absolute value of the difference between the current heading and the average heading of the target pedestrian within the first sliding window is less than the heading threshold. If so, it can be preliminarily determined that the target pedestrian is in a straight-ahead state, and a heading constraint is applied to the target pedestrian at the current moment, and the process can proceed to the third level of judgment; otherwise, it can be determined that the target pedestrian is not in a straight-ahead state, that is, the heading constraint is invalid at the current moment.

[0209] The third level of judgment: Determine whether the error distribution of the target pedestrian's heading within the second sliding window conforms to a Gaussian distribution, where the length of the second sliding window is greater than the length of the first sliding window. If so, no fault repair is required; otherwise, fault repair can be performed on the heading constraints already applied within the second sliding window.

[0210] Application Scenario 2:

[0211] In this application scenario, an inertial measurement unit installed on the foot of the target pedestrian can be used to obtain the pedestrian's inertial data. Based on this inertial data, the moment when the pedestrian's foot fully touches the ground can be extracted. Furthermore, based on the inertial data at that moment, the pedestrian's current heading can be obtained.

[0212] Next, we can perform a second-level judgment:

[0213] First-level judgment: Determine whether the standard deviation of the target pedestrian's heading within the first sliding window is less than the heading standard deviation threshold. If so, proceed to the second-level judgment; otherwise, it can be determined that the target pedestrian is not in a straight-ahead state, i.e., the heading constraint is invalid at the current moment.

[0214] The second level of judgment involves determining whether the absolute value of the difference between the current heading and the average heading of the target pedestrian within the first sliding window is less than a heading threshold, and whether the error distribution of the target pedestrian's heading within the second sliding window conforms to a Gaussian distribution. The length of the second sliding window is greater than the length of the first sliding window. If the absolute value of the difference between the current heading and the average heading of the target pedestrian within the first sliding window is less than the heading threshold, and the error distribution of the target pedestrian's heading within the second sliding window conforms to a Gaussian distribution, then the target pedestrian can be determined to be in a straight-ahead state, and a heading constraint can be applied to the target pedestrian at the current moment. If the absolute value of the difference between the current heading and the average heading of the target pedestrian within the first sliding window is less than the heading threshold, and the error distribution of the target pedestrian's heading within the second sliding window does not conform to a Gaussian distribution, then the target pedestrian can be determined to be not in a straight-ahead state, and the heading constraint already applied within the second sliding window can be repaired.

[0215] Because the errors of inertial navigation systems accumulate over time, the attitude, velocity, and position errors calculated by the entire system gradually diverge. The embodiments disclosed herein can constrain the heading without prior information and simultaneously verify the heading constraints, effectively suppressing the divergence of the navigation system's heading error and significantly improving the positioning accuracy of the inertial pedestrian navigation system. This approach offers advantages such as good stability, low cost, and ease of implementation.

[0216] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0217] In addition, this disclosure also provides pedestrian navigation heading constraint devices, electronic devices, computer-readable storage media, and computer program products, all of which can be used to implement any of the pedestrian navigation heading constraint methods provided in this disclosure. The corresponding technical solutions and effects can be found in the relevant descriptions in the method section, and will not be repeated here.

[0218] Figure 8 A block diagram of a pedestrian navigation heading constraint device provided in an embodiment of this disclosure is shown. Figure 8 As shown, the pedestrian navigation heading constraint device includes:

[0219] Module 81 is used to obtain the inertial data of the target pedestrian;

[0220] Extraction module 82 is used to extract the heading of the target pedestrian at the current time based on the inertial data;

[0221] The heading constraint processing module 83 is used to perform heading constraint processing on the target pedestrian at the current moment in response to the target pedestrian's heading in the first sliding window satisfying the first preset straight-line condition, based on the difference information between the heading at the current moment and the target pedestrian's heading in the first sliding window, and / or the error distribution of the target pedestrian's heading in the second sliding window, wherein the right boundary of the first sliding window and the second sliding window is the current moment, and the length of the second sliding window is greater than the length of the first sliding window.

[0222] In one possible implementation, the obtaining module 81 is used for:

[0223] The inertial data of the target pedestrian is obtained by using an inertial measurement unit installed on the foot of the target pedestrian.

[0224] In one possible implementation, the extraction module 82 is used for:

[0225] Based on the inertial data, the moment when the target pedestrian's foot fully touches the ground is extracted;

[0226] At the moment when the foot is fully on the ground, the heading of the target pedestrian is extracted based on the inertial data to obtain the heading of the target pedestrian at the current moment.

[0227] In one possible implementation, the extraction module 82 is used for:

[0228] For any given moment, extract the amplitude of the inertial data at that moment, and calculate the standard deviation of the inertial data within the third sliding window, wherein the right boundary of the third sliding window is the given moment;

[0229] In response to the amplitude being less than an amplitude threshold and the standard deviation being less than a standard deviation threshold, the moment is determined to be the moment when the target pedestrian's foot is fully on the ground.

[0230] In one possible implementation,

[0231] The amplitude of the inertial data includes: specific force amplitude and angular velocity amplitude;

[0232] The standard deviation of the inertial data includes: the standard deviation of specific force and the standard deviation of angular velocity;

[0233] The amplitude thresholds include: specific force amplitude threshold and angular velocity amplitude threshold;

[0234] The standard deviation thresholds include: specific force standard deviation threshold and angular velocity standard deviation threshold;

[0235] The extraction module 82 is configured to: in response to the specific force amplitude being less than the specific force amplitude threshold, the angular velocity amplitude being less than the angular velocity amplitude threshold, the specific force standard deviation being less than the specific force standard deviation threshold, and the angular velocity standard deviation being less than the angular velocity standard deviation threshold, determine that the moment is the moment when the target pedestrian's foot is completely on the ground.

[0236] In one possible implementation, the extraction module 82 is used for:

[0237] Based on the inertial data, the b-frame is updated using the equivalent rotation vector method to obtain the quaternion of the b-frame attitude transformation;

[0238] Based on the inertial data, the n-system is updated using the equivalent rotation vector method to obtain the quaternion of the n-system attitude transformation;

[0239] The quaternion of the current posture of the target pedestrian is calculated based on the quaternion of the b-series posture transformation and the quaternion of the n-series posture transformation.

[0240] Based on the quaternion of the current posture, the heading of the target pedestrian at the current moment is obtained.

[0241] In one possible implementation, the first preset straight-ahead condition includes:

[0242] The standard deviation of the heading is less than the standard deviation threshold of the heading.

[0243] In one possible implementation, the device further includes:

[0244] The failure determination module is used to determine that the heading constraint is invalid at the current moment in response to the target pedestrian's heading within the first sliding window not meeting the first preset straight-ahead condition.

[0245] In one possible implementation, the heading constraint processing module 83 is used for:

[0246] In response to the fact that the difference between the heading at the current moment and the heading of the target pedestrian within the first sliding window satisfies the second preset straight-ahead condition, a heading constraint is imposed on the target pedestrian at the current moment;

[0247] In response to the fact that the error distribution of the target pedestrian's heading within the second sliding window does not conform to a Gaussian distribution, fault repair is performed on the heading constraints already applied within the second sliding window.

[0248] In one possible implementation, the difference between the current heading and the heading of the target pedestrian within the first sliding window satisfies a second preset straight-ahead condition, including:

[0249] The absolute value of the difference between the current heading and the average heading of the target pedestrian within the first sliding window is less than the heading threshold.

[0250] In one possible implementation, the heading constraint processing module 83 is used for:

[0251] At the current moment, a heading constraint is applied to the target pedestrian using a first Kalman filter.

[0252] In one possible implementation, the heading constraint processing module 83 is used for:

[0253] The second Kalman filter is used to repair the heading constraints that have been applied within the second sliding window.

[0254] In one possible implementation, the heading constraint processing module 83 is used for:

[0255] If the difference between the heading at the current moment and the heading of the target pedestrian within the first sliding window does not meet the second preset straight-ahead condition, it is determined that the heading constraint is invalid at the current moment.

[0256] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation and technical effects can be referred to the description of the above method embodiments. For the sake of brevity, they will not be repeated here.

[0257] This disclosure also provides a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the above-described method. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium.

[0258] This disclosure also proposes a computer program including computer-readable code, wherein when the computer-readable code is run in an electronic device, a processor in the electronic device executes the above-described method.

[0259] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in an electronic device, the processor in the electronic device executes the above-described method.

[0260] This disclosure also provides an electronic device, including: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the executable instructions stored in the memory to perform the above-described method.

[0261] Electronic devices can be provided as terminals, servers, or other forms of devices.

[0262] Figure 9 A block diagram of an electronic device 1900 provided in an embodiment of this disclosure is shown. For example, the electronic device 1900 may be provided as a server or a terminal. (Refer to...) Figure 9 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0263] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). Electronic device 1900 can operate on an operating system stored in memory 1932, such as Microsoft Server operating system (Windows Server). TM Apple's graphical user interface-based operating system (MacOS X) TM ), a multi-user, multi-process computer operating system (Unix) TM Linux is a free and open-source Unix-like operating system. TM ), the open-source Unix-like operating system (FreeBSD) TM (or similar.)

[0264] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0265] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0266] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0267] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0268] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0269] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0270] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0271] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0272] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0273] The computer program product can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0274] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0275] If the technical solution of this disclosure involves personal information, the product applying the technical solution of this disclosure has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this disclosure involves sensitive personal information, the product applying the technical solution of this disclosure has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to indicate that the user has entered the scope of personal information collection and that personal information will be collected. If the user voluntarily enters the collection scope, it is deemed to have consented to the collection of their personal information; or on the personal information processing device, with clear signs / information informing the user of the personal information processing rules, authorization is obtained from the user through pop-up information or by asking the user to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0276] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for constraining the heading of pedestrian navigation, characterized in that, include: Obtain the inertial data of the target pedestrian; Based on the inertial data, the heading of the target pedestrian at the current moment is extracted; In response to the target pedestrian's heading within the first sliding window satisfying a first preset straight-ahead condition, based on the difference between the current heading and the target pedestrian's heading within the first sliding window, and the error distribution of the target pedestrian's heading within the second sliding window, a heading constraint is applied to the target pedestrian at the current moment. Both the first and second sliding windows include the current moment and a period prior to the current moment, and the length of the second sliding window is greater than the length of the first sliding window. The heading within the first sliding window includes headings at different times within the first sliding window, and the heading within the second sliding window includes headings at different times within the second sliding window. The first preset straight-ahead condition includes: the standard deviation of the heading is less than a heading standard deviation threshold. The method further includes: integrating the heading of the target pedestrian based on the inertial data of the target pedestrian within the second sliding window, and statistically analyzing the error distribution of the heading of the target pedestrian to obtain the error distribution of the heading of the target pedestrian within the second sliding window; The average heading within the first sliding window is determined, and the absolute value of the difference between the current heading and the average heading is determined as the difference information between the current heading and the heading of the target pedestrian within the first sliding window. The method of performing heading constraint processing on the target pedestrian at the current moment based on the difference information between the heading at the current moment and the heading of the target pedestrian in the first sliding window, and the error distribution of the heading of the target pedestrian in the second sliding window, includes: In response to the fact that the difference between the heading at the current moment and the heading of the target pedestrian within the first sliding window satisfies the second preset straight-ahead condition, a heading constraint is imposed on the target pedestrian at the current moment; In response to the fact that the error distribution of the target pedestrian's heading within the second sliding window does not conform to a Gaussian distribution, fault repair is performed on the heading constraints already applied within the second sliding window.

2. The method according to claim 1, characterized in that, The acquisition of the inertial data of the target pedestrian includes: The inertial data of the target pedestrian is obtained by using an inertial measurement unit installed on the foot of the target pedestrian.

3. The method according to claim 1, characterized in that, The step of extracting the target pedestrian's heading at the current moment based on the inertial data includes: Based on the inertial data, the moment when the target pedestrian's foot fully touches the ground is extracted; At the moment when the foot is fully on the ground, the heading of the target pedestrian is extracted based on the inertial data to obtain the heading of the target pedestrian at the current moment.

4. The method according to claim 3, characterized in that, The step of extracting the moment when the target pedestrian's foot fully touches the ground based on the inertial data includes: For any given moment, extract the amplitude of the inertial data at that moment, and calculate the standard deviation of the inertial data within the third sliding window, wherein the right boundary of the third sliding window is the given moment; In response to the amplitude being less than an amplitude threshold and the standard deviation being less than a standard deviation threshold, the moment is determined to be the moment when the target pedestrian's foot is fully on the ground.

5. The method according to claim 4, characterized in that, The amplitude of the inertial data includes: specific force amplitude and angular velocity amplitude; The standard deviation of the inertial data includes: the standard deviation of specific force and the standard deviation of angular velocity; The amplitude thresholds include: specific force amplitude threshold and angular velocity amplitude threshold; The standard deviation thresholds include: specific force standard deviation threshold and angular velocity standard deviation threshold; The step of determining the moment when the target pedestrian's foot is fully on the ground in response to the amplitude being less than the amplitude threshold and the standard deviation being less than the standard deviation threshold includes: determining the moment when the target pedestrian's foot is fully on the ground in response to the specific force amplitude being less than the specific force amplitude threshold, the angular velocity amplitude being less than the angular velocity amplitude threshold, the specific force standard deviation being less than the specific force standard deviation threshold, and the angular velocity standard deviation being less than the angular velocity standard deviation threshold.

6. The method according to claim 1, characterized in that, The step of extracting the target pedestrian's heading at the current moment based on the inertial data includes: Based on the inertial data, the b-frame is updated using the equivalent rotation vector method to obtain the quaternion of the b-frame attitude transformation; Based on the inertial data, the n-system is updated using the equivalent rotation vector method to obtain the quaternion of the n-system attitude transformation; The quaternion of the current posture of the target pedestrian is calculated based on the quaternion of the b-series posture transformation and the quaternion of the n-series posture transformation. Based on the quaternion of the current posture, the heading of the target pedestrian at the current moment is obtained.

7. The method according to claim 1, characterized in that, The method further includes: If the heading of the target pedestrian within the first sliding window does not meet the first preset straight-ahead condition, it is determined that the heading constraint is invalid at the current moment.

8. The method according to claim 1, characterized in that, The difference between the current heading and the heading of the target pedestrian within the first sliding window satisfies the second preset straight-ahead condition, including: The absolute value of the difference between the current heading and the average heading of the target pedestrian within the first sliding window is less than the heading threshold.

9. The method according to claim 1, characterized in that, The step of applying a heading constraint to the target pedestrian at the current time includes: At the current moment, a heading constraint is applied to the target pedestrian using a first Kalman filter.

10. The method according to claim 1, characterized in that, The fault repair of the heading constraints already applied within the second sliding window includes: The second Kalman filter is used to repair the heading constraints that have been applied within the second sliding window.

11. The method according to claim 1, characterized in that, The method of constraining the heading of the target pedestrian at the current moment based on the difference information between the heading at the current moment and the heading of the target pedestrian in the first sliding window and the error distribution of the heading of the target pedestrian in the second sliding window further includes: If the difference between the heading at the current moment and the heading of the target pedestrian within the first sliding window does not meet the second preset straight-ahead condition, it is determined that the heading constraint is invalid at the current moment.

12. A pedestrian navigation heading constraint device, characterized in that, include: The acquisition module is used to obtain the inertial data of the target pedestrian; An extraction module is used to extract the heading of the target pedestrian at the current moment based on the inertial data; A heading constraint processing module is used to respond to the target pedestrian's heading within a first sliding window satisfying a first preset straight-ahead condition. Based on the difference between the current heading and the target pedestrian's heading within the first sliding window, and the error distribution of the target pedestrian's heading within a second sliding window, the module performs heading constraint processing on the target pedestrian at the current moment. Both the first and second sliding windows include the current moment and a period prior to the current moment, and the length of the second sliding window is greater than the length of the first sliding window. The heading within the first sliding window includes headings at different times within the first sliding window, and the heading within the second sliding window includes headings at different times within the second sliding window. The first preset straight-ahead condition includes: the standard deviation of the heading is less than a heading standard deviation threshold. The heading constraint processing module is further configured to: integrate the heading of the target pedestrian based on the inertial data of the target pedestrian within the second sliding window, and statistically analyze the error distribution of the heading of the target pedestrian to obtain the error distribution of the heading of the target pedestrian within the second sliding window; determine the mean heading within the first sliding window, and determine the absolute value of the difference between the heading at the current moment and the mean heading as the difference information between the heading at the current moment and the heading of the target pedestrian within the first sliding window; The method of performing heading constraint processing on the target pedestrian at the current moment based on the difference information between the heading at the current moment and the heading of the target pedestrian in the first sliding window and the error distribution of the heading of the target pedestrian in the second sliding window includes: applying a heading constraint to the target pedestrian at the current moment in response to the difference information between the heading at the current moment and the heading of the target pedestrian in the first sliding window satisfying a second preset straight-ahead condition; and performing fault repair on the heading constraint already applied in the second sliding window in response to the error distribution of the heading of the target pedestrian in the second sliding window not conforming to a Gaussian distribution.

13. An electronic device, characterized in that, include: One or more processors; Memory used to store executable instructions; The one or more processors are configured to invoke executable instructions stored in the memory to perform the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 11.

15. A computer program product comprising computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, characterized in that, When the computer-readable code is run in an electronic device, the processor in the electronic device performs the method according to any one of claims 1 to 11.

Images