Indoor Navigation

Abstract

According to one embodiment of the present specification, a new method for determining the movement direction of a user in indoor navigation is proposed. Generally speaking, the device direction of the terminal device is obtained based on at least one signal stream collected from the terminal device carried by the mobile user (1110); the deviation degree is determined based on at least one signal stream, and the deviation degree represents the deviation between the movement direction of the user and the actual device direction of the terminal device (1120); according to the determination result that the deviation degree is lower than the threshold degree, the movement direction is determined based on the device direction (1130). Through the above embodiment, the movement direction of the user is determined in a more effective and accurate manner, thereby improving the accuracy of indoor navigation.

Description

Technical Field

[0001] Embodiments of the present specification relate to a navigation application technology, and more specifically, to a method, device, computer program product, and computer-readable storage medium for indoor navigation. Background Art

[0002] In recent years, navigation applications have become increasingly popular. A navigation application can be a computer software program installed on a terminal device equipped with a Global Positioning System (GPS) sensor. Based on the signals received from GPS satellites and a map, the navigation application can provide the location of the terminal device. However, when a user of the navigation application enters a building, the signals from GPS satellites may be attenuated and scattered by the building's roof, walls, and other objects. Therefore, the navigation application cannot provide accurate indoor navigation.

[0003] Special sensors have been developed to measure the acceleration and direction of a moving user. For example, terminal devices such as mobile phones are equipped with these sensors, so the acceleration and direction signals of the user can be detected. However, it is difficult to process these signals and obtain the accurate movement of the user in an effective and convenient manner. Summary of the Invention

[0004] According to an embodiment of the present specification, a new method for determining the movement direction of a user in indoor navigation is proposed. Generally, the device direction of a terminal device is obtained based on at least one signal stream collected from the terminal device carried by a moving user. A deviation degree is determined based on the at least one signal stream, and the deviation degree represents the deviation between the movement direction of the user and the actual device direction of the terminal device. Based on the determination result that the deviation degree is lower than a threshold degree, the movement direction is determined based on the device direction. Through the above embodiments, the movement direction of the user is determined in a more effective and accurate manner, thereby improving the accuracy of indoor navigation.

[0005] It can be understood that the purpose of the summary of the invention is not to determine the key or basic features of the embodiments of the present specification, nor is it intended to limit the scope of the present specification. Through the following description, other features of the present specification can be easily understood. Brief Description of the Drawings

[0006] Details of one or more embodiments of the present specification are illustrated in the drawings and the following description. Other functions, aspects, and advantages disclosed in the specification, drawings, and claims are obvious, where:

[0007] Figure 1 A block diagram showing the environment for implementing the embodiments of the present specification is shown;

[0008] Figure 2 A flowchart showing a method for implementing the identification of a step event according to an embodiment of the present specification is shown;

[0009] Figure 3 Shows a process block diagram for processing an acceleration signal stream based on two time windows according to an embodiment of the present specification;

[0010] Figure 4 Shows a process block diagram for determining a deviation according to an embodiment of the present specification;

[0011] Figure 5 Shows a process block diagram for determining a step intensity related to a first time window and a second time window according to an embodiment of the present specification;

[0012] Figure 6 Shows a block diagram for determining a step intensity curve based on an acceleration signal stream according to an embodiment of the present specification;

[0013] Figure 7 Shows a process block diagram for determining an acceleration magnitude according to an embodiment of the present specification;

[0014] Figure 8 Shows a process block diagram for determining a step length according to an embodiment of the present specification;

[0015] Figure 9 Shows a block diagram of an environment for implementing an embodiment of the present specification;

[0016] Figure 10 Shows a process block diagram for determining a user's movement direction according to an embodiment of the present specification;

[0017] Figure 11 Shows a flowchart of a method for determining a user's movement direction according to an embodiment of the present specification;

[0018] Figure 12 Shows a process block diagram for determining a user's movement direction according to an embodiment of the present specification;

[0019] Figure 13 Shows a block diagram of a step window related to a user's step event according to an embodiment of the present specification;

[0020] Figure 14 Shows a block diagram for determining a user's trajectory according to an embodiment of the present specification; and

[0021] Figure 15 Shows a block diagram of a device for implementing one or more embodiments of the present specification.

[0022] In the respective drawings, the same or corresponding reference numerals always denote the same or corresponding elements. Detailed Description

[0023] The principles of this specification will be described below with reference to certain embodiments. It should be understood that these embodiments are for illustrative purposes only, to assist those skilled in the art in understanding and implementing this specification, and are not intended to limit the scope of protection of this specification. This specification can be implemented through the embodiments described below and various other forms.

[0024] As described herein, the term "including" and its like shall be understood as open inclusion, i.e., "including but not limited to". The term "based on" shall be understood as "at least partially based on". Unless otherwise specified, the word "a" shall be understood as "one or more". The term "an embodiment" or "the embodiment" shall be understood as "at least one embodiment". The term "another embodiment" shall mean "at least one other embodiment". In addition, it should be understood that in the description of this specification, the terms "first", "second", etc. are used to indicate individual elements or components and are not intended to limit the order of these elements. In addition, the first element may be different from the second element. There may also be other explicit and implicit definitions hereinafter.

[0025] Navigation applications are widely installed in terminal devices. The GPS sensor in the terminal device can receive signals from GPS satellites and obtain the location of the terminal device. However, when the user of the terminal device enters a building, the signal may be weakened and become useless. Currently, inertial measurement unit (IMU) sensors have been applied in terminal devices and can process the signal stream collected from the IMU sensors to determine the movement of the mobile user carrying the terminal device.

[0026] For the working environment of this specification, reference will be made to Figure 1 . Figure 1 Block diagram 100 shows the environment for implementing the embodiments of this specification. In Figure 1 , a navigator 110 can be installed in the terminal device, and a sensor 130 can be equipped in the user's terminal device. Among them, the sensor 130 can include an accelerometer 132 for obtaining an acceleration signal stream and a gyroscope 134 for obtaining a direction signal stream. Then, the signal stream can be processed to determine a motion signal 120, and the motion signal 120 includes the user's speed 122 and direction 124. In addition, the motion signal 120 can be provided to the navigator 110 for indoor navigation.

[0027] A pedestrian dead reckoning (PDR) method is proposed for indoor navigation. The PDR method involves three main steps: 1) determining the user's stepping event, 2) determining the user's step length; 3) determining the user's direction. Among them, the stepping event refers to the event that the user extends his / her leg and takes a step. Since both determining the step length and direction are based on the determined stepping event, the identification of the stepping event becomes the basis of indoor navigation. Peak detection and zero-crossing detection algorithms are developed to identify the stepping event from the signal stream. However, due to the noise and bias of sensor 130, the signal stream should be filtered first. In addition, it is difficult to create criteria to detect peaks and zeros in the filtered signal stream.

[0028] To at least partially solve the above problems and other potential problems, this paper provides a new method and device to identify the stepping event. According to an embodiment of this specification, two windows with different widths are used to process the signal stream collected from sensor 130. Since the two windows have different widths, signal segments are respectively displayed in the two windows. Then, the stepping event can be identified from the signal stream by comparing the signal segments. Through the above embodiment, there is no need to filter the noise and bias in the signal stream and identify peaks and zeros, so that the stepping event can be determined more effectively and conveniently.

[0029] Refer to Figure 2 for a detailed description of the embodiments of this specification. Figure 2 A flowchart of method 200 for identifying a stepping event according to an embodiment of this specification is shown. At block 210, a first signal segment and a second signal segment are respectively obtained from the acceleration signal stream within the first time window and the second time window. Among them, the acceleration signal stream is obtained from an acceleration sensor associated with the mobile user, and the first time window is shorter than the second time window. For detailed information on obtaining the first signal segment and the second signal segment, see Figure 3 .

[0030] Figure 3 A block diagram 300 of the process of processing the acceleration signal stream based on two time windows according to an embodiment of this specification is shown. In Figure 3 , an acceleration signal stream 330 is shown, where the horizontal axis represents the frames in the acceleration signal stream 330, and the vertical axis represents the amplitude of the acceleration. The acceleration signal stream 330 can be collected from an acceleration sensor (such as the accelerometer 132 equipped in the user's terminal device). When the user walks, the acceleration sensor may continuously collect acceleration and output the acceleration signal stream 330. The acceleration sensor can collect signals at a predefined frequency, and the frequency can be adjusted based on the sensor type. In one example, the frequency may be 50Hz, which means the sensor collects 50 frames per second. In another example, the frequency may be another value.

[0031] A first time window 310 (illustrated by the shaded block) and a second time window 320 (illustrated by the blank block) are defined, and the first time window 310 is shorter than the second time window 320. A first signal segment 312 and a second signal segment 322 are respectively obtained from an acceleration signal stream 330 within the first time window 310 and the second time window 320. Among them, the first signal segment 312 is a part of the acceleration signal stream 330 in the first time window 310, and the second signal segment 322 is a part of the acceleration signal stream 330 in the second time window 320.

[0032] During user movement, the acceleration amplitudes of different frames in the acceleration signal stream 330 may vary. Generally, the variations in the short-term window are more drastic than those in the long-term window. Therefore, the first signal segment 312 and the second signal segment 322 may involve different amplitude variations, which can help determine the stepping event.

[0033] In one embodiment of the present specification, the first time window 310 may be within the second time window 320. Although Figure 3 in the illustrated example, the first time window 310 is located in the middle of the second time window 320, the first time window 310 may be located at any position. In another embodiment, the first time window 310 may be at the start or end of the second time window 320, or even outside the second time window 320. For example, the first time window 310 and the second time window 320 may start or end simultaneously. In one embodiment, the first time window 310 and the second time window 320 may have lengths of 20 frames and 100 frames respectively. In another example, the lengths of the first time window 310 and the second time window 320 may be set to other values and may be represented in another format (such as 0.2 seconds and 1 second).

[0034] It can be understood that Figure 3 only the situation related to a certain time point during user movement is illustrated. When the user walks, the first time window 310 and the second time window 320 can move forward along the acceleration signal stream 330, and more signal segments can be obtained from each movement of the two time windows. Since the first time window 310 is included in the second time window 320, the second signal segment 322 can include more acceleration information near the first time window 310. Therefore, comparing the two signal segments can help more effectively identify the amplitude variations caused by the stepping event.

[0035] After describing the details regarding the acquisition of the first signal segment 312 and the second signal segment 322, return to Figure 2 where the first signal segment 312 and the second signal segment 322 are processed. In Figure 2At the box 220 in, the first time window 310 and the second time window 320 are respectively determined based on the first signal segment 312 and the second signal segment 322. Among them, the first amplitude feature represents the amplitude level in the first signal segment 312. For more information, see Figure 4 Obtained.

[0036] Figure 4 The block diagram 400 showing the process of determining the deviation according to the embodiments of the present specification is shown. The first amplitude feature 412 can be determined based on the average value of the first signal segment 312. Assume that the first signal segment 312 includes n1 frames, and the amplitude at the i th frame is represented as Then the first amplitude feature 412 can be represented based on Formula 1 shown as follows.

[0037]

[0038] Among them, represents the first amplitude feature 412 of the first time window 310, n1 represents the number of frames included in the first time window 310, represents the amplitude at the i th frame in the first signal segment 312.

[0039] The second amplitude feature 422 can be determined in a similar manner based on Formula 2 shown as follows.

[0040]

[0041] Among them, represents the second amplitude feature 422 of the second time window 320, n2 represents the number of frames included in the second window 320, represents the amplitude at the i th frame in the second signal segment 322.

[0042] It can be understood that the above Formulas 1 and 2 are only examples for determining the first amplitude feature 412 and the second amplitude feature 422. Additionally and / or in addition to this, the first amplitude feature 412 and the second amplitude feature 422 can be determined based on the average value of the amplitude square, or based on another formula that can reflect the corresponding amplitude within each time window.

[0043] For ease of description, the first amplitude feature 412 and the second amplitude feature 422 can be determined by identifying the end frame. Assume that in the acceleration signal stream 330, both the first time window 310 and the second time window 320 end at the j th frame. Both the first time window 310 and the second time window 320 can be called the j thThe first time window and the second time window of the frame, as well as the first amplitude feature 412 and the second amplitude feature 422 can be referred to as j th The first amplitude feature and the second amplitude feature of the frame. In addition, the first amplitude feature 412 and the second amplitude feature 422 can be compared, so that the deviation 430 between the two can be determined.

[0044] Referring again to Figure 2 , at block 230, a stepping event of the user is determined based on the deviation 430 between the first amplitude feature 412 and the second amplitude feature 422. In one embodiment of the present specification, the deviation is also related to the frame and can be determined according to the difference between the first amplitude feature 412 and the second amplitude feature 422 of Equation 3j th The deviation at the frame.

[0045]

[0046] where dev j represents the deviation 430 of j th frame in the acceleration signal stream 330, and represent the first amplitude feature 412 and the second amplitude feature 422 of j th frame respectively. Continuing with the above example, when the first time window 310 and the second time window 320 have the same end frame, j th frame can be the end frame.

[0047] It can be understood that the above Equation 3 is only an example for determining the deviation 430. Additionally and / or alternatively, the deviation 430 can be determined according to any one of the following Equations 4 and 5.

[0048]

[0049]

[0050] In Equations 4 and 5, the meanings of the symbols are the same as those in Equation 3, and the specific content is omitted.

[0051] Since the change in the first time window 310 may be more drastic than that in the second time window 320, the stepping event can be easily identified based on the deviation 430. In one embodiment of the present specification, if the deviation 430 exceeds the threshold deviation, a stepping event can be determined at j th frame, and Equation 6 can be used to identify the stepping event at j th frame in the acceleration signal stream 330. Among them, the threshold deviation can be determined according to experience.

[0052]

[0053] Among them, indicates whether a step event is determined at the j th frame of the acceleration signal stream 330 based on the deviation 430, dev j represents the deviation 430 at the j th frame of the acceleration signal stream 330, TH dev represents the threshold deviation.

[0054] Since the deviation 430 exists between the first amplitude feature 412 and the second amplitude feature 422 in the same acceleration signal stream affected by the same noise and deviation, the noise and deviation have little impact on the deviation 430, thus more reliably identifying step events. Compared with the existing PDR methods based on peak detection and zero crossing detection, this specification can reduce the influence of noise and deviation, thereby improving the accuracy and operability of identifying step events.

[0055] It can be understood that the above formula 6 is only an example of directly identifying step events based on the deviation 430 related to a single time point when the user is walking. In other examples of this specification, step events can be further determined based on deviations determined in other ways. In one embodiment of this specification, the first time window 310 and the second time window 320 can move forward along the acceleration signal stream 330 respectively. Then, the step intensity can be determined based on the accumulation of multiple deviations at multiple time points when the user is walking. For more details, please refer to Figure 5 .

[0056] Figure 5 FIG. 500 shows a block diagram of a process for determining the step intensity related to the first time window and the second time window according to an embodiment of this specification. For simplicity, Figure 5 only the movement of the first time window 310 is illustrated, and the movement of the second time window 320 is similar and thus omitted. As time passes while the user is walking, more acceleration signal streams will be obtained, and the first time window 310 may move forward from the first time point 520 to the second time point 530, as shown by the arrow 510. Among them, the first time window 310 may move forward one or more frames at a time in the acceleration signal stream 330.

[0057] Assume that the first time window 310 and the second time window 320 have the same end frame, then the end frame may cover a set of frames during the movement. Therefore, the step intensity for a set of frames can be determined. Specifically, a first set of signal segments can be obtained within the first time window 310 during the movement, and similarly, a second set of signal segments can be obtained within the second time window 320. In addition, the deviations related to this set of frames can be summed to determine the step intensity.

[0058] In one embodiment of the present specification, the first set of amplitude features and the second set of amplitude features can be determined based on the first set of signal segments and the second set of signal segments, respectively. Since the first time window 310 and the second time window 320 move together, the deviation of each frame during the movement can be determined. Assuming that the movement covers a set of m frames, the deviation of k th frames in this set of frames can be determined. At this point, the sum of a set of deviations between the first set of amplitude features and the second set of amplitude features can be determined according to Equation 7 below.

[0059]

[0060] where intensity j represents the step intensity of a set of frames that ends at the j th th frame in the acceleration signal stream 330, m represents the number of frames included in this set of frames, and dev k represents the deviation of the k th th frame in this set of frames, and this dev k is determined based on the first signal segment and the second signal segment related to the k th th frame according to any one of Equations 3 to 5.

[0061] It can be understood that the above Equation 7 is only an example for determining the step intensity. In another exemplary embodiment of the present specification, the step intensity can be determined according to any one of the following Equations 8 to 11.

[0062]

[0063]

[0064]

[0065]

[0066] where each symbol in Equations 8 to 11 has the same meaning as the symbols in Equation 7, and Δt represents the sampling interval of the acceleration signal stream 330.

[0067] Through the above embodiments, when determining the step event during the user's walking, the deviation related to a set of frames is considered, and the continuous change of the amplitude of the acceleration signal stream 330 can be monitored to identify the step event. Therefore, the recognition error caused by a single deviation related to a single time point can be reduced, and the accuracy and operability of step event recognition can be further enhanced.

[0068] In one embodiment of the present specification, a step event can be determined based on the comparison result between the step intensity and the threshold intensity according to Equation 12, where the threshold intensity can be determined based on experience.

[0069]

[0070] Among them, indicates whether the stepping event is determined based on the stepping intensity of the j-th th frame in the acceleration signal stream 330. intensity j represents the stepping intensity associated with the j-th th frame in the acceleration signal stream 330, and TH intensity represents the threshold intensity.

[0071] In one embodiment of the present specification, if the stepping intensity is lower than the threshold intensity, the first time window 310 and the second time window 320 may be shifted forward by another frame to obtain a new stepping intensity for a new set of frames including more frames. If the stepping intensity of this set of frames exceeds the threshold intensity, a stepping event is identified (e.g., at the j-th th frame). Once the stepping event is determined, this set of frames can be reset starting from the frames after this set of frames. At this point, the first time window 310 and the second time window 320 may continuously move forward along the acceleration signal stream 330 to identify further stepping events. For more detailed information about the stepping intensity, reference will be made to Figure 6 .

[0072] Figure 6 FIG. 600 is a block diagram showing the determination of a stepping intensity curve based on an acceleration signal stream according to an embodiment of the present specification. In Figure 6 , the horizontal axis represents the frames in the acceleration signal stream 330, and the vertical axis represents the stepping intensity determined based on the above formula. Generally, the strength of the two legs of a user may be different, so the peak of one leg may be different from the peak of the other leg. The curve of the stepping intensity shows a periodic pattern, where star icons (e.g., peak point 610 and peak point 620) represent the peaks in the stepping intensity, and dot icons represent the non-peaks in the stepping intensity. In Figure 6 , the frames between two consecutive peak points 610 and 620 can be identified as the stepping window 630 of the stepping event. Additionally and / or alternatively, the valleys in the stepping intensity can also be used to identify the stepping event.

[0073] It should be understood that Figure 6 the stepping intensity curve in [FIG. X] is only an example, and the curves determined for other users may show different patterns. For example, the curves of other users may show larger peaks and higher frequencies. Through the above embodiments, the deviation related to the duration can be considered to identify the stepping event. Therefore, the long-term behavior habits of the user can be monitored, and the errors caused by accidents can be reduced.

[0074] In one embodiment of the present specification, the above methods for identifying a stepping event can be combined. For example, the identification based on a single deviation and a stepping intensity can be combined to enhance the identification. Specifically, according to the following formula 13, at the j th th frame in the acceleration signal stream 330, a stepping event can be identified.

[0075]

[0076] Wherein, represents the final result of identifying a stepping event at the j th th frame in the acceleration signal stream 330, represents the result determined based on formula 6, represents the result determined based on formula 12. Through the above embodiments, a single deviation and a stepping intensity can be considered simultaneously when identifying a stepping event. Therefore, the error caused by an accident can be further reduced.

[0077] The above paragraph describes the detailed information regarding the identification of a stepping event. Subsequently, the identified stepping event can be further verified based on any acceleration amplitude and frequency limitations of the acceleration signal stream 330. Refer to Figure 7 , Figure 7 which shows a process block diagram 700 for determining the acceleration amplitude according to an embodiment of the present specification. Generally, an acceleration sensor can collect signals in a three-dimensional space. Therefore, the acceleration amplitude of a certain frame in the acceleration signal stream 330 can be determined by using the amplitudes in the x, y, and z dimensions.

[0078] As Figure 7 shown, at the j th th frame in the acceleration signal stream 330, the acceleration 710 is related to the x, y, and z coordinates. Therefore, the acceleration amplitude at the j th th frame can be determined based on the following formula 14 through the amplitude x 712, the amplitude y 714, and the amplitude z 716.

[0079]

[0080] Wherein, a j represents the acceleration amplitude related to the j th th frame in the acceleration signal stream 330, amplitude_x j , amplitude_y j and amplitude_z j respectively represent the amplitudes in the x, y, and z coordinates.

[0081] In addition, the stepping event can be verified according to the comparison result between the acceleration amplitude and a threshold amplitude in the following formula 15, where the threshold amplitude can be determined based on experience.

[0082]

[0083] Among them, indicates whether a stepping event is determined based on the acceleration magnitude of the j th th frame in the acceleration signal stream 330, a j represents the acceleration magnitude associated with the j th th frame in the acceleration signal stream 330, TH amplitude represents the threshold magnitude.

[0084] According to the above formula 15, if the acceleration magnitude at a certain frame of the determined stepping event exceeds the threshold magnitude, the stepping event can be verified. Through this embodiment, the identification of the stepping event can be more accurate.

[0085] In an embodiment of the present specification, the identification of a single deviation, stepping intensity, and acceleration magnitude can be combined together to further enhance the identification. Specifically, according to the following formula 16, a stepping event can be determined at the j th th frame in the acceleration signal stream 330.

[0086]

[0087] Among them, represents the final result of whether a stepping event is determined at the j th th frame in the acceleration signal stream 330, represents the result determined based on formula 6, represents the result determined based on formula 12, represents whether the stepping event is verified based on formula 15. Through the above embodiments, the accuracy and operability of the stepping event identification can be further enhanced.

[0088] In an embodiment of the present specification, a stepping event can be verified based on a frequency limit. The frequency limit can be determined according to historical experience. Usually, historical experience may indicate that the common walking frequency is one to three steps per second. If the frequency exceeds the frequency limit, a warning can be provided. Assuming that the frequency of the stepping event is 2 steps per second, the stepping event can be verified. If the frequency is 5 steps per second, the stepping event cannot be verified and a warning will be output. Through the above embodiments, the determined stepping event can be verified based on common sense of frequency. Therefore, more reliable stepping event identification can be provided.

[0089] It can be understood that determining a stepping event is only the beginning of indoor navigation. Once the stepping event is determined, the step length related to the stepping event can be obtained. For more information about step length determination, please refer to Figure 8 . Figure 8The process block diagram 800 for determining the step size according to an embodiment of this specification is shown. In an embodiment of this specification, a machine learning model can be constructed to determine the step size. In Figure 8 a sample data set including an acceleration signal stream 810 collected from a reference user and the step size 830 of the reference user can be used to train the step size model 820.

[0090] In an embodiment of this specification, the step size model 820 characterizes the polynomial association between the step size of a reference user among multiple reference users and the extreme values, average value, and frequency of the acceleration signal stream, where the acceleration signal stream is collected by an acceleration sensor associated with the reference user. For example, the step size model can be determined according to Equation 17 below.

[0091]

[0092] where, Length step represents a step size, a, b, c, d, and u represent hyperparameters, and respectively represent the minimum value, maximum value, and average value of the amplitude of the acceleration signal stream within a determined step window, f represents the walking frequency associated with the step event, m represents the number of frames included in the step window, and ΔT represents the sampling interval of the acceleration signal stream 330.

[0093] Through the above embodiments, the polynomial association can be simply trained. In an embodiment of this specification, the values of Length step , f, m, and Δt can be obtained to train the step size model 820. After the step size model 820 is successfully trained, the acceleration signal stream 330 can be simply processed and then input into the step size model 820 to determine the step size.

[0094] To obtain the step size of a user, the extreme values (including the maximum value and minimum value) within the step window associated with the step event can be determined from the acceleration signal stream 330, the average value of the step window can be determined, and the frequency of the step event can also be determined. In addition, the step size can be determined based on the step size model 820, extreme values, average value, and frequency. In other words, the acceleration signal stream 330 can be processed to extract the parameters related to Length step , f, m, and Δt. Next, the extracted parameters can be input into the step size model 820, and then the step size can be output.

[0095] It can be understood that the above Equation 17 is only an example of the step size model. In other embodiments, based on various types of machine learning techniques, the step size model 820 can be characterized in other forms other than polynomial association.

[0096] In one embodiment of the present specification, the user's behavior pattern can be considered when determining a stepping event and / or determining a step length. For example, the stepping mode can be classified into walking, strolling, stationary, and swaying modes. Therefore, various models can be trained according to the above-mentioned behavior patterns in order to obtain more accurate stepping events and related step lengths. In one example of the present specification, the movement direction related to the determined stepping event can be determined. Thereby, the user's speed and trajectory can be determined.

[0097] In one embodiment of the present specification, the user's speed can be determined based on the step length and the duration related to the step length. Alternatively, in addition to this, the speed can be determined based on a plurality of step lengths and a plurality of durations. For example, the sum of lengths can be calculated based on the lengths of a plurality of step lengths, and the sum of times can be calculated based on a plurality of durations. Then, the speed can be determined based on the sum of lengths and the sum of times.

[0098] After describing how to identify a stepping event and determine the step length related to the stepping event, the following is referred to Figure 9 to introduce the details about determining the user's movement direction. Figure 9 FIG. 900 is a schematic diagram showing the environment of an embodiment of the present specification. In Figure 9 , the user 910 carries the terminal device 920 in his / her pocket or holds it in his / her hand. However, the movement direction 912 of the user 910 may be different from the device direction 922 of the terminal device 920. Generally, there may be an angular difference 930 between the movement direction 912 and the device direction 922.

[0099] Previously, methods for determining the user's movement direction have been proposed. The device direction 922 can be determined by a dedicated sensor in the terminal device 920, and the device direction 922 is generally regarded as the movement direction 912. However, the device direction 922 and the movement direction 912 are not always the same, and during movement, the angular difference 930 may also change. Therefore, how to detect the movement direction 912 of the user 910 becomes the focus.

[0100] To at least partially solve the above problems and other potential problems, a new method and device for determining the movement direction are provided. According to an embodiment of the present specification, a deviation degree is defined to represent the deviation between the movement direction 912 of the user 910 and the actual device direction of the terminal device 920. If the deviation degree is lower than the threshold degree, the movement direction 912 can be determined based on the device direction 922. Otherwise, the movement direction 912 can be estimated by machine learning techniques.

[0101] Refer to Figure 10 to briefly illustrate the present specification. Figure 10FIG. 1000 is a process block diagram for determining a user's movement direction according to an embodiment of the present specification. As Figure 10 shown, the sensor 130 may provide a signal stream 1010. Among them, the sensor 130 may include an accelerometer 132 for collecting an acceleration signal stream and a gyroscope 134 for collecting a direction signal stream. The device direction 922 of the terminal device 920 may be obtained from the signal stream 1010 collected by the sensor 130 equipped in the terminal device 920.

[0102] In addition, a deviation degree 1020 is determined based on the signal stream 1010. The greater the deviation degree 1010, the greater the angle difference 930 may be. A threshold degree 1022 may be preset according to experience, and then the deviation degree 1020 may be compared with the threshold degree 1022 to obtain a comparison result 1024. Based on the comparison result 1024, a direction may be selected from the device direction 922 and the estimated movement direction 1030 as the movement direction 912.

[0103] Through the above embodiment, the deviation degree 1020 can measure whether the movement direction 912 of the user 910 is consistent with the actual device direction of the terminal device 920. Only when the two directions are consistent with each other, the movement direction 912 can be determined based on the device direction 922. Compared with the solution of directly using the device direction 922 as the movement direction 912, the movement direction 912 can be determined more accurately and reliably.

[0104] Refer to Figure 11 for a detailed description of the embodiments of the present specification. Figure 11 FIG. 1100 is a flowchart of a method 1100 for determining a user's movement direction according to an embodiment of the present specification. At block 1110, the device direction 922 of the terminal device 920 is obtained based on at least one signal stream collected from the terminal device 920 carried by the mobile user. In an embodiment of the present specification, the terminal device 920 may be equipped with an accelerometer 132 and a gyroscope 134. Therefore, at least one signal stream may include an acceleration signal stream collected by the accelerometer 132 and a direction signal stream collected by the gyroscope 134.

[0105] For more detailed information on determining the device direction 922, please refer to Figure 12 . Figure 12 FIG. 1200 is a process block diagram for determining a user's movement direction according to an embodiment of the present specification. Among them, the device direction 922 may be determined from the signal stream 1010, and the signal stream 1010 includes an acceleration signal stream 330 and a direction signal stream 1230. Among them, the direction signal stream 1230 may be represented by the angular velocity collected from the gyroscope 134. In the embodiments of the present specification, there is no limitation on the method for obtaining the device direction 922. On the contrary, a variety of known and / or future-developed methods may be used.

[0106] In one example, the device orientation 922 can be determined based on the IMU orientation. Alternatively, a machine learning model can be established, which characterizes the association between the device orientation of the reference terminal device and the acceleration signal stream and the orientation signal stream collected from the reference terminal device. After the machine learning model is trained, the acceleration signal stream 330 and the orientation signal stream 1230 can be input into the model to obtain the device orientation 922.

[0107] In addition, referring back Figure 11 , the deviation degree 1020 can be determined. At block 1120, the deviation degree 1020 is determined based on at least one signal stream 1010. Wherein, the deviation degree 1020 represents the deviation between the movement direction 912 of the user 910 and the actual device orientation of the terminal device 920. In one example of this specification, the deviation degree 1020 can be determined by machine learning techniques. As Figure 12 shown, a machine learning model 1210 can be provided to determine parameters related to the movement direction 912. Wherein, the machine learning model 1210 can include a deviation model 1220 and an orientation model 1240 for determining the deviation degree 1020.

[0108] In one embodiment of this specification, the deviation degree 1020 may be related to the angle difference. Specifically, the deviation degree 1020 at the i th th frame in the signal stream 1010 can be represented by Equation 18 as follows.

[0109] DevDegree i = |β i | Equation 18

[0110] Wherein, DevDegree i represents the deviation 1020 at the i th th frame in the signal stream 1010, and β i represents the angle difference at the i th th frame.

[0111] Wherein, the angle difference can be estimated by the deviation model 1220 in Figure 12 . The deviation model 1220 can characterize the association between the angle differences collected from the reference user and the signal stream collected from the terminal device carried by the reference user. The angle difference refers to the angle difference between the movement direction of the reference user and the actual device orientation of the reference user's terminal device. Generally speaking, the deviation degree 1020 may be proportional to the angle difference and increase as the angle difference increases. It can be understood that this specification does not limit the method of establishing the machine learning model. In one example, the deviation 1220 can be built based on Resnet modeling, and finally a fully connected layer can be used to predict the angle difference during movement.

[0112] A sample data set of reference users can be collected to train the deviation model 1220. The sample data set can include multiple samples, each sample including parameters related to a reference user. For example, each sample can include the angular difference, acceleration signal stream, and direction signal stream collected during the movement of the reference user. In addition, the sample data set can be used to train the deviation model 1220 so that the trained deviation model 1220 can characterize the association between the angular difference and the signal stream. Next, the acceleration signal stream 330 and the direction signal stream 1230 of the user 910 can be processed and then input into the deviation model 1220 to obtain the angular difference.

[0113] Through the above embodiments, the deviation model 1220 can be trained using the historical experience regarding the association between the angular difference and the signal stream. In turn, the deviation model 1220 can provide reliable knowledge for estimating the angular difference during the movement of the user 910.

[0114] In one embodiment of the present specification, the deviation degree 1020 can also be associated with the estimated movement direction 1030 of the user 910 and the device direction 922. Specifically, the deviation degree 1020 of the i th th frame in the signal stream 1010 can be determined based on Equation 19 below.

[0115] DevDegree i =|β i -(θ i -α i )| Equation 19

[0116] Where, DevDegree i represents the deviation degree 1020 of the i th th frame in the signal stream 1010, β i represents the angular difference of the i th th frame, θ i represents the estimated movement direction of the i th th frame, and α i represents the device direction of the i th th frame. Through the above embodiments, the deviation degree 1020 also considers the influence of the movement direction and the device direction, so a more accurate deviation degree 1020 can be obtained.

[0117] Among them, a direction model 1240 can be constructed to provide the estimated movement direction 1030. The direction model 1240 can be constructed based on the (Long Short-Term Memory, LSTM) architecture to estimate the movement direction. Among them, the 2D vector of the sin() and cos() functions related to each frame in the signal stream 1010 and the movement direction collected by the reference user can be used to train the direction model 1240.

[0118] In the training of the direction model 1240, a sample data set related to a reference user can be collected. The sample data set can include multiple samples, each sample related to a reference user. Each sample can include the actual movement direction of the reference user, as well as the acceleration signal stream and the direction signal stream collected during the movement of the reference user. In addition, the sample data set can be used to train the direction model 1240 so that the trained direction model 1240 can characterize the association between the movement direction and the signal stream. Then, the acceleration signal stream 330 and the direction signal stream 1230 from the user 910 can be received and input into the direction model 1240 to obtain the estimated movement direction 1030.

[0119] Through the above embodiments, the direction model 1240 can be trained using historical experience regarding the association between the movement direction and the signal stream. In turn, the direction model 1240 can provide reliable knowledge to estimate the movement direction 910 during the movement of the user.

[0120] It should be understood that Figure 12 Only the process of processing the signal stream 1010 through the deviation model 1220 and the direction model 1240 in the machine learning model 1210 is illustrated. In another embodiment, the machine learning model 1210 can adopt a different architecture. For example, a new model characterizes the association between the degree of deviation and the signal stream. At this time, the process of determining the degree of deviation 1020 based on the above formula may become an internal procedure of the new model. At this time, the signal stream 1010 can be directly input into the new model to obtain the corresponding degree of deviation.

[0121] It can be understood that during the movement of the user 910, the movement directions of two stepping events may be different. Therefore, the direction model 1240 can be trained based on the stepping events. Therefore, the stepping signal segments can be extracted from the signal stream of the sample data set based on the stepping events. Among them, the stepping events can be identified according to the method 300 described in the previous paragraph, and the first signal segment and the second signal segment can be obtained from within the first time window and the second time window in the acceleration signal stream of the reference user, respectively. Or and / or in addition, the stepping events can be determined according to other methods.

[0122] At this point, each sample in the sample data set may be related to multiple stepping events and include the stepping signal segments and movement directions corresponding to the respective stepping events. Then, the direction model 1240 can be trained based on the stepping events to characterize the association between the movement direction of the stepping events and the signal segments. Once the direction model 1240 is successfully trained, the stepping signal segments can be extracted from the acceleration signal stream 330 and then input into the direction model 1240 to obtain the estimated movement direction.

[0123] To extract the step signal segments from the acceleration signal stream 330, the first signal segment and the second signal segment can be obtained separately, and the extracted signal segments can be used to identify step events. Specifically, the step events can be identified by method 300, and then the step window associated with the step events can be determined. For more details on extracting step signal segments from the signal stream 1010, please refer to Figure 13 .

[0124] Figure 13 Figure 1300 is a block diagram showing a step window related to a user's step event according to an embodiment of the present specification. In Figure 13 , the acceleration signal stream 330 and the direction signal stream 1230 are collected simultaneously. Therefore, method 300 for detecting the step window 1330 based on the acceleration signal stream 330 is also applicable to the direction signal stream 1230. The segments of the step window 1330 in the acceleration signal stream 330 and the direction signal stream 1230 can be input into the direction model 1240 respectively to obtain the motion direction corresponding to the determined step event. Through the above embodiments, the motion direction of each step event can be determined, so the accuracy and operability can be further improved.

[0125] It can be understood that the speed of the user 910 is also an important factor in motion. The greater the speed, the greater the impact on the trajectory deviation of the user 910. For example, even if the angle difference is very small, when the user 910 moves quickly, the trajectory deviation may reach a larger value. Although the angle difference is large, and the user walks very slowly and almost remains stationary, the trajectory deviation may be very low. Therefore, in some embodiments of the present specification, the deviation degree 1020 may also be associated with the speed of the user 910. Specifically, the deviation degree 1020 of the i th th frame in the signal stream 1010 can be determined based on formula 20 shown below.

[0126] DevDegree i = v i ×|β i -(θ i -α i )| Formula 20

[0127] Where, DevDegree i represents the deviation degree 1020 of the i th th frame in the signal stream 1010, v i represents the speed of the user 910 in the i th th frame, β i represents the angle difference of the i th th frame, θ i represents the estimated motion direction of the i th th frame, α i represents the i thThe device orientation of the frame. It can be understood that the deviation degree 1020 is a scalar value, so the velocity v i may only contain the numerical value of the velocity without considering the direction.

[0128] In this embodiment, the velocity can be determined according to the method described in the above paragraph. Alternatively, the velocity can be obtained from a machine learning model. For example, the deviation model 1220 can be modified to also include the association between the velocity of the reference user and the signal flow. Then, the velocity of the user 910 can be obtained based on the deviation model 1220 and the signal flow 1010 of the user 910. Using the above formula 20, the velocity of the user can also be considered when determining the deviation degree 1020, so a more reliable and accurate deviation degree 1020 can be obtained.

[0129] After determining the deviation degree 1020, the movement direction 912 can be determined based on the comparison result between the deviation degree 1020 and the threshold degree 1022. Returning to Figure 11 At block 1130, based on the determination result that the deviation degree 1020 is lower than the threshold degree 1022, the movement direction 912 is determined based on the device orientation 922. It can be understood that a lower deviation degree may indicate that the movement direction 912 is relatively consistent with the device orientation 922. Therefore, the device orientation 922 can be directly used as the movement direction 912. In this way, the deviation between the movement direction and the device orientation can be effectively reduced.

[0130] In an example of this specification, if the deviation degree 1020 exceeds the threshold degree 1022, the movement direction 912 is determined based on the estimated movement direction 1030. At this time, since the device orientation 922 is significantly different from the actual movement direction, the estimated movement direction 1030 obtained from the direction model 1240 can be used as the movement direction 912. Since the estimated movement direction 1030 is based on the correct historical knowledge contained in the direction model 1240, the estimated movement direction 1030 can be very close to the actual movement direction, thereby improving the accuracy of movement determination.

[0131] Although the above paragraphs describe multiple embodiments of the process of determining the stepping event, determining the step length, and determining the movement direction, these embodiments can be combined together to form another embodiment. For example, based on the above embodiments, the user movement including both velocity and direction can be determined, and then the trajectory can be obtained.

[0132] In an embodiment of this specification, the above method 300 and method 1100 can be repeated, so that the trajectory of the user 910 can be obtained. Figure 14 A block diagram 1400 showing the determination of the user's trajectory according to an embodiment of this specification is shown. In Figure 14In it, three positions in the user's trajectory are shown, where the user 910 can take the first step while at position 1410. The above method 300 can be used to identify the first step, and the above method 1100 can be used to determine the movement direction of the first step. Assuming that the deviation degree exceeds the threshold degree, the movement direction 1412 is set as the estimated movement direction of the first step. The position 1420 is determined based on the step length determined according to the above method 300 and the movement direction 1412.

[0133] At position 1420, the user 910 may take a second step in another direction. Assuming that the deviation degree of the second step also exceeds the threshold degree, the movement direction 1422 is also set as the estimated movement direction, and then the user 910 reaches position 1430. In addition, the user 910 may take a third step. Assuming that the deviation degree of the third step is lower than the threshold degree, the movement direction 1432 is set as the device direction. Through the above embodiments, the trajectory of indoor navigation can be determined.

[0134] According to an embodiment of this specification, the above embodiments of indoor navigation can be combined with outdoor navigation. For example, two navigation modes can be provided in the navigator 110, where the GPS sensor can be used in the outdoor navigation mode, and the acceleration and direction sensors can be used to support indoor navigation.

[0135] Although the embodiments of this specification are described by taking the terminal device as an example of the processing device, the embodiments can also be executed on a general processing device. Figure 15 The device block diagram 1500 showing one or more embodiments of this specification is presented. It can be understood that the device 1500 is not intended to impose any limitation on the scope of use or features related to this specification, and the embodiments can be implemented in various general-purpose or special-purpose computer environments.

[0136] As shown in the figure, the device 1500 includes at least one processing unit (or processor) 1510 and a memory 1520. The processing unit 1510 executes computer-executable instructions and can be a real or virtual processor. In a multi-processing system, multiple processing units execute computer-executable instructions to improve processing capabilities. The memory 1520 can be a volatile memory (e.g., registers, caches, RAM), a non-volatile memory (e.g., ROM, EEPROM, flash memory), or some combination thereof.

[0137] In Figure 15In the example shown, device 1500 further includes a memory 1530, one or more input devices 1540, one or more output devices 1550, and one or more communication interfaces 1560. An interconnection mechanism (not shown), such as a bus, controller, or network, interconnects the components of device 1500. Typically, an operating system software (not shown) provides an operating environment for other software executed in device 1500 and coordinates the activities of the components of device 1500.

[0138] The memory 1530 can be removable or non-removable and can also include a computer-readable storage medium, such as a flash drive, a magnetic disk, or any other medium that can be used to store information and can be accessed in device 1500. The input device 1540 can be one or more of various different input devices. For example, the input device 1540 can be a user device including a mouse, a keyboard, a trackball, etc. The input device 1540 can implement one or more natural user interface technologies, such as voice recognition or touch and stylus recognition. As other examples, the input device 1540 can include a scanning device; a network adapter; or another device that provides input to device 1500. The output device 1550 can be a display, a printer, a speaker, a network adapter, or another device that provides output of device 1500. The input device 1540 and the output device 1550 can be incorporated into a single system or device, such as a touch screen or a virtual reality system.

[0139] The communication interface 1560 can communicate with another computing entity via a communication medium. In addition, the functions of the components of device 1500 can be implemented in a single computer or in multiple computers capable of communicating via the communication interface. Thus, device 1500 can operate in a network environment using a logical connection to one or more other servers, network PCs, or other general network nodes. By way of example only, the communication medium includes wired or wireless network technologies.

[0140] According to one embodiment of this specification, a navigator 110 can be configured on device 1500 to identify step events, determine the user's step length and direction of movement. In addition, the navigator 110 can provide the user's speed and trajectory.

[0141] For illustrative purposes only, some examples of embodiments will be listed below.

[0142] According to an embodiment of the present specification, a computer-implemented method for indoor navigation is provided. The method includes: obtaining a first signal segment and a second signal segment in a first time window and a second time window in an acceleration signal stream respectively, where the acceleration signal stream is collected by an acceleration sensor associated with a moving user, and the first time window is shorter than the second time window; determining a first amplitude feature and a second amplitude feature of the first time window and the second time window based on the first signal segment and the second signal segment respectively; and identifying a stepping event of the user based on a deviation between the first amplitude feature and the second amplitude feature.

[0143] According to an embodiment of the present specification, the first time window is within the second time window and has the same end frame as the second time window.

[0144] According to an embodiment of the present specification, determining the first amplitude feature includes: determining the first amplitude feature based on an average value of the first signal segment.

[0145] According to an embodiment of the present specification, determining the stepping event includes: determining a deviation based on a difference between the first amplitude feature and the second amplitude feature; and determining the stepping event based on the deviation exceeding a threshold deviation.

[0146] According to an embodiment of the present specification, determining the stepping event further includes: moving the first time window and the second time window along the acceleration signal stream; determining a stepping intensity associated with the movement of the first time window and the second time window respectively based on a first set of signal segments and a second set of signal segments obtained during the movement; and determining the stepping event based on the determined stepping intensity exceeding a threshold intensity.

[0147] According to an embodiment of the present specification, determining the stepping intensity includes: determining a first set of amplitude features and a second set of amplitude features based on the first set of signal segments and the second set of signal segments; and obtaining a sum of a set of deviations between the first set of amplitude features and the second set of amplitude features.

[0148] According to an embodiment of the present specification, the method further includes: obtaining an acceleration amplitude at each frame in the acceleration signal stream based on a plurality of vector values of the acceleration signal stream; and verifying the stepping event based on the acceleration amplitude at a certain frame where the stepping event is identified exceeding a threshold amplitude.

[0149] According to an embodiment of the present specification, the method further includes: verifying the stepping event based on the frequency of the stepping event being within a frequency limit range.

[0150] According to one embodiment of the present specification, the method further includes: determining a step length associated with a stepping event based on a step length model, where the step length model characterizes the association between the step length of a reference user and an acceleration signal stream collected by an acceleration sensor carried by the reference user.

[0151] According to one embodiment of the present specification, the method for determining a step length includes: identifying extreme values from an acceleration signal stream within a stepping window associated with a stepping event; determining an average value of the stepping window; and determining the step length based on the step length model, the extreme values, the average value, and the frequency of the stepping event.

[0152] According to one embodiment of the present specification, the step length model characterizes a polynomial association between the step length of a reference user among multiple reference users and the extreme values, the average value, and the frequency of an acceleration signal stream, where the acceleration signal stream is collected by an acceleration sensor associated with the reference user.

[0153] According to one embodiment of the present specification, the method further includes: obtaining a direction signal stream collected by a direction sensor associated with the user; determining a movement direction associated with a stepping event based on the acceleration signal stream and the direction signal stream; and determining the trajectory of the user based on the movement direction and the step length.

[0154] According to one embodiment of the present specification, this is an electronic device for indoor navigation. The device includes: a processing unit; and a memory connected to the processing unit and storing instructions executed by the processing unit. When the processing unit executes the instructions, the memory causes the device to perform the following operations: obtaining a first signal segment and a second signal segment in a first time window and a second time window in an acceleration signal stream respectively, where the acceleration signal stream is collected by an acceleration sensor associated with a moving user, and the first time window is shorter than the second time window; determining a first amplitude feature and a second amplitude feature of the first time window and the second time window respectively based on the first signal segment and the second signal segment; and identifying a stepping event of the user based on the deviation between the first amplitude feature and the second amplitude feature.

[0155] According to one embodiment of the present specification, the first time window is within the second time window and has the same end frame as the second time window.

[0156] According to one embodiment of the present specification, determining the first amplitude feature includes: determining the first amplitude feature based on the average value of the first signal segment.

[0157] According to one embodiment of the present specification, determining a stepping event includes: determining a deviation based on the difference between the first amplitude feature and the second amplitude feature; and determining a stepping event based on the deviation exceeding a threshold deviation.

[0158] According to one embodiment of the present specification, determining a stepping event further includes: moving a first time window and a second time window along an acceleration signal stream; determining a stepping intensity associated with the movement of the first time window and the second time window respectively based on a first set of signal segments and a second set of signal segments obtained during the movement; and determining a stepping event based on the determined stepping intensity exceeding a threshold intensity.

[0159] According to one embodiment of the present specification, determining a stepping intensity includes: determining a first set of amplitude features and a second set of amplitude features based on the first set of signal segments and the second set of signal segments; and obtaining a sum of a set of deviations between the first set of amplitude features and the second set of amplitude features.

[0160] According to one embodiment of the present specification, the method further includes: determining an acceleration amplitude at each frame in the acceleration signal stream based on a plurality of vector values of the acceleration signal stream; and verifying the stepping event based on the acceleration amplitude of the frame where the stepping event is recognized exceeding a threshold amplitude.

[0161] According to one embodiment of the present specification, the method further includes: verifying the stepping event based on the frequency of the stepping event being within a frequency limit range.

[0162] According to one embodiment of the present specification, the method further includes: determining a step length associated with the stepping event based on a step length model, where the step length model characterizes the association between the step length of a reference user and the acceleration signal stream, and the acceleration signal stream is collected by an acceleration sensor carried by the reference user.

[0163] According to one embodiment of the present specification, the method for determining a step length includes: identifying extreme values from the acceleration signal stream within a stepping window associated with the stepping event; determining an average value of the stepping window; and determining the step length based on the step length model, the extreme values, the average value, and the frequency of the stepping event.

[0164] According to one embodiment of the present specification, the step length model characterizes a polynomial association between the step length of a reference user among a plurality of reference users and the extreme values, the average value, and the frequency of the acceleration signal stream, where the acceleration signal stream is collected by an acceleration sensor associated with the reference user.

[0165] According to one embodiment of the present specification, the method further includes: obtaining a direction signal stream collected by a direction sensor associated with the user; determining a movement direction associated with the stepping event based on the acceleration signal stream and the direction signal stream; and determining the trajectory of the user based on the movement direction and the step length.

[0166] According to one embodiment of the present specification, a computer program product is provided for indoor navigation. The computer program product includes a computer-readable storage medium containing program instructions, which when executed by an electronic device, cause the electronic device to execute a method for indoor navigation.

[0167] According to one embodiment of the present specification, a computer-readable storage medium is provided for indoor navigation. The medium contains program instructions, which when executed by an electronic device, cause the electronic device to execute an indoor navigation method.

[0168] According to one embodiment of the present specification, a computer-implemented method is provided for indoor navigation. The method includes: obtaining the device orientation of a terminal device based on at least one signal stream collected from the terminal device carried by a mobile user; determining a deviation degree based on the at least one signal stream, where the deviation degree characterizes the deviation between the movement direction of the user and the actual device orientation of the terminal device; and determining the movement direction based on the device orientation based on the determination result that the deviation degree is lower than a threshold degree.

[0169] According to one embodiment of the present specification, determining the deviation degree includes: obtaining an estimated movement direction of the user based on at least one signal stream; obtaining an estimated angular difference between the movement direction and the actual device orientation; and determining the deviation degree based on the device orientation, the estimated movement direction, and the estimated angular difference.

[0170] According to one embodiment of the present specification, the method further includes: determining the movement direction based on the estimated movement direction according to the determination result that the deviation degree exceeds the threshold degree.

[0171] According to one embodiment of the present specification, obtaining the estimated movement direction includes: obtaining a direction model, where the direction model characterizes the association between the movement direction of a reference user and the signal stream collected from the terminal device of the reference user; and obtaining the estimated movement direction based on the direction model and at least one signal stream of the user.

[0172] According to one embodiment of the present specification, obtaining the estimated movement direction further includes: identifying a step window related to the step event of the user in at least one signal stream; and obtaining the estimated movement direction based on the direction model and the step signal segment within the step window in the signal stream.

[0173] According to one embodiment of the present specification, the at least one signal stream includes an acceleration signal stream obtained by an acceleration sensor of the terminal device and an orientation signal stream obtained by an orientation sensor of the terminal device.

[0174] According to one embodiment of the present specification, identifying a stepping event includes: obtaining a first signal segment and a second signal segment within a first time window and a second time window respectively from an acceleration signal stream; and identifying a stepping event based on a comparison result between the first signal segment and the second signal segment.

[0175] According to one embodiment of the present specification, obtaining an estimated angle difference includes: obtaining a deviation model, where the deviation model characterizes the association between the angle difference of a reference user and the signal stream collected from the terminal device of the reference user, and one of the angle differences is the angle difference between the movement direction of a reference user among the reference users and the actual device direction of the terminal device of the reference user; and obtaining an estimated angle difference based on the deviation model and at least one signal stream of the user.

[0176] According to one embodiment of the present specification, determining the deviation degree further includes: determining the movement speed of the user based on at least one signal stream; and determining the deviation degree based on the movement speed, device direction, estimated movement direction, and estimated angle difference.

[0177] According to one embodiment of the present specification, the method further includes: determining the trajectory of the user based on the movement direction and movement speed.

[0178] According to one embodiment of the present specification, an electronic device is provided for indoor navigation. The device includes: a processing unit; a memory connected to the processing unit and storing instructions executed by the processing unit, and when the processing unit executes the instructions, the operations guided by the device include: obtaining the device direction of the terminal device based on at least one signal stream collected from the terminal device carried by a mobile user; determining the deviation degree based on at least one signal stream, where the deviation degree represents the deviation between the movement direction of the user and the actual device direction of the terminal device; and determining the movement direction based on the device direction according to the determination result that the deviation degree is lower than a threshold degree.

[0179] According to one embodiment of the present specification, determining the deviation degree includes: obtaining the estimated movement direction of the user based on at least one signal stream; obtaining the estimated angle difference between the movement direction and the actual device direction; and determining the deviation degree based on the device direction, estimated movement direction, and estimated angle difference.

[0180] According to one embodiment of the present specification, the operation further includes: determining the movement direction based on the estimated movement direction according to the determination result that the deviation degree exceeds the threshold degree.

[0181] According to one embodiment of the present specification, obtaining the estimated movement direction includes: obtaining a direction model, where the direction model characterizes the association between the movement direction of a reference user and the signal stream collected from the terminal device of the reference user; and obtaining the estimated movement direction based on the direction model and at least one signal stream of the user.

[0182] According to one embodiment of this specification, obtaining an estimated motion direction further includes: identifying a step window associated with a user's step event in at least one signal stream; and obtaining the estimated motion direction based on a direction model and step signal segments within the step window in the signal stream.

[0183] According to one embodiment of this specification, the at least one signal stream includes an acceleration signal stream obtained by an acceleration sensor in a terminal device and a direction signal stream obtained by a direction sensor in the terminal device.

[0184] According to one embodiment of this specification, identifying a step event includes: obtaining a first signal segment and a second signal segment within a first time window and a second time window respectively from the acceleration signal stream; and identifying the step event based on a comparison result between the first signal segment and the second signal segment.

[0185] According to one embodiment of this specification, obtaining an estimated angular difference includes: obtaining a deviation model, where the deviation model characterizes the association between the angular differences of a reference user and the signal streams collected from the terminal devices of the reference users, and one of the angular differences is the angular difference between the motion direction of a reference user among the reference users and the actual device direction of the reference user's terminal device; and obtaining the estimated angular difference based on the deviation model and at least one signal stream of the user.

[0186] According to one implementation of this specification, determining the deviation degree further includes: determining the motion speed of the user based on at least one signal stream; and determining the deviation degree based on the motion speed, device direction, estimated motion direction, and estimated angular difference.

[0187] According to one embodiment of this specification, the operation further includes: determining the trajectory of the user based on the motion direction and motion speed.

[0188] According to one embodiment of this specification, a computer program product is provided for indoor navigation. The computer program product includes a computer-readable storage medium containing program instructions, and the program instructions are executed by an electronic device to enable the electronic device to execute a method for indoor navigation.

[0189] According to one embodiment of this specification, a computer-readable storage medium is provided. The storage medium contains program instructions, and the program instructions are executed by an electronic device to enable the electronic device to execute an indoor navigation method.

[0190] Embodiments of this specification may further include one or more computer program products tangibly stored on a non-transitory machine-readable medium and including machine-executable instructions. When the instructions are executed on an electronic device, the electronic device is caused to execute the one or more processes described above.

[0191] In general, various embodiments can be implemented in hardware or special-purpose circuitry, software, logical operations, or any combination thereof. Some embodiments can be implemented in hardware, while other embodiments can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. Although the various embodiments of this specification are illustrated and described by block diagrams, flowcharts, or some other graphical representation, it is to be understood that the modules, devices, systems, techniques, or methods described herein can be implemented, by way of non-limiting example, in hardware, software, firmware, special-purpose circuitry or logic, general-purpose hardware or controllers, or other computing devices, or any combination thereof.

[0192] In the context of this specification, a machine-readable medium can be any tangible medium that can contain or store a program associated with an instruction execution system, apparatus, or device, or used in connection therewith. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatus, or any suitable combination thereof. More specific examples of a machine-readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0193] The computer program code for carrying out the methods of this specification can be written in one or more programming languages, or any combination thereof. This computer program code can be provided to the processor of a general-purpose computer, special-purpose computer, or other programmable data processing device, so that when the program code is executed by the computer or other programmable data processing device, the functions or operations specified in the flowchart and / or block diagram to be implemented can be achieved. The program code can be executed entirely on the computer, as a stand-alone software package, partly on the computer, partly on a remote computer, or entirely on a remote computer or server.

[0194] In addition, although the operations are depicted in a particular order, it should not be construed that the operations are required to be performed in the particular order shown or in sequential order, or that all of the recited operations be performed, to achieve the desired result. In some cases, multitasking and parallel processing may be more advantageous. Similarly, although the detailed information of several specific embodiments is included in the foregoing discussion, it should not be construed as a limitation of any scope of disclosure or the scope that may be claimed, but rather as a description of specific features of particular embodiments of the specific disclosure. Certain features that are separately described in the context of separate embodiments in this specification may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in any suitable combination.

[0195] When read in conjunction with the accompanying drawings, various modifications and applications of the foregoing embodiments of this specification will be apparent to those skilled in the art in light of the above description. Any and all such modifications will still fall within the scope of the non-limiting and exemplary embodiments of this specification. In addition, other embodiments related to this specification will be readily contemplated by those skilled in the art, and the foregoing description and the introduction of the drawings teach such related embodiments. Accordingly, it is to be understood that the embodiments of this specification are not limited to the specific embodiments disclosed, and are intended to include modifications and other embodiments within the scope of the claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

1. A computer-implemented method, comprising: Obtaining a device orientation of the terminal device based on at least one signal stream collected from a terminal device carried by a mobile user; Obtaining a deviation model, the deviation model characterizing an association between an angular difference of a reference user and a signal stream collected from a terminal device of the reference user, one of the angular differences being an angular difference between a movement direction of one of the reference users and an actual device orientation of the terminal device of the reference user; The deviation model is a machine learning model; Obtaining an estimated angular difference based on the deviation model and the at least one signal stream of the user; Obtaining an estimated movement direction of the user based on the at least one signal stream; Determining a deviation degree based on the device orientation, the estimated movement direction, and the estimated angular difference; The deviation degree is determined based on a second difference between the estimated angular difference and a first difference, the first difference being a difference between the estimated movement direction and the device orientation, and the deviation degree characterizes a deviation between the movement direction of the user and the actual device orientation of the terminal device; And Based on a determination result that the deviation degree is lower than a threshold degree, determining the movement direction based on the device orientation.

2. The method according to claim 1, wherein Further comprising: Based on a determination result that the deviation degree exceeds the threshold degree, determining the movement direction based on the estimated movement direction.

3. The method according to claim 1, wherein Obtaining the estimated movement direction includes: obtaining a direction model, the direction model characterizing an association between a movement direction of a reference user and a signal stream collected from a terminal device of the reference user; and Obtaining the estimated movement direction based on the direction model and the at least one signal stream of the user.

4. The method according to claim 3, wherein Obtaining the estimated movement direction further includes: Identifying a step window related to a step event of the user in the at least one signal stream; and Obtaining the estimated movement direction based on the direction model and a step signal segment within the step window in the signal stream.

5. The method according to claim 4, characterized in that, The at least one signal stream includes an acceleration signal stream acquired by an acceleration sensor of the terminal device and a direction signal stream acquired by a direction sensor of the terminal device.

6. The method according to claim 5, characterized in that Identifying the step event includes: Obtaining a first signal segment and a second signal segment within a first time window and a second time window respectively from the acceleration signal stream; and Identifying the step event based on a comparison result of the first signal segment and the second signal segment.

7. The method according to claim 1, characterized in that Determining the deviation degree further includes: determining a movement speed of the user based on the at least one signal stream; and Determining the deviation degree based on the movement speed, the device orientation, the estimated movement direction, and the estimated angular difference.

8. The method according to claim 7, wherein Further comprising: Determining a trajectory of the user based on the movement direction and the movement speed.

9. An electronic device, comprising: A processing unit; And A memory connected to the processing unit and storing instructions executed by the processing unit, when the processing unit executes the instructions, the operations guided by the device include: Obtaining a device orientation of the terminal device based on at least one signal stream collected from a terminal device carried by a mobile user; Obtain a deviation model, where the deviation model characterizes the association between the angular difference of a reference user and the signal flow collected from the terminal device of the reference user, and one of the angular differences is the angular difference between the movement direction of a reference user among the reference users and the actual device direction of the terminal device of the reference user; the deviation model is a machine learning model; Based on the deviation model and the at least one signal flow of the user, obtain an estimated angular difference; Obtain the estimated movement direction of the user based on the at least one signal flow; Determine a deviation degree based on the device direction, the estimated movement direction, and the estimated angular difference; the deviation degree is determined based on the second difference between the estimated angular difference and a first difference, and the first difference is the difference between the estimated movement direction and the device direction; The deviation degree characterizes the deviation between the movement direction of the user and the actual device direction of the terminal device; and Based on the determination result that the deviation degree is lower than a threshold degree, determine the movement direction based on the device direction.

10. The device according to claim 9, characterized in that, The operation further includes: Based on the determination result that the deviation degree exceeds the threshold degree, determine the movement direction based on the estimated movement direction.

11. The device according to claim 9, wherein, Obtaining the estimated movement direction includes: obtaining a direction model, where the direction model characterizes the association between the movement direction of a reference user and the signal flow collected from the terminal device of the reference user; and Obtain the estimated movement direction based on the direction model and the at least one signal flow of the user.

12. The device according to claim 11, wherein, Obtaining the estimated movement direction further includes: Identifying a step window related to the step event of the user in the at least one signal flow; and Obtain the estimated movement direction based on the direction model and the step signal segment within the step window in the signal flow.

13. The device according to claim 12, characterized in that, The at least one signal flow includes an acceleration signal flow obtained by an acceleration sensor of the terminal device and a direction signal flow obtained by a direction sensor of the terminal device.

14. The device according to claim 13, characterized in that, Identifying the step event includes: Obtaining a first signal segment and a second signal segment within a first time window and a second time window respectively from the acceleration signal flow; and Based on the comparison result of the first signal segment and the second signal segment, identify the step event.

15. The device according to claim 9, wherein, Determining the deviation degree further includes: determining the movement speed of the user based on the at least one signal flow; and Determine the deviation degree based on the movement speed, the device direction, the estimated movement direction, and the estimated angular difference.

16. The device according to claim 15, characterized in that, The operation further includes: Determine the trajectory of the user based on the movement direction and the movement speed.

17. A computer program product, the computer program product includes a computer-readable storage medium containing program instructions, and when the program instructions are executed by an electronic device, the electronic device executes the method according to any one of claims 1 to 8.

18. A computer-readable storage medium containing program instructions, and when the program instructions are executed by an electronic device, the electronic device executes the method according to any one of claims 1 to 8.

Images