Indoor positioning, indoor navigation method, device, electronic equipment and storage medium

Determining the usage status of the mobile device through IMU sensors and status recognition networks solves the positioning accuracy problem caused by the difficulty of users to maintain a fixed grip posture, and realizes high-precision indoor positioning in different states, reducing costs and dependencies.

CN114061579BActive Publication Date: 2024-11-15HUAWEI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010754482.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-07-30
Publication Date
2024-11-15
Estimated Expiration
2040-07-30

AI Technical Summary

Technical Problem

The existing indoor positioning method requires the user to maintain a fixed grip posture of the mobile device, making it difficult to ensure positioning accuracy in actual use.

Method used

The usage status of the mobile device is determined through the IMU sensor, and the status identification network is used to select the corresponding location identification network, determine the amount of location changes, and reduce the dependence on external systems and deployment costs.

Benefits of technology

Selecting the appropriate location identification network under different usage states improves positioning accuracy and robustness, reduces dependence on external systems, and reduces maintenance costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114061579B_ABST
    Figure CN114061579B_ABST
Patent Text Reader

Abstract

This application discloses a method, apparatus, electronic device, and storage medium for indoor positioning and navigation, belonging to the field of positioning technology. In this method, after a mobile device determines its usage state, it can select a corresponding location identification network to determine the amount of position change based on that state. That is, different location identification networks are selected to determine the amount of position change under different usage states, eliminating the need for the user to maintain a fixed grip on the mobile device, resulting in strong robustness and guaranteed positioning accuracy. Furthermore, the mobile device can achieve indoor positioning through an IMU sensor, reducing dependence on external systems and lowering deployment and maintenance costs.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of positioning technology, and in particular to a method, device, electronic device and storage medium for indoor positioning and indoor navigation. Background Art

[0002] Indoor positioning refers to the realization of position positioning in an indoor environment. Since satellite positioning cannot be used in indoor environments, how to accurately achieve indoor positioning has become a problem that needs to be solved urgently.

[0003] Related technologies propose an indoor positioning method based on pedestrian dead reckoning (PDR) technology. PDR technology mainly determines the change in the user's position indoors through the inertial measurement unit (IMU) built into the mobile device. Specifically, the mobile device determines a step gain through calibration during initialization. When positioning indoors, the acceleration signal and angular velocity signal are determined through the built-in IMU, and the acceleration signal is subjected to footstep detection. The step length of the current walk is determined by the result of the footstep detection and the step length gain obtained by calibration. At the same time, the angular velocity signal is processed to determine the current walking direction. Afterwards, the current position is determined by the step length and direction of the current walk.

[0004] However, the above method requires the user to maintain a fixed holding posture of the mobile device, for example, the mobile device is held flat in front of the chest with the top of the mobile device facing forward, that is, the orientation of the mobile device is aligned with the orientation of the user, and the orientation of the mobile device is used to represent the walking direction of the user, thereby ensuring the accuracy of indoor positioning. However, in actual use, it is difficult to align the orientation of the mobile device with the orientation of the user, which makes it difficult to ensure the accuracy of indoor positioning. Summary of the invention

[0005] The embodiments of the present application provide a method, device, electronic device and storage medium for indoor positioning and indoor navigation, which can solve the problem in the related art that the orientation of the mobile device and the orientation of the user are difficult to align, thereby making it difficult to ensure indoor positioning accuracy. The technical solution is as follows:

[0006] In a first aspect, an indoor positioning method is provided, in which a mobile device determines a first IMU signal through an IMU sensor, uses the first IMU signal as an input of a state recognition network, and determines the use state of the mobile device. A position recognition network corresponding to the use state of the mobile device is determined according to a first corresponding relationship, a position change is determined through the position recognition network, and a current indoor position is determined according to the position change.

[0007] Generally, an IMU sensor includes an accelerometer, a gyroscope, a magnetometer, etc. In the embodiment of the present application, an IMU sensor including an accelerometer and a gyroscope is used as an example for explanation. Among them, the accelerometer is used to collect acceleration signals, and the gyroscope is used to collect angular velocity signals.

[0008] The usage status of a mobile device refers to the status of a user using the mobile device, such as putting it in a pocket, talking on the phone, holding it in a natural swing, holding it up, or using it in a backpack.

[0009] After the mobile device determines its own usage status, it can select the corresponding location identification network to determine the location change according to the determined usage status. That is, in different usage states, different location identification networks are selected to determine the location change. The user does not need to maintain a fixed holding posture of the mobile device, which is more robust and the positioning accuracy can be guaranteed. In addition, the mobile device can achieve indoor positioning through the IMU sensor, reducing the dependence on external systems, as well as reducing deployment and maintenance costs.

[0010] The mobile device uses the first IMU signal as an input of the state recognition network. There are multiple implementation methods for determining the usage status of the mobile device, and two of the implementation methods are introduced below.

[0011] In a first implementation, the first IMU signal is divided according to the step detection method to obtain multiple signal segments, each of which includes an IMU signal of a step. Each of the multiple signal segments is used as an input of a state recognition network to determine the usage state of the mobile device when each of the multiple signal segments corresponds to a step.

[0012] That is, in the first implementation, the mobile device divides the first IMU signal into frames, and each signal segment after framing includes an IMU signal of a step. Then, for each step, the use state of the mobile device is determined through the state recognition network.

[0013] The state recognition network is used to recognize the state of the user using the mobile device. The input of the state recognition network is the divided signal segment, and the output is the recognition result of the use state of the mobile device. In the embodiment of the present application, the state recognition network is determined by using a neural network and deep learning. Next, the determination process of the state recognition network is introduced.

[0014] As an example, a mobile device obtains a plurality of sample signal segments and a usage state corresponding to each of the plurality of sample signal segments. The plurality of sample signal segments are used as inputs of an initial state recognition network, and the usage state corresponding to each of the plurality of sample signal segments is used as outputs of the initial state recognition network. The initial state recognition network is trained to obtain a state recognition network.

[0015] Since the usage state of the mobile device is different in different situations, in order to improve the applicability of the state recognition network, the usage states corresponding to the multiple sample signal segments are also different. In this way, after the initial state recognition network is trained by the multiple sample signal segments and the usage state corresponding to each of the multiple sample signal segments, the success rate of the usage state recognized by the state recognition network is higher.

[0016] In a second implementation, the first IMU signal is used as an input of a state recognition network to determine the usage state of the mobile device in a plurality of time windows, where the plurality of time windows are used to divide the first IMU signal.

[0017] That is, in the second implementation, the mobile device divides the first IMU signal according to multiple time windows through the state recognition network, and determines the usage status of the mobile device in each time window. Different from the first implementation, the state recognition network is not only used to identify the user's use of the mobile device, but also used to divide the first IMU signal according to multiple time windows. That is, in the second implementation, the state recognition network integrates the function of IMU signal division and the function of usage status recognition.

[0018] Since the input of the state recognition network in the second implementation is different from that in the first implementation, the determination process of the state recognition network in the second implementation is also different from that in the first implementation. Next, the determination process of the state recognition network in the second implementation is introduced.

[0019] As an example, a mobile device obtains multiple sample IMU signals and the usage status of each sample IMU signal in multiple time windows. The multiple sample IMU signals are used as inputs of an initial state recognition network, and the usage status of each sample IMU signal in multiple time windows is used as output of the initial state recognition network. The initial state recognition network is trained to obtain a state recognition network.

[0020] Since the usage state of the mobile device is different in different situations, in order to improve the applicability of the state recognition network, the usage states of the multiple sample IMU signals in multiple time windows are also different. In this way, after the initial state recognition network is trained by the multiple sample IMU signals and the usage state of each sample IMU signal in the multiple time windows, the success rate of the usage state recognized by the state recognition network is higher.

[0021] Since the dynamic change of the relative position between the user and the mobile device will have a great impact on the positioning accuracy, the embodiment of the present application uses a state recognition network to identify the use state of the mobile device. Moreover, for the above two implementations, the state recognition network is determined by a neural network and a deep learning method, which can improve the robustness and accuracy of state recognition.

[0022] Optionally, the indoor positioning method provided in the embodiment of the present application is implemented when the user is walking, so the mobile device uses the first IMU signal as the input of the state recognition network to determine the use state of the mobile device, and it is also necessary to determine whether it is currently in a walking state. That is, the first IMU signal is used as the input of the behavior recognition network to determine the current user behavior state. If the determined user behavior state is a walking state, the step of using the first IMU signal as the input of the state recognition network to determine the use state of the mobile device is executed.

[0023] The behavior recognition network is used to determine the user's behavior status, such as driving, walking, climbing stairs, etc. The input of the behavior recognition network is the IMU signal, and the output of the behavior recognition network is the recognition result of the user's behavior status. Next, the determination process of the behavior recognition network is introduced.

[0024] As an example, a mobile device obtains multiple sample IMU signals and user behavior states corresponding to the multiple sample IMU signals. The multiple sample IMU signals are used as inputs of an initial behavior recognition network, and the user behavior states corresponding to the multiple sample IMU signals are used as outputs of the initial behavior recognition network. The initial behavior recognition network is trained to obtain a behavior recognition network.

[0025] Since the user's behavior state may be different in different situations, in order to improve the applicability of the behavior recognition network, the user behavior states corresponding to the multiple sample IMU signals are different. In this way, after the behavior recognition network is trained by the multiple sample IMU signals and the user behavior states corresponding to the multiple sample IMU signals, the success rate of the user behavior state recognized by the behavior recognition network is higher.

[0026] In the first implementation mode described above, the mobile device determines the location identification network corresponding to its own usage state according to the first corresponding relationship, and determines the location change amount through the location identification network, including: selecting a signal segment from the multiple signal segments as a reference signal segment, performing the following operations on the reference signal segment until the following operations have been performed on each of the multiple signal segments: extracting features from the reference signal segment to obtain a feature vector corresponding to the reference signal segment. Determine the location identification network corresponding to the usage state of the mobile device when a step corresponding to the reference signal segment is performed according to the first corresponding relationship, and use the determined location identification network as the reference location identification network. Use the feature vector corresponding to the reference signal segment as the input of the reference location identification network to determine the location change amount of a step corresponding to the reference signal segment.

[0027] Since a signal segment includes the IMU signal of a footstep, taking the reference signal segment among multiple signal segments as an example, after the mobile device determines the feature vector corresponding to the reference signal segment, it determines the position change of a footstep corresponding to the reference signal segment through the reference position recognition network. In this way, after determining the position change of each footstep, the position of the mobile device in the room can be determined, which is relatively simple to operate and does not rely on external systems.

[0028] Of course, after the mobile device determines the usage status of the mobile device at each step according to the first implementation method, it can also determine the position change of each step in other ways.

[0029] In the first implementation, the position recognition network is used to determine the position change of each footstep. The input of the position recognition network is the feature vector corresponding to the divided signal segment, and the output is the position change of one footstep corresponding to the corresponding signal segment. The position change includes the change in step length and the change in direction. Next, the determination process of the position recognition network is introduced.

[0030] As an example, a location recognition network is selected from the location recognition networks included in the first correspondence as the first location recognition network, and the first location recognition network is trained by the following operations until each location recognition network included in the first correspondence is trained: multiple sample feature vectors and the position change amount corresponding to each of the multiple sample feature vectors are obtained, and the multiple sample feature vectors refer to feature vectors corresponding to IMU signal segments obtained by a mobile device in a first state of use, and the first state refers to the use state corresponding to the first location recognition network. The multiple sample feature vectors are used as the input of the initial location recognition network, and the position change amount corresponding to each of the multiple sample feature vectors is used as the output of the initial location recognition network, and the initial location recognition network is trained to obtain the first location recognition network.

[0031] Since different usage states correspond to different position recognition networks, that is, a position recognition network can only determine the position change under a corresponding usage state. Therefore, in order to improve the recognition accuracy of the position recognition network, the multiple sample feature vectors are feature vectors corresponding to the IMU signal segments obtained under the same usage state.

[0032] In the second implementation mode described above, the mobile device determines the location identification network corresponding to its usage state according to the first corresponding relationship, and determines the location change amount through the location identification network, including: selecting a time window from multiple time windows as a reference time window, and performing the following operations on the reference time window until the following operations have been performed on each of the multiple time windows: determining the location identification network corresponding to the usage state of the mobile device in the reference time window according to the first corresponding relationship, and using the determined location identification network as the reference location identification network. Using the signal segment in the first IMU signal within the reference time window as the input of the reference location identification network, the location change amount corresponding to the reference time window is determined.

[0033] Dividing the IMU signal by footsteps through footstep detection to determine the position change of each footstep is only one implementation method. The mobile device can also divide the IMU signal by time window to determine the position change corresponding to each time window, thereby determining the current indoor position of the mobile device.

[0034] Of course, after the mobile device determines the usage status of the mobile device in each time window according to the second implementation manner, the mobile device can also determine the position change amount in each time window in other ways.

[0035] In the second implementation, the position recognition network is used to determine the position change in each time window. The input of the position recognition network is a signal segment in the first IMU signal that is within a time window, and the output is the position change in the corresponding time window. The position change includes the change in step length and the change in orientation. Next, the determination process of the position recognition network is introduced.

[0036] As an example, a location recognition network is selected from the location recognition networks included in the first correspondence as the first location recognition network, and the first location recognition network is trained by the following operations until each location recognition network included in the first correspondence is trained: a plurality of sample signal segments and the position change amount corresponding to each of the plurality of sample signal segments are obtained, the plurality of sample signal segments refer to IMU signal segments obtained by a mobile device in a first state of use, and the first state refers to the use state corresponding to the first location recognition network. The plurality of sample signal segments are used as the input of the initial location recognition network, and the position change amount corresponding to each of the plurality of sample signal segments is used as the output of the initial location recognition network, and the initial location recognition network is trained to obtain the first location recognition network.

[0037] Since different usage states correspond to different position identification networks, that is, a position identification network can only determine the position change under a corresponding usage state. Therefore, in order to improve the recognition accuracy of the position identification network, the multiple sample signal segments are IMU signal segments obtained under the same usage state.

[0038] The implementation process of the mobile device determining the current indoor position according to the determined position change is as follows: the mobile device obtains first position information, which refers to the position information of the starting position of the mobile device. Then, the first position information is added to the determined position change to obtain the current indoor position of the mobile device.

[0039] Since the position change includes the change in step length and the change in direction, and the first position information includes position and direction, the change in step length determined this time is added to the position included in the first position information to obtain the current position of the mobile device. Based on the direction included in the first position information, the change in direction determined this time is added to obtain the current direction of the mobile device. In this way, the current indoor position of the mobile device can be accurately determined.

[0040] The indoor positioning method provided in the embodiment of the present application can be applied to a variety of scenarios, such as indoor navigation. This scenario is introduced below.

[0041] In the first case, the mobile device also obtains second location information, which refers to the location information of the destination location to be reached by the mobile device, and determines a navigation path from the starting location to the destination location based on the first location information and the second location information. The navigation path and the current indoor location of the mobile device are displayed in the navigation interface to perform indoor navigation.

[0042] In this case, the mobile device may obtain a map of the indoor location where the mobile device is located, and calculate a navigation path from the starting location to the destination location according to the map of the indoor location based on the first location information and the second location information.

[0043] The map of the indoor parking lot can be a map drawn in advance for the indoor parking lot, or a trajectory map generated by the cloud server after merging the walking trajectories of multiple users. For the latter, the mobile device needs to obtain the outdoor positioning information obtained last time before the mobile device enters the indoor place, and then obtain the trajectory map of the indoor place from the cloud server based on the outdoor positioning information.

[0044] In the first case, the scene of finding a car in an indoor parking lot, so the second location information is the parking position of the target vehicle in the indoor parking lot. Before the mobile device obtains the second location information, it is also necessary to determine the second location. That is, determine the third location information, which refers to the location information of the parking position in the local coordinate system. According to the third location information, determine the user's walking trajectory; when the outdoor positioning information is obtained, determine the second location information according to the location information corresponding to the outdoor positioning information on the user's walking trajectory, and the third location information.

[0045] In some embodiments, the mobile device determines a second IMU signal through an IMU sensor. The second IMU signal is used as an input to the behavior recognition network to determine the current user behavior state. If the determined user behavior state is a driving state, the step of determining the second IMU signal through the IMU sensor is returned until the determined user behavior state is a walking state, and it is determined that the user has currently parked the target vehicle, that is, the user's parking event is currently captured. At this time, the location information determined by the above-mentioned indoor positioning method is the location information of the parking position.

[0046] After the mobile device stores the location information of the parking location, in the process of finding a car, the mobile device can obtain the location information of the current location in the world coordinate system, and then determine a navigation path. That is, based on the first location information and the second location information, a navigation path can be determined. The navigation path is then displayed in the navigation interface, so that the user can find the target vehicle in the indoor parking lot according to the information displayed on the navigation interface, solving the difficulty of not being able to obtain satellite signals in indoor places and thus being unable to locate. Moreover, the method provided in the embodiment of the present application only needs to be completed at the architecture layer of the mobile device, does not rely on external hardware, and has a low cost.

[0047] When the user enters the indoor place and is not very clear about the layout of the indoor place, or cannot obtain the map of the indoor place, and does not have or cannot access the positioning system of the indoor place, in this scenario, it may be difficult for the user to walk out of the indoor place, or the user may get lost, which will take a long time. Therefore, in this scenario, the mobile device determines a navigation path from the starting position to the destination position based on the first position information and the second position information, including: selecting a target user walking trajectory including the first position information and the second position information from one or more stored user walking trajectories; and determining the target user walking trajectory as the navigation path from the starting position to the destination position.

[0048] In the above scenario, the second location information is the location information of the entrance through which the user enters the indoor place. Before the mobile device selects the target user walking trajectory including the first location information and the second location information from one or more stored user walking trajectories, it also includes: determining whether the outdoor positioning information can be obtained at the current moment; if the outdoor positioning information cannot be obtained at the current moment and the outdoor positioning information can be obtained at the previous moment, then determining the third IMU signal through the IMU sensor; using the third IMU signal as the input of the behavior recognition network to determine the current user behavior state; if the determined user behavior state is a walking state, then determining that the user is currently entering the indoor place, and recording and storing the user's walking trajectory in the indoor place.

[0049] Since satellite signals cannot be obtained in indoor environments, and outdoor positioning information cannot be obtained, in the embodiment of the present application, the mobile device can determine whether outdoor positioning information can be obtained at the current moment. If outdoor positioning information cannot be obtained at the current moment and outdoor positioning information can be obtained at the previous moment, then it can be determined that the user is currently in an indoor place. For further verification, the mobile device can also determine the third IMU signal through the IMU sensor, use the third IMU signal as the input of the behavior recognition network, determine the current user behavior state, and if the determined user behavior state is a walking state, then it can be determined that the user is currently entering an indoor place.

[0050] In the second case, the mobile device obtains the outdoor positioning information obtained for the last time before the mobile device enters the indoor place, and obtains the trajectory map of the indoor place from the cloud server based on the outdoor positioning information. The trajectory map of the indoor place is generated by the cloud server after fusion of the walking trajectories of multiple users. The current indoor position of the mobile device and the trajectory map of the indoor place are displayed in the navigation interface for indoor navigation.

[0051] The cloud server generates a trajectory map of the indoor place by fusing the walking trajectories of multiple users. That is, by crowdsourcing the walking trajectories of users, the map path information of the indoor place is collected, which can reduce the cost of building the trajectory map of the indoor place. In addition, when other users enter the indoor place later, the mobile device obtains the trajectory map of the indoor place from the cloud server, and navigates according to the trajectory map of the indoor place.

[0052] In some embodiments, the user may determine a destination location. In this way, after the mobile device displays the current indoor location of the mobile device and the track map of the indoor place in the navigation interface, it also includes: obtaining the destination location information in the world coordinate system, and determining the navigation path through the track map of the indoor place according to the location information of the current indoor location of the mobile device and the destination location. The navigation path is displayed in the navigation interface for navigation. That is, the mobile device determines which paths can be walked and which paths cannot be walked through the track map of the indoor place, thereby determining a navigation path according to the track map, and the user can walk to the destination location according to the navigation path.

[0053] Based on the above description, the trajectory map of the indoor place is generated by the cloud server after merging the trajectory maps of multiple users, and during this navigation process, the user may find other feasible paths. Therefore, after the mobile device determines the current indoor position of the mobile device through the above-mentioned indoor positioning method based on the acquired outdoor positioning information, it also includes: determining the current user walking trajectory based on the current indoor position of the mobile device, and sending the current user walking trajectory to the cloud server so that the cloud server updates the trajectory map of the indoor place, which can ensure the real-time update of the trajectory map of the indoor place and reduce the maintenance cost of the trajectory map.

[0054] Optionally, the track map of the indoor place is also marked with key landmark information, and the key landmark information is used to indicate the key landmarks in the indoor place. For example, the key landmarks in the indoor place are entrances, exits, escalators, elevators, etc. In this way, after the track map is displayed in the navigation interface, the user can also learn about the key landmarks of the indoor place in a timely manner.

[0055] In the case where key landmark information is marked on the trajectory map of the indoor place, after the mobile device determines the current user walking trajectory according to its current indoor position, it also includes: determining the change of the outdoor positioning signal and the change of the indoor elevation corresponding to each trajectory point on the current user walking trajectory. Sending the change of the outdoor positioning signal and the change of the indoor elevation of each trajectory point on the current user walking trajectory to the cloud server, so that the cloud server updates the key landmark information on the trajectory map of the indoor place.

[0056] In a second aspect, an indoor navigation method is provided, in which a cloud server receives outdoor positioning information obtained by a mobile device, and obtains a track map of an indoor place corresponding to the outdoor positioning information, wherein the track map of the indoor place is generated by the cloud server after performing track fusion on multiple user walking tracks. The cloud server sends the track map of the indoor place to the mobile device, so that the mobile device navigates in the indoor place according to the track map of the indoor place.

[0057] The cloud server generates a trajectory map of the indoor place by fusing the walking trajectories of multiple users. That is, by crowdsourcing the walking trajectories of users, the map path information of the indoor place is collected, which can reduce the cost of building the trajectory map of the indoor place. In addition, when other users enter the indoor place later, the mobile device obtains the trajectory map of the indoor place from the cloud server, and navigates according to the trajectory map of the indoor place.

[0058] In the process of mobile device navigating through the track map obtained from the cloud server, the user may find other feasible paths. Therefore, after the cloud server sends the track map of the indoor place to the mobile device, it also includes: receiving the user walking track currently in the indoor place sent by the mobile device, generating a track heat map according to the user walking track sent by the mobile device and the user walking track currently stored in the indoor place, and regenerating the track map of the indoor place according to the track heat map to update the track map. In this way, the real-time update of the track map of the indoor place can be guaranteed, and the maintenance cost of the track map is reduced.

[0059] Optionally, the trajectory map of the indoor place is also marked with key landmark information, and the key landmark information is used to indicate the key landmarks in the indoor place. For example, the key landmarks in the indoor place are entrances, exits, escalators, elevators, etc. Therefore, after the cloud server sends the trajectory map of the indoor place to the mobile device, it also includes: receiving the changes in the outdoor positioning signal and the changes in the indoor elevation of each trajectory point on the user's walking trajectory currently in the indoor place sent by the mobile device. In this way, after the cloud server generates a trajectory heat map based on the user's walking trajectory sent by the mobile device and the user's walking trajectory currently stored in the indoor place, it can also mark the changes in the outdoor positioning signal and the changes in the indoor elevation of each trajectory point on the trajectory heat map.

[0060] The change of the outdoor positioning signal refers to whether the mobile device can detect the outdoor positioning signal at the current track point. If the outdoor positioning signal cannot be detected at the current track point, but the outdoor positioning signal can be detected at the previous track point, it indicates that the user is currently entering the room from the outdoors, that is, the current track point may be the entrance to the indoor place. On the contrary, if the outdoor positioning signal can be detected at the current track point, but the outdoor positioning signal cannot be detected at the previous track point, it indicates that the user is currently walking from the indoor place to the outdoors, that is, the current track point may be the exit of the indoor place.

[0061] The change of indoor elevation refers to whether there is a change in elevation of the mobile device at the current trajectory point. If there is a change in elevation of the mobile device at the current trajectory point, then the current trajectory point may be an escalator or a straight ladder. Specifically, if there is a change in elevation at the current trajectory point, and there are also changes in elevation at the next multiple consecutive trajectory points, then it indicates that the current trajectory point may be an escalator. If there is a change in elevation at the current trajectory point, and there is no change in elevation at the next multiple consecutive trajectory points, then it indicates that the current trajectory point may be a straight ladder.

[0062] Since the user's walking trajectory is composed of multiple trajectory points, and each trajectory point on the user's trajectory is in the same coordinate system, the cloud server draws the user's walking trajectory sent by the mobile device and the user's walking trajectory currently stored in the indoor place on a map. This map includes multiple trajectory points, and each trajectory point also corresponds to a heat value. The heat value is used to indicate the number of user trajectories including the trajectory point. This map is called a trajectory heat map. In this way, the cloud server selects multiple candidate trajectory points from the multiple trajectory points included in the trajectory heat map according to the heat value corresponding to each trajectory point in the trajectory heat map, regenerates the trajectory map of the indoor place according to the multiple candidate trajectory points, and marks key landmark information on the trajectory map according to the changes in the outdoor positioning signals and the changes in the indoor elevation at the multiple candidate trajectory points.

[0063] In a third aspect, an indoor positioning device is provided, wherein the indoor positioning device has the function of implementing the indoor positioning method in the first aspect. The indoor positioning device includes at least one module, and the at least one module is used to implement the indoor positioning method provided in the first aspect.

[0064] In a fourth aspect, an indoor navigation device is provided, wherein the indoor navigation device has the function of implementing the indoor navigation method in the second aspect. The indoor navigation device includes at least one module, and the at least one module is used to implement the indoor navigation method provided in the second aspect.

[0065] In a fifth aspect, an electronic device is provided, specifically a mobile device, the mobile device comprising a processor and a memory, the memory being used to store a program for executing the indoor positioning method provided in the first aspect, and storing data involved in implementing the indoor positioning method provided in the first aspect. The processor is configured to execute the program stored in the memory. The operating device of the storage device may also include a communication bus, which is used to establish a connection between the processor and the memory.

[0066] In a sixth aspect, an electronic device is provided, specifically a cloud server, the cloud server comprising a processor and a memory, the memory being used to store a program for executing the indoor navigation method provided in the second aspect, and storing data involved in implementing the indoor navigation method provided in the second aspect. The processor is configured to execute the program stored in the memory. The operating device of the storage device may further comprise a communication bus, the communication bus being used to establish a connection between the processor and the memory.

[0067] In a seventh aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the indoor positioning method described in the first aspect.

[0068] In an eighth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is executed on a computer, the computer executes the indoor navigation method described in the second aspect.

[0069] In a ninth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the indoor positioning method described in the first aspect.

[0070] In a tenth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the indoor navigation method described in the second aspect.

[0071] The technical effects obtained in the third to tenth aspects mentioned above are similar to the technical effects obtained by the corresponding technical means in the above method, and will not be repeated here.

[0072] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:

[0073] After the mobile device determines its own usage status, it can select the corresponding location identification network to determine the location change according to the determined usage status. That is, in different usage states, different location identification networks are selected to determine the location change. The user does not need to maintain a fixed holding posture of the mobile device, which is more robust and the positioning accuracy can be guaranteed. In addition, the mobile device can achieve indoor positioning through the IMU sensor, reducing dependence on external systems and reducing deployment and maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 This is an internal architecture diagram of a mobile device provided in an embodiment of the present application;

[0075] Figure 2 is a flow chart of an indoor positioning method provided by an embodiment of the present application;

[0076] Figure 3 is a schematic diagram of an acceleration signal provided in an embodiment of the present application;

[0077] Figure 4 It is a structural diagram of a state recognition network provided in an embodiment of the present application;

[0078] Figure 5 is a schematic diagram of the structure of a location identification network provided in an embodiment of the present application;

[0079] Figure 6 is a schematic diagram of the structure of another location identification network provided in an embodiment of the present application;

[0080] Figure 7 This is an architecture diagram of a method for implementing indoor positioning provided by an embodiment of the present application;

[0081] Figure 8 It is an architecture diagram of another method for implementing indoor positioning provided by an embodiment of the present application;

[0082] Fig. 9 This is a flow chart of a method for finding a car in an indoor parking lot provided by an embodiment of the present application;

[0083] Fig.10 is a schematic diagram of a navigation interface provided in an embodiment of the present application;

[0084] Fig.11 is a schematic diagram of another navigation interface provided in an embodiment of the present application;

[0085] Fig.12 This is a flow chart of an indoor navigation method provided by an embodiment of the present application;

[0086] Fig.13 is a schematic diagram of a user's walking trajectory in an indoor place provided by an embodiment of the present application;

[0087] Fig.14 is a flow chart of another method for indoor navigation provided by an embodiment of the present application;

[0088] Fig.15 is a schematic diagram of generating a trajectory heat map provided in an embodiment of the present application;

[0089] Fig.16 is a structural schematic diagram of an indoor positioning device provided in an embodiment of the present application;

[0090] Fig.17 is a structural schematic diagram of an indoor navigation device provided in an embodiment of the present application;

[0091] Fig.18 is a structural schematic diagram of a computer device provided in an embodiment of the present application;

[0092] Fig.19 A schematic diagram of a terminal provided in an embodiment of the present application;

[0093] Fig. 20 A schematic diagram of the structure of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0094] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0095] Since satellite signals cannot be obtained in indoor environments, satellite positioning cannot be used. The indoor positioning method proposed in the related art requires the orientation of the mobile device to be aligned with the orientation of the user, but in actual use, the orientation of the mobile device and the orientation of the user are difficult to align, making it difficult to ensure the accuracy of indoor positioning. Based on this, an embodiment of the present application provides an indoor positioning method, which is applied to a mobile device, such as a smart phone, a smart watch, a bracelet, smart glasses, augmented reality glasses, etc.

[0096] In actual use, the indoor positioning method provided in the embodiment of the present application can be deployed in a mobile device as a module of a positioning basic service at the architecture layer. Figure 1 The mobile device includes an application layer, an architecture layer and a hardware layer. The application layer is used to run various applications that can achieve positioning or navigation. The architecture layer includes a PDR positioning module and an outdoor positioning module, and both the PDR module and the outdoor module can provide a positioning service interface for the upper layer application. Among them, the PDR positioning module is used to implement the indoor positioning method provided in the embodiment of the present application, and the outdoor positioning module is an existing positioning module, including a global navigation satellite system (GNSS) positioning module or an LBS positioning module. The hardware layer includes an IMU sensor, and the IMU sensor includes an accelerometer, a gyroscope, a magnetometer, and the like.

[0097] Figure 2 This is a flowchart of an indoor positioning method provided by an embodiment of the present application. Please refer to Figure 2 ,The indoor positioning method includes the following steps.

[0098] Step 201: The mobile device determines a first IMU signal through an IMU sensor.

[0099] Typically, an IMU sensor includes an accelerometer, a gyroscope, a magnetometer, and the like. In an embodiment of the present application, an IMU sensor including an accelerometer and a gyroscope is used as an example for illustration. Among them, the accelerometer is used to collect acceleration signals, and the gyroscope is used to collect angular velocity signals. In the case where the IMU sensor includes an accelerometer and a gyroscope, the acceleration signal collected by the accelerometer and the angular velocity signal collected by the gyroscope are both referred to as IMU signals. In order to facilitate the distinction from the IMU signals in subsequent embodiments, the IMU signal here is referred to as the first IMU signal.

[0100] Step 202: The mobile device determines a usage status of the mobile device according to the first IMU signal.

[0101] The use state of the mobile device refers to the state of the user using the mobile device, such as putting it in a pocket, talking on the phone, holding it for natural swing, holding it up, wearing it in a backpack, etc. There are many ways to implement the mobile device to determine the use state of the mobile device according to the first IMU signal, and two of them are introduced below.

[0102] In a first implementation, the first IMU signal is divided according to the step detection method to obtain multiple signal segments, each of which includes an IMU signal of a step. Each of the multiple signal segments is used as an input of a state recognition network to determine the usage state of the mobile device when each of the multiple signal segments corresponds to a step.

[0103] That is, in the first implementation, the mobile device divides the first IMU signal into frames, and each signal segment after framing includes an IMU signal of a step. Then, for each step, the use state of the mobile device is determined through the state recognition network.

[0104] Among them, footstep detection refers to identifying the user's walking footsteps by detecting two adjacent peaks of the first IMU signal. Figure 3 The acceleration signal shown has time on the horizontal axis and the amplitude value of acceleration on the vertical axis. For the peak whose amplitude value is greater than the amplitude threshold, the signal segment between two adjacent peaks is used as the acceleration signal of one step. In this way, not only can the acceleration signal be divided according to the footsteps, but also the influence of the noise signal can be eliminated by the amplitude threshold. For the angular velocity signal, it can also be divided in a similar way to the acceleration signal.

[0105] After the mobile device divides the first IMU signal into multiple signal segments by means of footstep detection, the use state of the mobile device is determined according to each signal segment when the user takes a step. Taking the reference signal segment among the multiple signal segments as an example, the mobile device uses the reference signal segment as the input of the state recognition network, and outputs the use state of the mobile device at a step corresponding to the reference signal segment through the state recognition network. Among them, the reference signal segment is a signal segment among the multiple signal segments, and for other signal segments among the multiple signal segments, the use state of the mobile device at a corresponding step can also be determined in a similar manner to the reference signal segment.

[0106] The state recognition network is used to recognize the state of the user using the mobile device. The input of the state recognition network is the divided signal segment, and the output is the recognition result of the use state of the mobile device. In the embodiment of the present application, the state recognition network is determined by using a neural network and deep learning. Next, the determination process of the state recognition network is introduced.

[0107] As an example, a mobile device obtains a plurality of sample signal segments and a usage state corresponding to each of the plurality of sample signal segments. The plurality of sample signal segments are used as inputs of an initial state recognition network, and the usage state corresponding to each of the plurality of sample signal segments is used as outputs of the initial state recognition network. The initial state recognition network is trained to obtain a state recognition network.

[0108] Since the usage state of the mobile device is different in different situations, in order to improve the applicability of the state recognition network, the usage states corresponding to the multiple sample signal segments are also different. In this way, after the initial state recognition network is trained by the multiple sample signal segments and the usage state corresponding to each of the multiple sample signal segments, the success rate of the usage state recognized by the state recognition network is higher.

[0109] In the embodiment of the present application, the structure of the state recognition network includes multiple types, and one structure of the state recognition network is listed here. Figure 4 As shown in FIG, the state recognition network includes a convolutional layer C1, a convolutional layer C2, a pooling layer P1, a flattening layer F1, a dense layer D1 and a dense layer D2. Figure 4 In the state recognition network shown, a signal segment passes through two convolutional layers, a pooling layer, a flattening layer, and two dense layers to output the recognition result of the usage status of the mobile device.

[0110] In a second implementation, the first IMU signal is used as an input of a state recognition network to determine the usage state of the mobile device in a plurality of time windows, where the plurality of time windows are used to divide the first IMU signal.

[0111] That is, in the second implementation, the mobile device divides the first IMU signal according to multiple time windows through the state recognition network, and determines the usage status of the mobile device in each time window. Different from the first implementation, the state recognition network is not only used to identify the user's use of the mobile device, but also used to divide the first IMU signal according to multiple time windows. That is, in the second implementation, the state recognition network integrates the function of IMU signal division and the function of usage status recognition.

[0112] Among them, the sizes of the multiple time windows are equal. The size of each time window is set according to the demand. As an example, the size of the time window is determined according to the sampling frequency of the first IMU signal. For example, the sampling frequency of the first IMU signal is 200HZ, that is, 200 points are collected in one second, and the size of the time window can be set to 0.5 seconds.

[0113] Since the input of the state recognition network in the second implementation is different from that in the first implementation, the determination process of the state recognition network in the second implementation is also different from that in the first implementation. Next, the determination process of the state recognition network in the second implementation is introduced.

[0114] As an example, a mobile device obtains multiple sample IMU signals and the usage status of each sample IMU signal in multiple time windows. The multiple sample IMU signals are used as inputs of an initial state recognition network, and the usage status of each sample IMU signal in multiple time windows is used as output of the initial state recognition network. The initial state recognition network is trained to obtain a state recognition network.

[0115] Since the usage state of the mobile device is different in different situations, in order to improve the applicability of the state recognition network, the usage states of the multiple sample IMU signals in multiple time windows are also different. In this way, after the initial state recognition network is trained by the multiple sample IMU signals and the usage state of each sample IMU signal in the multiple time windows, the success rate of the usage state recognized by the state recognition network is higher.

[0116] Since the dynamic change of the relative position of the user and the mobile device will have a great impact on the positioning accuracy, the embodiment of the present application adds a process for identifying the usage status of the mobile device, that is, the usage status of the mobile device is identified through a state recognition network, and the robustness and accuracy of state recognition are improved through neural networks and deep learning methods.

[0117] Optionally, the indoor positioning method provided in the embodiment of the present application is implemented when the user is walking, so the mobile device uses the first IMU signal as the input of the state recognition network to determine the use state of the mobile device, and it is also necessary to determine whether it is currently in a walking state. That is, the first IMU signal is used as the input of the behavior recognition network to determine the current user behavior state. If the determined user behavior state is a walking state, the step of using the first IMU signal as the input of the state recognition network to determine the use state of the mobile device is executed.

[0118] The behavior recognition network is used to determine the user's behavior status, such as driving, walking, climbing stairs, etc. The input of the behavior recognition network is the IMU signal, and the output of the behavior recognition network is the recognition result of the user's behavior status. Next, the determination process of the behavior recognition network is introduced.

[0119] As an example, a mobile device obtains multiple sample IMU signals and user behavior states corresponding to the multiple sample IMU signals. The multiple sample IMU signals are used as inputs of an initial behavior recognition network, and the user behavior states corresponding to the multiple sample IMU signals are used as outputs of the initial behavior recognition network. The initial behavior recognition network is trained to obtain a behavior recognition network.

[0120] Since the user's behavior state may be different in different situations, in order to improve the applicability of the behavior recognition network, the user behavior states corresponding to the multiple sample IMU signals are different. In this way, after the behavior recognition network is trained by the multiple sample IMU signals and the user behavior states corresponding to the multiple sample IMU signals, the success rate of the user behavior state recognized by the behavior recognition network is higher.

[0121] Step 203: The mobile device determines the location identification network corresponding to the usage state of the mobile device according to the first corresponding relationship, and determines the location change amount through the location identification network.

[0122] In step 202, two implementation methods for determining the usage status of the mobile device are introduced. Next, the implementation process of step 203 will be introduced in two implementation methods.

[0123] In a first implementation, a signal segment is selected from the multiple signal segments as a reference signal segment, and the following operations are performed on the reference signal segment until each of the multiple signal segments has performed the following operations: feature extraction is performed on the reference signal segment to obtain a feature vector corresponding to the reference signal segment. A location recognition network corresponding to the usage state of the mobile device when a step corresponding to the reference signal segment is determined according to the first corresponding relationship, and the determined location recognition network is used as a reference location recognition network. The feature vector corresponding to the reference signal segment is used as the input of the reference location recognition network to determine the location change of a step corresponding to the reference signal segment.

[0124] Corresponding to the first implementation method in step 202, after the mobile device is divided to obtain multiple signal segments, for the reference signal segment in the multiple signal segments, the mobile device extracts features of the reference signal segment to obtain a feature vector corresponding to the reference signal segment. Since different usage states correspond to different location identification networks, and the first corresponding relationship stores location identification networks corresponding to different usage states, the mobile device selects the corresponding location identification network from the first corresponding relationship as the reference location identification network according to the usage state of the mobile device at the time of a step corresponding to the reference signal segment. The feature vector corresponding to the reference signal segment is used as the input of the reference location identification network to determine the position change amount of a step corresponding to the reference signal segment.

[0125] Among them, the mobile device extracts features from the reference signal segment to obtain a feature vector corresponding to the reference signal segment as follows: the mobile device downsamples the reference signal segment, performs feature calculations based on the downsampled reference signal segment, and obtains a feature vector corresponding to the reference signal segment.

[0126] Feature operations include but are not limited to operations such as the mean, variance, and standard deviation of the signal. If the mobile device performs feature operations in N dimensions based on the reference signal segment after downsampling, then the feature vector corresponding to the reference signal segment also includes elements in N dimensions. For example, if the mobile device performs operations in the three dimensions of mean, variance, and standard deviation based on the reference signal segment after downsampling, then the feature vector corresponding to the reference signal segment also includes elements in the three dimensions of mean, variance, and standard deviation.

[0127] In some embodiments, the mobile device stores the multiple location identification networks according to the usage status corresponding to each location identification network, that is, stores the first correspondence relationship. In this way, after determining the usage status of the mobile device during a footstep corresponding to the reference signal segment, the corresponding location identification network can be determined from the first correspondence relationship.

[0128] In the first implementation, the position recognition network is used to determine the position change of each footstep. The input of the position recognition network is the feature vector corresponding to the divided signal segment, and the output is the position change of one footstep corresponding to the corresponding signal segment. The position change includes the change in step length and the change in direction. Next, the determination process of the position recognition network is introduced.

[0129] As an example, a location recognition network is selected from the location recognition networks included in the first correspondence as the first location recognition network, and the first location recognition network is trained by the following operations until each location recognition network included in the first correspondence is trained: multiple sample feature vectors and the position change amount corresponding to each of the multiple sample feature vectors are obtained, and the multiple sample feature vectors refer to feature vectors corresponding to IMU signal segments obtained by a mobile device in a first state of use, and the first state refers to the use state corresponding to the first location recognition network. The multiple sample feature vectors are used as the input of the initial location recognition network, and the position change amount corresponding to each of the multiple sample feature vectors is used as the output of the initial location recognition network, and the initial location recognition network is trained to obtain the first location recognition network.

[0130] Since different usage states correspond to different position recognition networks, that is, a position recognition network can only determine the position change under a corresponding usage state. Therefore, in order to improve the recognition accuracy of the position recognition network, the multiple sample feature vectors are feature vectors corresponding to the IMU signal segments obtained under the same usage state.

[0131] It should be noted that the structures of the multiple position recognition networks are the same or different, and the structures of each position recognition network include multiple structures. For example, for the position recognition network in the natural swing state of holding, two structures will be listed below. Figure 5 As shown in FIG. 1 , the position recognition network for holding the natural swing state includes a convolution layer C1, a convolution layer C2, a flattening layer F1, a dense layer D1, and a dense layer D2. Figure 6 As shown, the position recognition network for the natural swing state includes a deep residual network R18 and a dense layer D1. When a signal segment corresponds to a footstep, the use state of the mobile device is the natural swing state. Figure 5 The position recognition network shown in the figure, the feature vector corresponding to this signal segment passes through two convolutional layers, one flattening layer, and two dense layers, and outputs the position change corresponding to this signal segment at one step. Figure 6 The position recognition network shown in the figure, the feature vector corresponding to this signal segment passes through a deep residual network and a dense layer, and outputs the position change corresponding to this signal segment in one step.

[0132] Of course, after the mobile device determines the usage status of the mobile device at each step according to the first implementation method of step 202, it can also determine the position change of each step in other ways, not limited to the first implementation method in step 203.

[0133] In a second implementation, a time window is selected from multiple time windows as a reference time window, and the following operations are performed on the reference time window until each of the multiple time windows has performed the following operations: a location identification network corresponding to the use state of the mobile device in the reference time window is determined according to the first corresponding relationship, and the determined location identification network is used as the reference location identification network. A signal segment in the first IMU signal that is within the reference time window is used as an input of the reference location identification network to determine the position change corresponding to the reference time window.

[0134] Corresponding to the second implementation in step 202, since different usage states correspond to different location identification networks, and the first corresponding relationship stores location identification networks corresponding to different usage states, for the reference time window in the multiple time windows, after the mobile device determines its own usage state in the reference time window, it selects the corresponding location identification network from the first corresponding relationship as the reference location identification network according to the usage state of the mobile device in the reference time window. The signal segment in the first IMU signal that is within the reference time window is used as the input of the reference location identification network to determine the position change corresponding to the reference time window.

[0135] In some embodiments, the mobile device stores the multiple location identification networks according to the usage status corresponding to each location identification network, that is, stores the first corresponding relationship, so that after determining the usage status of the mobile device in the reference time window, the corresponding location identification network can be determined from the first corresponding relationship.

[0136] In the second implementation, the position recognition network is used to determine the position change in each time window. The input of the position recognition network is a signal segment in the first IMU signal that is within a time window, and the output is the position change in the corresponding time window. The position change includes the change in step length and the change in orientation. Next, the determination process of the position recognition network is introduced.

[0137] As an example, a location recognition network is selected from the location recognition networks included in the first correspondence as the first location recognition network, and the first location recognition network is trained by the following operations until each location recognition network included in the first correspondence is trained: a plurality of sample signal segments and the position change amount corresponding to each of the plurality of sample signal segments are obtained, the plurality of sample signal segments refer to IMU signal segments obtained by a mobile device in a first state of use, and the first state refers to the use state corresponding to the first location recognition network. The plurality of sample signal segments are used as the input of the initial location recognition network, and the position change amount corresponding to each of the plurality of sample signal segments is used as the output of the initial location recognition network, and the initial location recognition network is trained to obtain the first location recognition network.

[0138] Since different usage states correspond to different position identification networks, that is, a position identification network can only determine the position change under a corresponding usage state. Therefore, in order to improve the recognition accuracy of the position identification network, the multiple sample signal segments are IMU signal segments obtained under the same usage state.

[0139] Of course, after the mobile device determines the usage status of the mobile device in each time window according to the second implementation method of step 202, it can also determine the position change amount in each time window in other ways, not limited to the second implementation method in step 203.

[0140] Combined with the description of steps 202-203 above, it can be seen that the mobile device can determine the position change in two ways, that is, the mobile device includes two architectures to implement the indoor positioning method provided in the embodiment of the present application. One of the architectures is referenced as follows: Figure 7 As shown, after the mobile device determines the first IMU signal through the IMU sensor, after the footstep detection module divides the signal into multiple segments, the feature extraction module determines the feature vector of each signal segment, and the state recognition module determines the use state of the mobile device when each signal segment corresponds to a footstep. Next, the PDR position recognition module selects a position recognition network, and the selected position recognition network determines the position change amount when each signal segment corresponds to a footstep.

[0141] Another architecture reference Figure 8 As shown, after the mobile device determines the first IMU signal through the IMU sensor, on the one hand, the use status of the mobile device in each of the multiple time windows is determined through the state recognition module, and on the other hand, the output of the state recognition module and the first IMU signal are combined to select the position recognition network through the PDR position recognition module, and the position change of each time window is determined through the selected position recognition network.

[0142] Step 204: The mobile device determines the current indoor location according to the determined location change amount.

[0143] In the embodiment of the present application, the mobile device obtains the first location information, which refers to the location information of the starting location of the mobile device. Then, the first location information is added to the determined location change amount to obtain the current indoor location of the mobile device.

[0144] Since the position change includes the change in step length and the change in direction, and the first position information includes position and direction, the change in step length determined this time is added to the position included in the first position information to obtain the current position of the mobile device. Based on the direction included in the first position information, the change in direction determined this time is added to obtain the current direction of the mobile device. In this way, the current indoor position of the mobile device can be accurately determined.

[0145] It should be noted that if the above-mentioned first IMU signal is the first IMU signal detected by the mobile device in an indoor place, then the mobile device directly uses the current indoor position as the starting position in the indoor place. If the above-mentioned first IMU signal is the second IMU signal detected by the mobile device in an indoor place, then the mobile device can obtain the first position information, and then add the position change determined this time on the basis of the first position information to obtain the current indoor position of the mobile device. If the above-mentioned first IMU signal is an IMU signal after the second IMU signal detected by the mobile device in an indoor place, then the mobile device can obtain the most recently determined indoor position before the first IMU signal is detected, and add the position change determined this time on the basis of this indoor position to obtain the current indoor position of the mobile device.

[0146] If the starting position of the mobile device is the position information in the world coordinate system, then the current indoor position of the mobile device determined in the above steps is also the position information in the world coordinate system. If the starting position of the mobile device is not the position information in the world coordinate system, then the current indoor position of the mobile device determined in the above steps is not the position information in the world coordinate system, but the position in the local coordinate system, that is, the current indoor position of the mobile device determined in the above steps is not aligned with the world coordinate system.

[0147] The local coordinate system is a coordinate system defined by the mobile device with a certain point as the origin and three directions defined by the mobile device as the three coordinate axes. For example, a coordinate system is established with the upper left corner of the mobile device as the origin and the long side, short side and height of the mobile device as the three coordinate axes.

[0148] According to the above method, the mobile device can determine its own position change, and then determine the indoor position of the mobile device at each moment. In this way, a continuous user walking trajectory can be drawn through the indoor position of the mobile device at each moment. The user walking trajectory is a walking trajectory in a local coordinate system, or a walking trajectory in a world coordinate system.

[0149] In summary, after the mobile device determines its own usage status, it can select the corresponding location identification network to determine the location change according to the determined usage status. That is, in different usage states, different location identification networks are selected to determine the location change. The user does not need to maintain a fixed holding posture of the mobile device, which is more robust and the positioning accuracy can be guaranteed. In addition, the mobile device can achieve indoor positioning through the built-in IMU sensor, reducing dependence on external systems, as well as reducing deployment and maintenance costs.

[0150] The indoor positioning method provided in the embodiment of the present application can be applied to a variety of scenarios, such as parking and finding a car in an indoor parking lot, indoor navigation, etc. The following is divided into three embodiments, and three application scenarios are described in detail.

[0151] Fig. 9 This is a flow chart of a method for finding a car in an indoor parking lot provided by an embodiment of the present application. Please refer to Fig. 9 , the method includes the following steps.

[0152] Step 901: The mobile device obtains first location information and second location information, where the first location information refers to the location information of the starting location of the mobile device, and the second location information refers to the location information of the parking position of the target vehicle in the indoor parking lot.

[0153] In the scenario of searching for a car in an indoor parking lot, the parking position of the target vehicle in the indoor parking lot is the destination position to be reached by the mobile device, that is, the second position information is the position information of the destination position to be reached by the mobile device.

[0154] In an embodiment of the present application, the first position information may be position information in a local coordinate system or in a world coordinate system. When the first position information is position information in a world coordinate system, the first position information can be obtained through a variety of implementation methods. As an example, the first position information is the outdoor positioning information obtained for the last time before the user enters the indoor parking lot. For example, the outdoor positioning information obtained by the user at the ground entrance of the indoor parking lot. As another example, the first position information is obtained by scanning a graphic code pasted in the indoor parking lot after the user enters the indoor parking lot, that is, a graphic code is pasted in the indoor parking lot, and the graphic code includes the position information of the current position in the world coordinate system. The mobile device can obtain the first position information by scanning the graphic code. Of course, the graphic code is not necessarily pasted in the indoor parking lot. Optionally, the graphic code can also be pasted outside the indoor parking lot, such as at the ground entrance of the indoor parking lot.

[0155] Generally, parking in a parking lot requires payment, so for the second example above, the mobile device can display the payment page by scanning the graphic code. After the user completes the payment through the payment page, the mobile device can display the navigation page, which displays the second location information. Of course, paying by scanning the graphic code and then displaying the navigation page is only one implementation method. In actual applications, there are other implementation methods.

[0156] Among them, the outdoor positioning information includes GPS location information or LBS location information, etc., and the graphic code includes a QR code, a bar code, etc., which is not limited in the embodiment of the present application.

[0157] Since the second position information refers to the position information of the parking position of the target vehicle in the indoor parking lot, it is necessary for the mobile device to obtain and store the second position information after the target vehicle is parked in the indoor parking lot. Among them, the second position information can be the position information in the local coordinate system, or it can be the position information in the world coordinate system. When the second position information is the position information in the world coordinate system, the mobile device can obtain the second position information in a variety of ways. For example, a graphic code is pasted on the parking space, and the graphic code includes the position information of the parking space in the world coordinate system. Since the target vehicle is parked in the parking space, the position information of the parking space in the world coordinate system can be considered as the position information of the parking position of the target vehicle in the indoor parking lot in the world coordinate system. In this way, the mobile device can obtain the second position information by scanning the graphic code pasted on the parking space. Of course, the mobile device can also obtain the second position information in other ways, such as obtaining the second position information through the above-mentioned indoor positioning method. This method is not described in detail here, and will be elaborated in detail later.

[0158] In some embodiments, a user's vehicle is parked in the indoor parking lot, and the mobile device stores the location information of the parking location. After finding the vehicle according to the location information of the parking location, the location information of the parking location may be cleared, or the location information of the parking location may not be cleared and the storage may continue. In this way, in one case, the mobile device only stores the location information of one parking location. In another case, the mobile device stores the location information of multiple parking locations. In this way, when the mobile device stores the location information of the parking location, it also needs to store the corresponding parking time. For the first case, the location information of a parking location stored by the mobile device is directly determined as the second location information. For the second case, the mobile device determines the location information of the multiple parking locations stored, which is the closest to the first location information and has the parking time closest to the current time, as the second location information.

[0159] In other embodiments, there may be multiple vehicles of the user parked in the indoor parking lot, so the mobile device needs to store the parking time and vehicle identification when storing the location information of the parking location. In this way, regardless of whether the mobile device clears the location information of the parking location after finding the vehicle, when the mobile device obtains the second location information, it needs to determine the vehicle identification of the target vehicle, and then determine the location information of the stored parking location whose parking time is closest to the current time and whose corresponding vehicle identification is the same as the vehicle identification of the target vehicle as the second location information.

[0160] Step 902: The mobile device determines a navigation path from a starting position to a parking position according to the first position information and the second position information.

[0161] There are multiple methods for a mobile device to determine a navigation path from a starting position to a parking position based on the first position information and the second position information. As an example, the mobile device obtains a map of an indoor parking lot where the mobile device is located, and calculates a navigation path from the starting position to the parking position based on the first position information and the second position information and the map of the indoor parking lot.

[0162] Among them, the map of the indoor parking lot can be a map under the local coordinate system. In this way, when the first position information and the second position information are position information under the local coordinate system, the mobile device directly calculates the navigation path from the starting position to the parking position according to the map of the indoor parking lot. However, when the first position information and the second position information are position information under the world coordinate system, the mobile device needs to convert the first position information and the second position information into position information under the local coordinate system, and then calculate the navigation path from the starting position to the parking position according to the map of the indoor parking lot.

[0163] Of course, the map of the indoor parking lot can also be a map in the world coordinate system. In this way, when the first position information and the second position information are position information in the local coordinate system, the mobile device needs to convert the first position information and the second position information into position information in the world coordinate system, and then calculate the navigation path from the starting position to the parking position according to the map of the indoor parking lot. However, when the first position information and the second position information are position information in the world coordinate system, the mobile device directly calculates the navigation path from the starting position to the parking position according to the map of the indoor parking lot.

[0164] The conversion method of the position information in the local coordinate system and the position information in the world coordinate system can refer to the relevant technology, which is not elaborated in detail in the embodiment of the present application. In addition, the map of the indoor parking lot can be a map drawn in advance for the indoor parking lot, or it can be a trajectory map generated by the cloud server after merging the walking trajectories of multiple users. The latter is elaborated in detail later and will not be elaborated here.

[0165] Step 903: The mobile device determines the current indoor position of the mobile device through the indoor positioning method according to the first position information.

[0166] Since the first position information can be position information in a local coordinate system or in a world coordinate system, the mobile device determines the position change of the mobile device through the indoor positioning method provided by the above embodiment, and then takes the position corresponding to the first position information as a reference point, adds the position change amount on the basis of the first position information, and obtains the current indoor position of the mobile device, which can be position information in a local coordinate system or in a world coordinate system.

[0167] Step 904: The mobile device displays the navigation path and the current indoor location of the mobile device in the navigation interface to search for the target vehicle in the indoor parking lot.

[0168] The mobile device displays the navigation path in the navigation interface by highlighting the navigation path, for example, by using a bold and colored line to display the navigation path in the navigation interface.

[0169] When the mobile device displays the navigation path in the navigation interface, it can also mark the position corresponding to the first position information and the position corresponding to the second position information, that is, mark the starting position and the parking position in the navigation interface, or mark the navigation starting point and the navigation end point in the navigation interface.

[0170] For example, Fig.10In the navigation interface shown, point C refers to the starting position of the mobile device, point P refers to the parking position, the dotted line between point C and point P refers to the determined navigation path, and point S refers to the current indoor position of the mobile device. In this way, the user can be guided to find the target vehicle in the indoor parking lot through the displayed information.

[0171] Optionally, when the user searches for the target vehicle in an indoor parking lot through the navigation path displayed in the navigation interface, the user may deviate from the navigation path. At this time, the navigation path may not be of much use to the user. In this case, the mobile device also needs to determine whether the real-time position of the mobile device in the world coordinate system deviates from the navigation path. If not, continue to display the navigation path. If it has deviated, the mobile device needs to redefine a navigation path according to the current indoor position and the second position information, and then display the redetermined navigation path in the navigation interface. In this way, by detecting in real time whether the user's position deviates from the navigation path, it is determined whether the navigation path needs to be updated, which is convenient for users to find the target vehicle in the indoor parking lot, and the efficiency of finding the target vehicle can also be improved.

[0172] Among them, there are multiple ways to determine whether the current indoor position of the mobile device deviates from the navigation path. For example, the mobile device determines the shortest distance between its current indoor position and the navigation path. If the shortest distance is greater than a distance threshold, then it is determined that the real-time position of the mobile device in the world coordinate system has deviated from the navigation path. Otherwise, it is determined that the real-time position of the mobile device in the world coordinate system has not deviated from the navigation path.

[0173] Optionally, when the mobile device displays the above information through the navigation interface to help the user find the target vehicle in the indoor parking lot, the mobile device can also determine whether the current indoor location of the mobile device has reached the parking location. If not, the mobile device continues to display the above information through the navigation interface. If it has arrived, the mobile device ends the navigation process. Alternatively, when the mobile device receives an instruction to end the search for the vehicle, the navigation process ends.

[0174] At this point, the process of finding a car in an indoor parking lot has been completed. However, based on the above description, the second location information refers to the location information of the parking position of the target vehicle in the indoor parking lot, and the second location information can be determined according to the indoor positioning method provided in the above embodiment. Next, through the following steps (1)-(3), a method for determining the second location information of a mobile device after parking in an indoor parking lot based on the above indoor positioning method is introduced.

[0175] (1) The mobile device determines the third position information, where the third position information refers to the position information of the parking position in the local coordinate system.

[0176] In some embodiments, the mobile device determines a second IMU signal through an IMU sensor. The second IMU signal is used as an input to the behavior recognition network to determine the current user behavior state. If the determined user behavior state is a driving state, the step of determining the second IMU signal through the IMU sensor is returned until the determined user behavior state is a walking state, and it is determined that the user has currently parked the target vehicle, that is, the user's parking event is currently captured. At this time, the location information determined by the above-mentioned indoor positioning method is the location information of the parking position.

[0177] Since the parking event occurs in an indoor environment, outdoor positioning information cannot be obtained, that is, the position information in the world coordinate system is not involved in the parking process. Therefore, the position information of the parking position determined here refers to the position information of the parking position in the local coordinate system.

[0178] For the above embodiment, the indoor parking lot parking and car finding application starts running after the mobile device is turned on. Through the method of timed query, the IMU sensor of the mobile device keeps reading and analyzing the data. Because the entire process runs in the background of the mobile device, the acquisition process of the IMU signal does not require the user to start the indoor parking lot parking and car finding application, that is, the user does not need to actively participate in the interaction to mark the parking location, the operation is relatively convenient, and even if the user forgets to mark the parking location, there will be no situation where the car cannot be found.

[0179] In other embodiments, after the user parks the target vehicle in the indoor parking lot, he can open the indoor parking lot parking and car finding application. In this way, the mobile device determines that the user's parking event is currently captured. At this time, the location information determined by the above-mentioned indoor positioning method is the location information of the parking location.

[0180] Of course, the mobile device can also capture the user's parking event in other ways to determine the third location information.

[0181] (2) The mobile device determines the user's walking trajectory based on the third location information.

[0182] Based on the description of the indoor positioning method above, when the user is walking, the mobile device will update its indoor position in real time, and then draw the user's walking trajectory. For the embodiment of the present application, after the mobile device determines the third position information, the third position information is used as the starting point, and the user's walking trajectory is determined while the indoor position is subsequently updated. Among them, the user's walking trajectory is also in the local coordinate system and is not aligned with the world coordinate system.

[0183] (3) When the outdoor positioning information is obtained, the second position information is determined according to the position information corresponding to the outdoor positioning information on the user's walking trajectory and the third position information.

[0184] Since the indoor parking lot cannot obtain outdoor positioning information, the above third location information and the user's walking trajectory are both information in the local coordinate system. However, when the mobile device can obtain outdoor positioning information, since the outdoor positioning information is the location information in the world coordinate system, the outdoor positioning information also corresponds to a location information in the local coordinate system on the user's walking trajectory. Therefore, according to this outdoor positioning information, the location information corresponding to the outdoor positioning information on the user's walking trajectory, and the third location information, the location information of the parking position in the world coordinate system can be inferred, that is, the second location information.

[0185] For example, Fig.11 As shown, point P0 is the parking position in the local coordinate system, that is, the position corresponding to the third position information. i Point is the position of the mobile device in the local coordinate system updated in real time with P0 as the reference point, and the displacement transformation matrix and orientation transformation matrix (or translation transformation matrix and rotation transformation matrix) in the local coordinate system are T loc and R loc .P exit Point is the location corresponding to the outdoor positioning information obtained by the mobile device. Assume that the displacement transformation matrix and orientation transformation matrix from the local coordinate system to the world coordinate system are T glob and R glob , then, we can infer that the parking position in the world coordinate system is T loc -1 R loc -1 T glob -1 R glob -1 P exit Among them, R loc -1 and T loc -1 is the inverse matrix of the orientation transformation matrix and the displacement transformation matrix in the local coordinate system, R glob -1 and T glob -1 It is the inverse matrix of the orientation transformation matrix and the displacement transformation matrix in the world coordinate system.

[0186] In summary, after the mobile device stores the location information of the parking location, in the process of finding a car, the mobile device can obtain the location information of the starting location, and then determine a navigation path. That is, according to the first location information and the second location information, a navigation path can be determined. The navigation path is then displayed in the navigation interface, so that the user can find the target vehicle in the indoor parking lot according to the information displayed on the navigation interface, solving the difficulty of not being able to obtain satellite signals in indoor places and thus being unable to locate. Moreover, the method provided in the embodiment of the present application only needs to be completed at the architecture layer of the mobile device, does not rely on external hardware, and has a low cost.

[0187] Fig.12 This is a flowchart of a method for navigating indoor places provided by an embodiment of the present application. Please refer to Fig.12 , the method includes the following steps.

[0188] Step 1201: The mobile device obtains first location information and second location information, where the first location information refers to the location information of the starting location of the mobile device, and the second location information refers to the location information of the destination location to be reached by the mobile device.

[0189] When the user enters the indoor place and is not very clear about the layout of the indoor place, or cannot obtain a map of the indoor place, and does not have or cannot access the positioning system of the indoor place, in this scenario, it may be difficult for the user to walk out of the indoor place, or the user may get lost, which will take a long time. Therefore, in order to ensure that the user can walk out of the indoor place, navigation is required in the indoor place.

[0190] In the scenario of indoor place navigation, the indoor place may be an indoor parking lot, or other places, such as a shopping mall, an office building, etc. The embodiments of the present application do not limit indoor places.

[0191] When the user wants to leave the indoor place, the user's current location can be used as the starting location of the mobile device, that is, the location information of the user's current location is used as the first location information, and the entrance through which the user enters the indoor place is used as the destination location to be reached by the mobile device. That is, the location information of the entrance through which the user enters the indoor place is used as the second location information.

[0192] In the embodiment of the present application, the first location information may be obtained by the mobile device through the above-mentioned indoor positioning method after the user enters the indoor place, and the first location information may be location information in the local coordinate system or in the world coordinate system. The second location information may be location information obtained by the mobile device when the user enters the indoor place, and the second location information may be location information in the local coordinate system or in the world coordinate system.

[0193] Step 1202: The mobile device selects a target user walking trajectory including first position information and second position information from one or more stored user walking trajectories, and determines the target user walking trajectory as a navigation path from the starting position to the destination position.

[0194] Based on the above-mentioned embodiments of the indoor positioning method, it can be known that when a user walks in an indoor place, the mobile device can not only determine the current indoor location, but also determine the user's walking trajectory. In this way, each time the user enters the indoor place, the mobile device will determine a user walking trajectory. Therefore, the mobile device will store one or more user walking trajectories.

[0195] In an embodiment of the present application, the target user's walking trajectory refers to the user's walking trajectory stored in the mobile device, including the first location information and the second location information. Moreover, the user's walking trajectory in the indoor place this time may include the first location information and the second location information, and the user's historical walking trajectory in the indoor place may also include the first location information and the second location information. In other words, among the user's walking trajectories stored in the mobile device, there may be more than one user walking trajectory including the first location information and the second location information. Therefore, the mobile device can determine the user's walking trajectory in the indoor place this time as the target user's walking trajectory, and can also determine the historical walking trajectory in the indoor place as the target user's walking trajectory. These two situations are explained below.

[0196] In the first implementation, the target user walking trajectory refers to the walking trajectory of the user in the indoor place this time. That is, when the user walks in the indoor place this time, the mobile device determines and stores the walking trajectory of the user in the indoor place, so that when the user wants to leave the indoor place, the mobile device can obtain the walking trajectory of the user in the indoor place this time, that is, the walking trajectory of the target user.

[0197] It should be noted that after each user walks out of an indoor place, the mobile device may clear the walking track in the indoor place this time, or may not clear the walking track in the indoor place this time and continue to store it. In this way, when the user wants to walk out of the indoor place, in one case, only one user walking track is stored in the mobile device, that is, the walking track in the indoor place this time, and in another case, multiple user walking tracks are stored in the mobile device, and the multiple user walking tracks may be the walking tracks of the user in the same indoor place at different times, or may be the walking tracks of the user in different indoor places at different times. In this way, when storing the user walking track, the mobile device also needs to store the corresponding entry time, that is, the time of entering the indoor place. For the first case, a user walking track stored in the mobile device is directly determined as the walking track of the user in the indoor place this time. For the second case, the mobile device determines the user walking track with the corresponding entry time closest to the current time among the stored multiple user walking tracks as the walking track of the user in the indoor place this time.

[0198] For the first implementation manner, the walking trajectory of the target user is a walking trajectory in a local coordinate system, or a walking trajectory in a world coordinate system.

[0199] For example, Fig.13 In the schematic diagram of the indoor place shown, point A is the starting point for the user to enter the indoor place, and by updating the real-time position of the mobile device, a user walking track B can be obtained. That is, the user walking track B is the walking track in the indoor place this time.

[0200] Before the mobile device determines the user's walking trajectory in the indoor place, it needs to determine whether the user is currently entering the indoor place. As an example, the mobile device determines whether the outdoor positioning information can be obtained at the current moment. If the outdoor positioning information cannot be obtained at the current moment and the outdoor positioning information can be obtained at the previous moment, the third IMU signal is determined by the IMU sensor. The third IMU signal is used as the input of the behavior recognition network to determine the current user behavior state. If the determined user behavior state is a walking state, it is determined that the user is currently entering the indoor place.

[0201] That is, whether the mobile device is currently entering the indoor place is determined by the change of the outdoor positioning information. Of course, the mobile device can also determine in other ways, which will not be listed one by one in the embodiment of the present application.

[0202] In the second implementation, the user walking trajectory refers to the walking trajectory of the user entering the indoor place in the past and including the first location information and the second location information. That is, when the user walks in the indoor place in the past, the mobile device determines and stores the walking trajectory of the user in the indoor place in the past, and the walking trajectory includes the first location information and the second location information. In this way, when the user wants to leave the indoor place, the mobile device can obtain the historical walking trajectory.

[0203] For the second implementation method described above, after each user walks out of an indoor place, the mobile device does not clear the walking track in the indoor place and continues to store it. In this way, when the user wants to walk out of the indoor place, multiple historical walking tracks are stored in the mobile device. At this time, the mobile device determines whether there is a historical walking track including the first location information and the second location information among the stored multiple historical walking tracks. If yes, the mobile device determines the historical walking track including the first location information and the second location information as the walking track of the target user.

[0204] It should be noted that step 1201 may be executed when the mobile device receives a navigation instruction. Of course, the execution may also be triggered by other methods, which is not limited in the embodiments of the present application.

[0205] In addition, in some scenarios, the mobile device may not be able to obtain the second location information. In this case, the mobile device can determine the target user's walking trajectory including the first location information from one or more stored user walking trajectories. In other words, as long as the user has walked in the indoor place, the user's walking trajectory must contain the location information of the entrance through which the user entered the indoor place. In this case, the target user's walking trajectory only needs to include the first location information, and the user can still walk out of the indoor place according to the target user's walking trajectory.

[0206] Step 1203: The mobile device determines the current indoor position of the mobile device through the indoor positioning method according to the first position information.

[0207] Since the first location information may be location information in a local coordinate system or in a world coordinate system, the current indoor location of the mobile device determined by the first location information may be a location in a local coordinate system or in a world coordinate system.

[0208] Step 1204: The mobile device displays the navigation path and the current indoor location of the mobile device on the navigation interface to navigate in the indoor place.

[0209] The mobile device displays the navigation path in the navigation interface by highlighting the navigation path, for example, by using a bold and colored line to display the navigation path in the navigation interface.

[0210] When the mobile device displays the navigation path in the navigation interface, it can also mark the navigation start point and the navigation end point in the navigation interface, wherein the navigation start point is the current location, and the navigation end point is the entrance of the user into the indoor place.

[0211] Based on the above description, the walking trajectory of the target user may be a walking trajectory in the local coordinate system or a walking trajectory in the world coordinate system. That is, the navigation path may be a path in the local coordinate system or a path in the world coordinate system. When the navigation path is a path in the local coordinate system, the mobile device displays the navigation path and the real-time position of the mobile device in the local coordinate system in the navigation interface, so as to navigate the user out of the indoor place. When the navigation path is a path in the world coordinate system, the mobile device displays the navigation path and the real-time position of the mobile device in the world coordinate system in the navigation interface, so as to navigate the user out of the indoor place.

[0212] Optionally, when the mobile device displays the above information through the navigation interface to help the user walk out of the indoor place, the mobile device can also determine whether its real-time location has reached the end of the navigation path. If not, the mobile device continues to display the above information through the navigation interface. If it has arrived, the mobile device ends the navigation process. Alternatively, when the mobile device receives an instruction to end navigation, the navigation process ends.

[0213] Based on the description of steps 1201 to 1204, it can be seen that in one case, the mobile device does not need to know the user's walking track in the world coordinate system and the real-time position of the mobile device, and only needs to record the user's walking track in the local coordinate system and the real-time position of the mobile device to navigate the user out of the indoor place. In another case, the mobile device needs to know the user's walking track in the world coordinate system and the real-time position of the mobile device to navigate the user out of the indoor place.

[0214] In the second case, the mobile device can not only navigate the user out of the indoor place, but also obtain some ticket information, discount information, etc. of the indoor place according to the real-time position in the world coordinate system. That is, the identification of the current indoor place is determined through the real-time position in the world coordinate system, and then some ticket information, discount information, etc. of the indoor place is obtained according to the identification of the current indoor place.

[0215] It should be noted that for indoor places such as museums, there are no different stores in these indoor places, so the mobile device obtains the ticket information, discount information, etc. of the indoor place according to the outdoor positioning information obtained last time before entering the indoor place, or according to any real-time position of the mobile device in the indoor place in the world coordinate system. However, for indoor places such as shopping malls, these indoor places include multiple different stores, and different stores may correspond to different ticket information, discounts, etc. Therefore, the mobile device can obtain it according to the outdoor positioning information obtained last time before entering the indoor place, or according to any real-time position of the mobile device in the indoor place in the world coordinate system. However, obtaining the ticket information, discount information, etc. of the store at the current location according to any real-time position of the mobile device in the indoor place in the world coordinate system is more targeted and may be more helpful to users.

[0216] In summary, when a user wants to walk out of an indoor place, but is not very clear about the layout of the indoor place, it is easy to get lost, which will take a long time. Therefore, the embodiment of the present application provides a method for navigating in an indoor place, that is, when a user wants to walk out of an indoor place, a mobile device obtains a target user's walking track, and then displays the target user's walking track and the real-time location of the mobile device in a navigation interface, thereby navigating the user out of the indoor place and preventing the user from getting lost.

[0217] Fig.14 This is a flowchart of a method for navigating indoor places provided by an embodiment of the present application. Please refer to Fig.14 , the method includes the following steps.

[0218] Step 1401: The mobile device obtains the outdoor positioning information obtained last time before entering the indoor place.

[0219] Step 1402: The mobile device obtains a track map of the indoor place from the cloud server according to the outdoor positioning information. The track map of the indoor place is generated by the cloud server after merging the walking tracks of multiple users.

[0220] The cloud server stores different track maps for different indoor places, and the outdoor positioning information obtained last time before entering the indoor place can represent the location of the indoor place, so the mobile device can obtain the track map of the indoor place from the cloud server according to the outdoor positioning information obtained last time before entering the indoor place. Specifically, the mobile device sends the outdoor positioning information to the cloud server, and the cloud server obtains the track map of the indoor place according to the outdoor positioning information, and sends the track map of the indoor place to the mobile device.

[0221] In some embodiments, the cloud server stores a mapping relationship between the location range of indoor places and the track map. After the mobile device obtains the outdoor positioning information, the mobile device sends the outdoor positioning information to the cloud server. The cloud server determines an indoor place closest to the outdoor positioning information according to the location range of the stored indoor place, and then sends the track map of the determined indoor place to the mobile device.

[0222] It should be noted that the trajectory map of the indoor place can be a map in the local coordinate system or a map in the world coordinate system. Since the trajectory map of the indoor place is generated by the cloud server after fusion of the walking trajectories of multiple users, if the trajectory map of the indoor place is a map in the local coordinate system, then the walking trajectories of the multiple users are required to be walking trajectories in the local coordinate system, and the local coordinate systems of the mobile devices that collect the walking trajectories of the multiple users are the same, so that trajectory fusion can be performed. If the trajectory map of the indoor place is a map in the world coordinate system, then the walking trajectories of the multiple users are required to be walking trajectories in the world coordinate system.

[0223] Step 1403: The mobile device obtains the first location information, and determines the current indoor location of the mobile device through the above-mentioned indoor positioning method according to the first location information.

[0224] In an embodiment of the present application, the first position information may be position information in a local coordinate system or in a world coordinate system. In this way, the current indoor position of the mobile device determined by the mobile device through the above-mentioned indoor positioning method based on the first position information may be position information in a local coordinate system or in a world coordinate system. When the first position information is position information in a local coordinate system, the first position information may be position information of the starting position determined by the above-mentioned indoor positioning method. When the first position information is position information in a world coordinate system, the first position information may be the outdoor positioning information obtained in step 1401.

[0225] Since the outdoor positioning information is the location information in the world coordinate system, the mobile device uses the location corresponding to the outdoor positioning information as a reference point, determines the location change of the mobile device through the indoor positioning method provided by the above embodiment, and then adds the location change to the outdoor positioning information to obtain the real-time location of the mobile device in the world coordinate system.

[0226] Step 1404: The mobile device displays the current indoor location of the mobile device and a track map of the indoor place in the navigation interface to perform indoor navigation.

[0227] In some embodiments, since the trajectory map includes the paths that can be walked in the indoor place, the current indoor position of the mobile device can represent the current indoor position of the user. Therefore, after the mobile device displays the current indoor position of the mobile device and the trajectory map of the indoor place in the navigation interface, the user can not only see his or her current position, but also see the feasible paths around him or her. In this way, the user walks in the indoor place according to the trajectory map, and then realizes navigation in the indoor place.

[0228] In other embodiments, the user may determine a destination location. In this way, after the mobile device displays the current indoor location of the mobile device and the track map of the indoor place in the navigation interface, it also includes: obtaining the location information of the destination location, and determining the navigation path through the track map of the indoor place according to the current indoor location of the mobile device and the location information of the destination location. The navigation path is displayed in the navigation interface for navigation. That is, the mobile device determines which paths can be walked and which paths cannot be walked through the track map of the indoor place, thereby determining a navigation path according to the track map, and the user can walk to the destination location according to the navigation path.

[0229] Among them, the mobile device includes multiple ways to determine the navigation path according to the trajectory map, for example, determining the navigation path by the shortest distance method, which is not limited in the embodiments of the present application. In addition, when the trajectory map of the indoor place and the current indoor position of the mobile device are both information in the local coordinate system, the location information of the destination location obtained by the mobile device also needs to be information in the local coordinate system, or the location information of the destination location in the world coordinate system needs to be converted into the location information of the destination location in the local coordinate system. When the trajectory map of the indoor place and the current indoor position of the mobile device are both information in the world coordinate system, the location information of the destination location obtained by the mobile device also needs to be information in the world coordinate system, or the location information of the destination location in the local coordinate system needs to be converted into the location information of the destination location in the world coordinate system.

[0230] Optionally, the track map of the indoor place is also marked with key landmark information, and the key landmark information is used to indicate the key landmarks in the indoor place. For example, the key landmarks in the indoor place are entrances, exits, escalators, elevators, etc. In this way, after the track map is displayed in the navigation interface, the user can also learn about the key landmarks of the indoor place in a timely manner.

[0231] At this point, the process of navigating through the trajectory map of the indoor place has been completed. However, based on the above description, the trajectory map of the indoor place is generated by the cloud server after merging the trajectory maps of multiple users, and during this navigation process, the user may find other feasible paths. Therefore, the mobile device can determine the user's walking trajectory during this navigation process, and then send the user's walking trajectory to the cloud server, which updates the trajectory map of the indoor place. This can ensure the real-time update of the trajectory map of the indoor place and reduce the maintenance cost of the trajectory map.

[0232] Based on the above description, the trajectory map may or may not be marked with key landmark information. Therefore, the following will be divided into two cases for introduction.

[0233] In the first case, when the track map does not mark the key landmark information, after the mobile device determines the current indoor location of the mobile device through the above step 1403, the current user walking track is determined according to the current indoor location of the mobile device. Afterwards, the mobile device sends the current user walking track to the cloud server. In this way, the cloud server receives the user walking track currently in the indoor place sent by the mobile device, and generates a track heat map according to the user walking track sent by the mobile device and the user walking track currently stored in the indoor place. According to the track heat map, the track map of the indoor place is regenerated to update the track map.

[0234] The manner in which the mobile device determines the current user walking trajectory according to its current indoor position is described in the above embodiment and will not be repeated here.

[0235] Since the user's walking trajectory is composed of multiple trajectory points, and each trajectory point on the user's trajectory is in the same coordinate system, the cloud server draws the user's walking trajectory sent by the mobile device and the user's walking trajectory currently stored in the indoor place on a map. This map includes multiple trajectory points, and each trajectory point also corresponds to a heat value. The heat value is used to indicate the number of user trajectories including the trajectory point. This map is called a trajectory heat map. In this way, the cloud server selects multiple candidate trajectory points from the multiple trajectory points included in the trajectory heat map according to the heat value corresponding to each trajectory point in the trajectory heat map, and regenerates the trajectory map of the indoor place according to the multiple candidate trajectory points.

[0236] In some embodiments, the cloud server selects multiple trajectory points whose heat values ​​are greater than a heat threshold from the multiple trajectory points included in the trajectory heat map as candidate trajectory points. Of course, the cloud server can also select candidate trajectory points from the multiple trajectory points included in the trajectory heat map in other ways. After the cloud server selects the candidate trajectory points, it connects the candidate trajectory points to form a trajectory map of the indoor place.

[0237] It should be noted that the cloud server stores the mapping relationship between the location range of indoor places and the user's walking trajectory. When the mobile device sends the user's walking trajectory currently in the indoor place to the cloud server, it is also necessary to send the outdoor positioning information last obtained before the mobile device entered the indoor place to the cloud server. In this way, the cloud server can determine the indoor place closest to the outdoor positioning information according to the location range of the stored indoor place, and then obtain the user's walking trajectory corresponding to this indoor place, and then merge the trajectory with the user's walking trajectory currently sent by the mobile device to regenerate the trajectory map of the indoor place.

[0238] For example, Fig.15 As shown in FIG. 1 , the cloud server generates a trajectory heat map based on the user walking trajectory sent by the mobile device and the currently stored user walking trajectory in the indoor place. Fig.15 The left picture shows that the cloud server then generates a trajectory map of the indoor place based on the trajectory heat map. Fig.15 Right picture.

[0239] In the second case, when the key landmark information is marked on the trajectory map, after the mobile device determines the current indoor position of the mobile device through the above step 1403, the current user walking trajectory is determined according to the current indoor position of the mobile device. After that, the mobile device determines the change of the outdoor positioning signal corresponding to each trajectory point on the current user walking trajectory and the change of the indoor elevation, and sends the current user walking trajectory and the change of the outdoor positioning signal and the change of the indoor elevation of each trajectory point on the current user walking trajectory to the cloud server. In this way, the cloud server receives the user walking trajectory currently in the indoor place and the change of the outdoor positioning signal and the change of the indoor elevation of each trajectory point on the user walking trajectory sent by the mobile device. According to the user walking trajectory sent by the mobile device and the user walking trajectory currently stored in the indoor place, a trajectory heat map is generated, and the change of the outdoor positioning signal and the change of the indoor elevation of each trajectory point are marked on the trajectory heat map. According to the trajectory heat map marked with the change of the outdoor positioning signal and the change of the indoor elevation of each trajectory point, the trajectory map of the indoor place is regenerated to achieve the update of the trajectory map.

[0240] The change of the outdoor positioning signal refers to whether the mobile device can detect the outdoor positioning signal at the current track point. If the outdoor positioning signal cannot be detected at the current track point, but the outdoor positioning signal can be detected at the previous track point, it indicates that the user is currently entering the room from the outdoors, that is, the current track point may be the entrance to the indoor place. On the contrary, if the outdoor positioning signal can be detected at the current track point, but the outdoor positioning signal cannot be detected at the previous track point, it indicates that the user is currently walking from the indoor place to the outdoors, that is, the current track point may be the exit of the indoor place.

[0241] The change of indoor elevation refers to whether there is a change in elevation of the mobile device at the current trajectory point. If there is a change in elevation of the mobile device at the current trajectory point, then the current trajectory point may be an escalator or a straight ladder. Specifically, if there is a change in elevation at the current trajectory point, and there are also changes in elevation at the next multiple consecutive trajectory points, then it indicates that the current trajectory point may be an escalator. If there is a change in elevation at the current trajectory point, and there is no change in elevation at the next multiple consecutive trajectory points, then it indicates that the current trajectory point may be a straight ladder.

[0242] Among them, there are multiple ways for the mobile device to determine the change in indoor elevation corresponding to each trajectory point on the current user's walking trajectory. As an example, when the mobile device includes a barometer, the mobile device determines whether there is a change in elevation through the change in air pressure detected by the barometer. That is, when the air pressure detected by the barometer at the current trajectory point is less than the air pressure detected at the previous trajectory point, it is determined that there is a change in elevation at the current trajectory point, otherwise, it is determined that there is no change in elevation at the current trajectory point. As another example, whether there is a change in elevation is determined through a behavior recognition network. That is, the IMU signal of the current trajectory point is used as the input of the behavior recognition network to determine whether there is a change in elevation at the current trajectory point.

[0243] Since the user's walking trajectory is composed of multiple trajectory points, and each trajectory point on the user's trajectory is in the same coordinate system, the cloud server draws the user's walking trajectory sent by the mobile device and the user's walking trajectory currently stored in the indoor place on a map. This map includes multiple trajectory points, and each trajectory point also corresponds to a heat value. The heat value is used to indicate the number of user trajectories including the trajectory point. This map is called a trajectory heat map. In this way, the cloud server selects multiple candidate trajectory points from the multiple trajectory points included in the trajectory heat map according to the heat value corresponding to each trajectory point in the trajectory heat map, regenerates the trajectory map of the indoor place according to the multiple candidate trajectory points, and marks key landmark information on the trajectory map according to the changes in the outdoor positioning signals and the changes in the indoor elevation at the multiple candidate trajectory points.

[0244] In some embodiments, the cloud server selects multiple trajectory points whose heat values ​​are greater than a heat threshold from the multiple trajectory points included in the trajectory heat map as candidate trajectory points. Of course, the cloud server can also select candidate trajectory points from the multiple trajectory points included in the trajectory heat map in other ways. After the cloud server selects the candidate trajectory points, it connects the candidate trajectory points to form a trajectory map of the indoor place.

[0245] It should be noted that the cloud server stores the mapping relationship between the location range of indoor places and the user's walking trajectory, and each trajectory point on the user's walking trajectory in the mapping relationship also corresponds to the change of the outdoor positioning signal and the change of the indoor elevation. When the mobile device sends the user's walking trajectory currently in the indoor place to the cloud server, it is also necessary to send the outdoor positioning information obtained by the mobile device before entering the indoor place to the cloud server. In this way, the cloud server can determine the indoor place closest to the outdoor positioning information according to the location range of the stored indoor place, and then obtain the user's walking trajectory corresponding to this indoor place, and then merge the trajectory with the user's walking trajectory currently sent by the mobile device to regenerate the trajectory map of the indoor place. Afterwards, according to the changes in the outdoor positioning signal and the changes in the indoor elevation corresponding to the trajectory points on each user's trajectory, the key landmark information is marked on the trajectory map.

[0246] In summary, the cloud server generates a trajectory map of the indoor place by fusing the walking trajectories of multiple users, that is, by crowdsourcing the walking trajectories of users to collect map path information of the indoor place, which can reduce the cost of building the trajectory map of the indoor place. In addition, when other users enter the indoor place later, the mobile device obtains the trajectory map of the indoor place from the cloud server, so that in the process of navigating according to the trajectory map of the indoor place, the walking trajectory of the user can be determined, and then the cloud server updates the path changes of the indoor place in real time, reducing the maintenance cost of the trajectory map.

[0247] Fig.16 This is a schematic diagram of the structure of an indoor positioning device provided by an embodiment of the present application. The indoor positioning device can be implemented by software, hardware or a combination of both to become part or all of a mobile device. Please refer to Fig.16 , the indoor positioning device comprises:

[0248] The first determining module 1601 is used to execute the above Figure 2 The operation of step 201 in the embodiment;

[0249] The second determining module 1602 is used to execute the above Figure 2 The operation of step 202 in the embodiment;

[0250] The third determining module 1603 is used to execute the above Figure 2 The operation of step 203 in the embodiment;

[0251] The fourth determining module 1604 is used to execute the above Figure 2 The operation of step 204 in the embodiment.

[0252] Optionally, the second determining module 1602 is specifically configured to:

[0253] According to the step detection method, the first IMU signal is divided to obtain a plurality of signal segments, each of the plurality of signal segments includes an IMU signal of a step;

[0254] Each signal segment in the multiple signal segments is used as an input of a state recognition network to determine the usage state of the mobile device when each signal segment in the multiple signal segments corresponds to a step.

[0255] Optionally, the third determining module 1603 is specifically configured to:

[0256] A signal segment is selected from the multiple signal segments as a reference signal segment, and the following operations are performed on the reference signal segment until the following operations are performed on each of the multiple signal segments:

[0257] Extracting features from the reference signal segment to obtain a feature vector corresponding to the reference signal segment;

[0258] Determine, according to the first corresponding relationship, a location identification network corresponding to the usage state of the mobile device during a footstep corresponding to the reference signal segment, and use the determined location identification network as a reference location identification network;

[0259] The feature vector corresponding to the reference signal segment is used as the input of the reference position recognition network to determine the position change of a step corresponding to the reference signal segment.

[0260] Optionally, the device further comprises:

[0261] A first acquisition module, configured to acquire a plurality of sample signal segments and a usage status corresponding to each of the plurality of sample signal segments;

[0262] The first training module is used to use multiple sample signal segments as input of the initial state recognition network, use the usage state corresponding to each sample signal segment in the multiple sample signal segments as the output of the initial state recognition network, train the initial state recognition network, and obtain a state recognition network.

[0263] Optionally, the device further includes a second training module, and the second training module is specifically used for:

[0264] A location recognition network is selected from the location recognition networks included in the first corresponding relationship as the first location recognition network, and the first location recognition network is obtained by training through the following operations until each location recognition network included in the first corresponding relationship is obtained by training:

[0265] Obtaining a plurality of sample feature vectors and a position change amount corresponding to each of the plurality of sample feature vectors, wherein the plurality of sample feature vectors refer to feature vectors corresponding to an IMU signal segment obtained by a mobile device in a first state of use, wherein the first state refers to a use state corresponding to a first location identification network;

[0266] Taking multiple sample feature vectors as input of the initial position recognition network, taking the position change amount corresponding to each sample feature vector in the multiple sample feature vectors as output of the initial position recognition network, the initial position recognition network is trained to obtain a first position recognition network.

[0267] Optionally, the second determining module 1602 is specifically configured to:

[0268] The first IMU signal is used as an input of a state recognition network to determine the usage state of the mobile device in multiple time windows, where the multiple time windows are used to divide the first IMU signal.

[0269] Optionally, the third determining module 1603 is specifically configured to:

[0270] Select a time window from the multiple time windows as a reference time window, and perform the following operations on the reference time window until the following operations are performed on each time window in the multiple time windows:

[0271] Determine the location identification network corresponding to the usage state of the mobile device in the reference time window according to the first corresponding relationship, and use the determined location identification network as the reference location identification network;

[0272] The signal segment in the first IMU signal within the reference time window is used as the input of the reference position recognition network to determine the position change corresponding to the reference time window.

[0273] Optionally, the device further comprises:

[0274] A second acquisition module is used to acquire a plurality of sample IMU signals and a usage status of each of the plurality of sample IMU signals in a plurality of time windows;

[0275] The third training module is used to use multiple sample IMU signals as the input of the initial state recognition network, and use the usage status of each sample IMU signal in multiple time windows as the output of the initial state recognition network to train the initial state recognition network to obtain a state recognition network.

[0276] Optionally, the device further includes a fourth training module, and the fourth training module is specifically used for:

[0277] A location recognition network is selected from the location recognition networks included in the first corresponding relationship as the first location recognition network, and the first location recognition network is obtained by training through the following operations until each location recognition network included in the first corresponding relationship is obtained by training:

[0278] Acquire a plurality of sample signal segments and a position change amount corresponding to each of the plurality of sample signal segments, wherein the plurality of sample signal segments refer to IMU signal segments acquired by a mobile device in a first state of use, and the first state refers to a use state corresponding to a first location identification network;

[0279] Taking multiple sample signal segments as input of an initial position recognition network, taking the position change amount corresponding to each sample signal segment in the multiple sample signal segments as output of the initial position recognition network, the initial position recognition network is trained to obtain a first position recognition network.

[0280] Optionally, the device further comprises:

[0281] a fifth determination module, configured to use the first IMU signal as an input of a behavior recognition network to determine a current user behavior state;

[0282] The trigger module is used to trigger the first determination module to perform an operation of using the first IMU signal as an input of the state recognition network to determine the use state of the mobile device if the determined user behavior state is a walking state.

[0283] Optionally, the fourth determining module 1604 includes:

[0284] A third acquisition module is used to acquire first location information, where the first location information refers to location information of a starting location of the mobile device;

[0285] The superposition module is used to add the first position information and the position change amount to obtain the current indoor position of the mobile device.

[0286] Optionally, the device further comprises:

[0287] A fourth acquisition module is used to acquire second location information, where the second location information refers to location information of a destination location to be reached by the mobile device;

[0288] A sixth determination module, configured to determine a navigation path from a starting position to a destination position according to the first position information and the second position information;

[0289] The first display module is used to display the navigation path and the current indoor position of the mobile device in the navigation interface to perform indoor navigation.

[0290] Optionally, the device further comprises:

[0291] A fifth acquisition module, used to acquire outdoor positioning information last acquired before the mobile device enters the indoor place;

[0292] A sixth acquisition module is used to acquire a trajectory map of an indoor place from a cloud server according to the outdoor positioning information, wherein the trajectory map of the indoor place is generated by the cloud server after fusion of the walking trajectories of multiple users;

[0293] The second display module is used to display the current indoor location of the mobile device and a track map of the indoor place in the navigation interface to perform indoor navigation.

[0294] Optionally, the device further comprises:

[0295] A seventh determination module, used to determine the current user walking trajectory according to the indoor location of the mobile device;

[0296] The first sending module is used to send the current user walking trajectory to the cloud server, so that the cloud server can update the trajectory map of the indoor place.

[0297] Optionally, the trajectory map of the indoor place is also marked with key landmark information, and the key landmark information is used to indicate key landmarks in the indoor place;

[0298] The device also includes:

[0299] An eighth determination module is used to determine the change of the outdoor positioning signal and the change of the indoor elevation corresponding to each track point on the current user walking track;

[0300] The second sending module is used to send the changes in the outdoor positioning signal and the indoor elevation of each track point on the current user's walking track to the cloud server, so that the cloud server can update the key landmark information on the track map of the indoor place.

[0301] In summary, after the mobile device determines its own usage status, it can select the corresponding location identification network to determine the position change according to the determined usage status. That is, in different usage states, different location identification networks are selected to determine the position change. The user does not need to maintain a fixed holding posture of the mobile device, the robustness is strong, and the positioning accuracy can be guaranteed. In addition, the mobile device can achieve indoor positioning through the IMU sensor, reducing dependence on external systems, as well as reducing deployment and maintenance costs. Moreover, the mobile device can determine a navigation path based on the first location information and the second location information, and then display the navigation path in the navigation interface, so that the user can navigate indoors according to the information displayed on the navigation interface, solving the difficulty of not being able to obtain satellite signals in indoor places and thus being unable to locate.

[0302] Furthermore, the cloud server generates a trajectory map of the indoor place by fusing the walking trajectories of multiple users, that is, by crowdsourcing the walking trajectories of users to collect map path information of the indoor place, which can reduce the cost of building the trajectory map of the indoor place. In addition, when other users enter the indoor place later, the mobile device obtains the trajectory map of the indoor place from the cloud server, so that in the process of navigating according to the trajectory map of the indoor place, the walking trajectory of the user can be determined, and then the cloud server can update the path changes of the indoor place in real time, reducing the maintenance cost of the trajectory map.

[0303] It should be noted that: the indoor positioning device provided in the above embodiment only uses the division of the above functional modules as an example when performing indoor positioning. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the indoor positioning device provided in the above embodiment and the indoor positioning method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0304] Fig.17 This is a schematic diagram of the structure of an indoor navigation device provided by an embodiment of the present application. The indoor navigation device can be implemented as part or all of a cloud server by software, hardware, or a combination of both. Please refer to Fig.17 , the indoor navigation device comprises:

[0305] The first receiving module 1701 is used to receive outdoor positioning information acquired by a mobile device;

[0306] An acquisition module 1702 is used to acquire a trajectory map of an indoor place corresponding to the outdoor positioning information. The trajectory map of the indoor place is generated by a cloud server after merging the walking trajectories of multiple users;

[0307] The sending module 1703 is used to send the track map of the indoor place to the mobile device, so that the mobile device navigates in the indoor place according to the track map of the indoor place.

[0308] Optionally, the device further comprises:

[0309] The second receiving module is used to receive the walking trajectory of the user currently in the indoor place sent by the mobile device;

[0310] A first generating module, configured to generate a trajectory heat map according to the user walking trajectory sent by the mobile device and the user walking trajectory currently stored in the indoor place;

[0311] The second generating module is used to regenerate the trajectory map of the indoor place according to the trajectory heat map to update the trajectory map.

[0312] Optionally, the device further comprises:

[0313] The third receiving module is used to receive the change of the outdoor positioning signal of each track point on the user's walking track currently in the indoor place and the change of the indoor elevation sent by the mobile device;

[0314] The annotation module is used to annotate the changes in the outdoor positioning signal and the indoor elevation of each trajectory point on the tracked heat map.

[0315] Optionally, the second generating module is specifically used for:

[0316] Selecting multiple candidate trajectory points from multiple trajectory points included in the trajectory heat map;

[0317] Regenerating a trajectory map of the indoor place according to the multiple candidate trajectory points;

[0318] According to the changes in the outdoor positioning signals and the changes in the indoor elevations of the multiple candidate trajectory points, key landmark information is marked on the trajectory map, and the key landmark information is used to indicate key landmarks in indoor places.

[0319] In summary, the cloud server generates a trajectory map of the indoor place by fusing the walking trajectories of multiple users, that is, by crowdsourcing the walking trajectories of users to collect map path information of the indoor place, which can reduce the cost of building the trajectory map of the indoor place. In addition, when other users enter the indoor place later, the mobile device obtains the trajectory map of the indoor place from the cloud server, so that in the process of navigating according to the trajectory map of the indoor place, the walking trajectory of the user can be determined, and then the cloud server updates the path changes of the indoor place in real time, reducing the maintenance cost of the trajectory map.

[0320] It should be noted that: the indoor navigation device provided in the above embodiment only uses the division of the above functional modules as an example when navigating indoors. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the indoor navigation device provided in the above embodiment and the indoor navigation method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0321] Please refer to Fig.18 , Fig.18 1801, a communication bus 1802, a memory 1803, and at least one communication interface 1804.

[0322] Processor 1801 may be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, or may be one or more integrated circuits for implementing the solution of the present application, such as an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0323] The communication bus 1802 is used to transmit information between the above components. The communication bus 1802 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0324] The memory 1803 may be a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), an optical disc (including a compact disc read-only memory (CD-ROM), a compressed optical disc, a laser disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1803 may exist independently and be connected to the processor 1801 via the communication bus 1802. The memory 1803 may also be integrated with the processor 1801.

[0325] The communication interface 1804 uses any transceiver-like device for communicating with other devices or communication networks. The communication interface 1804 includes a wired communication interface and may also include a wireless communication interface. Among them, the wired communication interface may be, for example, an Ethernet interface. The Ethernet interface may be an optical interface, an electrical interface, or a combination thereof. The wireless communication interface may be a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof, etc.

[0326] In a specific implementation, as an embodiment, the processor 1801 may include one or more CPUs, such as Fig.18 CPU0 and CPU1 are shown in the figure.

[0327] In a specific implementation, as an embodiment, a computer device may include multiple processors, such as Fig.18 1801 and processor 1805 shown in FIG. Each of these processors may be a single-core processor or a multi-core processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0328] In a specific implementation, as an embodiment, the computer device may further include an output device 1806 and an input device 1807. The output device 1806 communicates with the processor 1801 and may display information in a variety of ways. For example, the output device 1806 may be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device 1807 communicates with the processor 1801 and may receive user input in a variety of ways. For example, the input device 1807 may be a mouse, a keyboard, a touch screen device, or a sensor device.

[0329] In some embodiments, the memory 1803 is used to store the program code 1810 for executing the solution of the present application, and the processor 1801 can execute the program code 1810 stored in the memory 1803. The program code 1810 may include one or more software modules, and the computer device can implement the method provided in the above embodiment through the processor 1801 and the program code 1810 in the memory 1803.

[0330] The mobile device described in the embodiment of the present application may be a terminal. Fig.19 , is a schematic diagram of a terminal in an embodiment of the present application;

[0331] The terminal of the present application includes a sensor unit 1110 , a computing unit 1120 , a storage unit 1140 , and an interaction unit 1130 .

[0332] The sensor unit 1110 generally includes a visual sensor (such as a camera) for acquiring 2D image information of a scene; an inertial sensor (IMU) for acquiring motion information of the terminal, such as linear acceleration, angular velocity, etc.; a depth sensor / laser sensor (optional) for acquiring depth information of the scene;

[0333] The computing unit 1120 usually includes a CPU, a GPU, a cache, a register, etc., and is mainly used to run the operating system and process the various algorithm modules involved in this application, such as the SLAM system, bone detection, face recognition, etc.;

[0334] The storage unit 1140 mainly includes memory and external storage, and is mainly used for reading and writing user local and temporary data;

[0335] The interaction unit 1130 mainly includes a display screen, a touch panel, a speaker, a microphone, etc., and is mainly used to interact with the user, obtain input, and implement the presentation algorithm effect, etc.

[0336] For ease of understanding, the structure of a terminal 100 provided in an embodiment of the present application is described below by way of example. Fig. 20 , Fig. 20 It is a structural diagram of a terminal provided in an embodiment of the present application.

[0337] like Fig. 20 As shown, the terminal 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, an inertial measurement unit IMU sensor 180N (not shown in the figure), etc.

[0338] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal 100. In other embodiments of the present application, the terminal 100 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0339] The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural-network processing unit (neural-network processing unit, NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors. The processor 110 can execute a computer program to implement any indoor positioning method in the embodiments of the present application. Or an indoor navigation method.

[0340] The controller may be the nerve center and command center of the terminal 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0341] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0342] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (IC) interface, an inter-integrated circuit sound (IS) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0343] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the terminal 100. In other embodiments of the present application, the terminal 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0344] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from the wired charger through the USB interface 130 .

[0345] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160.

[0346] The wireless communication function of the terminal 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0347] In some feasible implementations, the terminal 100 may use a wireless communication function to communicate with other devices. For example, the terminal 100 may communicate with a second electronic device, the terminal 100 establishes a screen projection connection with the second electronic device, and the terminal 100 outputs the projection data to the second electronic device. The projection data output by the terminal 100 may be audio and video data.

[0348] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0349] The mobile communication module 150 can provide solutions for wireless communications including 1G / 3G / 4G / 5G applied to the terminal 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 2. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0350] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0351] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the terminal 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 1, modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through antenna 2.

[0352] In some embodiments, the antenna 1 of the terminal 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0353] The terminal 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0354] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0355] In some feasible implementations, the display screen 194 may be used to display various interfaces output by the system of the terminal 100. For various interfaces output by the terminal 100, reference may be made to the relevant descriptions of the subsequent embodiments.

[0356] The terminal 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor.

[0357] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0358] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0359] Digital signal processors are used to process digital signals. In addition to processing digital image signals, they can also process other digital signals.

[0360] Video codecs are used to compress or decompress digital videos. Terminal 100 may support one or more video codecs. Thus, terminal 100 may play or record videos in various coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG1, MPEG3, MPEG4, etc.

[0361] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the terminal 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0362] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0363] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the terminal 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as the indoor positioning method in the embodiment of the present application, etc.), etc. The data storage area may store data created during the use of the terminal 100 (such as audio data, phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0364] The terminal 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor, etc. For example, music playing, recording, etc. In some feasible implementations, the audio module 170 can be used to play the sound corresponding to the video. For example, when the display screen 194 displays the video playing screen, the audio module 170 outputs the sound of the video playing.

[0365] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals.

[0366] The speaker 170A, also called a "horn", is used to convert audio electrical signals into sound signals.

[0367] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals.

[0368] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals.

[0369] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0370] The pressure sensor 180A is used to sense pressure signals and can convert pressure signals into electrical signals. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194. The gyroscope sensor 180B can be used to determine the motion posture of the terminal 100. The air pressure sensor 180C is used to measure air pressure.

[0371] The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal 100 in all directions (including three axes or six axes). When the terminal 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the terminal posture and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0372] The distance sensor 180F is used to measure the distance.

[0373] The ambient light sensor 180L is used to sense the brightness of ambient light.

[0374] The fingerprint sensor 180H is used to collect fingerprints.

[0375] The temperature sensor 180J is used to detect the temperature.

[0376] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be arranged on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be arranged on the surface of the terminal 100, which is different from the position of the display screen 194.

[0377] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The terminal 100 may receive key input and generate key signal input related to user settings and function control of the terminal 100.

[0378] Motor 191 can generate vibration prompts.

[0379] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0380] The SIM card interface 195 is used to connect a SIM card.

[0381] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)), etc. It is worth noting that the computer-readable storage medium mentioned in the present application may be a non-volatile storage medium, in other words, a non-transient storage medium.

[0382] It should be understood that the "multiple" mentioned herein refers to two or more. In the description of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish between the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not limit them to be necessarily different.

[0383] The above-mentioned embodiments are provided for the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An indoor positioning method, characterized in that: Applied in a mobile device, the method comprises: Determine a first IMU signal by an inertial measurement unit (IMU) sensor; Using the first IMU signal as an input of a state recognition network to determine a usage state of the mobile device; Determine a location identification network corresponding to the use state according to a first corresponding relationship, and determine the location change amount through the location identification network, wherein the first corresponding relationship includes a corresponding relationship between the use state and the location identification network; The current indoor position is determined according to the position change.

2. The method according to claim 1, characterized in that The using the first IMU signal as an input of a state recognition network to determine the use state of the mobile device includes: According to the method of footstep detection, the first IMU signal is divided to obtain a plurality of signal segments, each of the plurality of signal segments includes an IMU signal of a footstep; Each signal segment in the multiple signal segments is used as an input of the state recognition network to determine the usage state of the mobile device when a step corresponding to each signal segment in the multiple signal segments occurs.

3. The method according to claim 2, characterized in that Determining the location recognition network corresponding to the use state according to the first corresponding relationship, and determining the location change amount through the location recognition network, includes: Select one signal segment from the multiple signal segments as a reference signal segment, and perform the following operations on the reference signal segment until the following operations are performed on each signal segment in the multiple signal segments: Performing feature extraction on the reference signal segment to obtain a feature vector corresponding to the reference signal segment; Determine, according to the first corresponding relationship, a location identification network corresponding to the usage state of the mobile device at a step corresponding to the reference signal segment, and use the determined location identification network as a reference location identification network; The feature vector corresponding to the reference signal segment is used as the input of the reference position recognition network to determine the position change of a step corresponding to the reference signal segment.

4. The method according to claim 2 or 3, characterized in that The method further comprises: Acquire a plurality of sample signal segments and a usage status corresponding to each of the plurality of sample signal segments; The multiple sample signal segments are used as inputs of an initial state recognition network, and the usage state corresponding to each sample signal segment in the multiple sample signal segments is used as output of the initial state recognition network. The initial state recognition network is trained to obtain the state recognition network.

5. The method according to claim 2 or 3, characterized in that: The method further comprises: A location recognition network is selected from the location recognition networks included in the first corresponding relationship as the first location recognition network, and the first location recognition network is obtained by training through the following operations until each location recognition network included in the first corresponding relationship is obtained by training: Acquire multiple sample feature vectors and a position change amount corresponding to each of the multiple sample feature vectors, wherein the multiple sample feature vectors refer to feature vectors corresponding to an IMU signal segment acquired by a mobile device in a first state, wherein the first state refers to a usage state corresponding to the first location identification network; The multiple sample feature vectors are used as inputs of an initial position recognition network, and the position change corresponding to each sample feature vector in the multiple sample feature vectors is used as output of the initial position recognition network. The initial position recognition network is trained to obtain the first position recognition network.

6. The method according to claim 1, characterized in that The using the first IMU signal as an input of a state recognition network to determine the use state of the mobile device includes: The first IMU signal is used as an input of the state recognition network to determine the usage state of the mobile device in multiple time windows, and the multiple time windows are used to divide the first IMU signal.

7. The method according to claim 6, characterized in that Determining the location recognition network corresponding to the use state according to the first corresponding relationship, and determining the location change amount through the location recognition network, includes: Select one time window from the multiple time windows as a reference time window, and perform the following operations on the reference time window until the following operations are performed on each of the multiple time windows: Determine, according to the first corresponding relationship, a location identification network corresponding to the usage state of the mobile device in the reference time window, and use the determined location identification network as a reference location identification network; The signal segment in the first IMU signal that is within the reference time window is used as the input of the reference position recognition network to determine the position change corresponding to the reference time window.

8. The method according to claim 6 or 7, characterized in that The method further comprises: Acquire multiple sample IMU signals and the usage status of each sample IMU signal in the multiple sample IMU signals in multiple time windows; The multiple sample IMU signals are used as inputs of an initial state recognition network, and the usage status of each sample IMU signal in the multiple time windows is used as output of the initial state recognition network. The initial state recognition network is trained to obtain the state recognition network.

9. The method according to claim 6 or 7, characterized in that The method further comprises: A location recognition network is selected from the location recognition networks included in the first corresponding relationship as the first location recognition network, and the first location recognition network is obtained by training through the following operations until each location recognition network included in the first corresponding relationship is obtained by training: Acquire a plurality of sample signal segments and a position change amount corresponding to each of the plurality of sample signal segments, wherein the plurality of sample signal segments refer to IMU signal segments acquired by a mobile device in a first usage state, wherein the first state refers to a usage state corresponding to the first location identification network; The multiple sample signal segments are used as inputs of an initial position recognition network, and the position change corresponding to each sample signal segment in the multiple sample signal segments is used as output of the initial position recognition network. The initial position recognition network is trained to obtain the first position recognition network.

10. The method according to claim 1, characterized in that The method further comprises: Using the first IMU signal as input to a behavior recognition network to determine a current user behavior state; If the determined user behavior state is a walking state, the step of using the first IMU signal as an input of a state recognition network to determine the use state of the mobile device is performed.

11. The method according to claim 1, characterized in that Determining the current indoor position according to the position change includes: Acquire first location information, where the first location information refers to location information of a starting location of the mobile device; The first location information is added to the location change to obtain the current indoor location of the mobile device.

12. The method according to claim 11, characterized in that The method further comprises: Acquire second location information, where the second location information refers to location information of a destination location to be reached by the mobile device; Determining a navigation path from the starting position to the destination position according to the first position information and the second position information; The navigation path and the current indoor location of the mobile device are displayed in a navigation interface to perform indoor navigation.

13. The method according to claim 11, characterized in that The method further comprises: Obtaining outdoor positioning information last obtained before the mobile device enters the indoor place; According to the outdoor positioning information, a trajectory map of the indoor place is obtained from a cloud server, where the trajectory map of the indoor place is generated by the cloud server after fusion of the walking trajectories of multiple users; The current indoor location of the mobile device and a track map of the indoor place are displayed in the navigation interface to perform indoor navigation.

14. The method according to claim 13, characterized in that The method further comprises: Determine a current user walking trajectory according to the indoor location of the mobile device; The current user walking trajectory is sent to the cloud server so that the cloud server updates the trajectory map of the indoor place.

15. The method according to claim 14, characterized in that The trajectory map of the indoor place is also marked with key landmark information, and the key landmark information is used to indicate the key landmarks in the indoor place; The method further comprises: Determine the change of the outdoor positioning signal and the change of the indoor elevation corresponding to each track point on the current user walking track; Send the cloud server the change of the outdoor positioning signal and the change of the indoor elevation of each track point on the current user walking track, so that the cloud server can update the key landmark information on the track map of the indoor place.

16. An indoor navigation method, characterized in that: Applied in a cloud server, the method includes: Receiving outdoor positioning information obtained by a mobile device; Acquire a trajectory map of an indoor place corresponding to the outdoor positioning information, wherein the trajectory map of the indoor place is generated by the cloud server after merging the walking trajectories of multiple users; Sending a trajectory map of the indoor place to the mobile device so that the mobile device navigates in the indoor place according to the current indoor position and the trajectory map of the indoor place, wherein the current indoor position of the mobile device is determined according to the indoor positioning method described in any one of claims 1-15.

17. The method according to claim 16, characterized in that The method further comprises: receiving a walking track of a user currently in the indoor place sent by the mobile device; Generate a trajectory heat map according to the user walking trajectory sent by the mobile device and the user walking trajectory currently stored in the indoor place; According to the trajectory heat map, the trajectory map of the indoor place is regenerated to achieve the update of the trajectory map.

18. The method according to claim 17, characterized in that The method further comprises: receiving a change in an outdoor positioning signal and a change in an indoor elevation of each track point on a user's walking track currently in the indoor place, sent by the mobile device; The change of the outdoor positioning signal and the change of the indoor elevation of each trajectory point are marked on the trajectory heat map.

19. The method according to claim 18, characterized in that The step of regenerating the trajectory map of the indoor place according to the trajectory heat map comprises: Selecting a plurality of candidate trajectory points from a plurality of trajectory points included in the trajectory heat map; Regenerating a trajectory map of the indoor place according to the plurality of candidate trajectory points; According to the changes in the outdoor positioning signals and the changes in the indoor elevations of the multiple candidate trajectory points, key landmark information is marked on the trajectory map, and the key landmark information is used to indicate the key landmarks in the indoor place.

20. An indoor positioning device, characterized in that: Applied in a mobile device, the device comprises: A first determination module, configured to determine a first IMU signal through an inertial measurement unit IMU sensor; A second determination module, configured to use the first IMU signal as an input of a state recognition network to determine a usage state of the mobile device; A third determination module is used to determine a location identification network corresponding to the use state according to a first corresponding relationship, and determine the location change amount through the location identification network, wherein the first corresponding relationship includes a corresponding relationship between the use state and the location identification network; The fourth determination module is used to determine the current indoor position according to the position change.

21. The device according to claim 20, characterized in that The second determining module is specifically used for: According to the method of footstep detection, the first IMU signal is divided to obtain a plurality of signal segments, each of the plurality of signal segments includes an IMU signal of a footstep; Each signal segment in the multiple signal segments is used as an input of the state recognition network to determine the usage state of the mobile device when a step corresponding to each signal segment in the multiple signal segments occurs.

22. The device according to claim 21, characterized in that The third determination module is specifically used for: Select one signal segment from the multiple signal segments as a reference signal segment, and perform the following operations on the reference signal segment until the following operations are performed on each signal segment in the multiple signal segments: Performing feature extraction on the reference signal segment to obtain a feature vector corresponding to the reference signal segment; Determine, according to the first corresponding relationship, a location identification network corresponding to the usage state of the mobile device at a step corresponding to the reference signal segment, and use the determined location identification network as a reference location identification network; The feature vector corresponding to the reference signal segment is used as the input of the reference position recognition network to determine the position change of a step corresponding to the reference signal segment.

23. The device according to claim 21 or 22, characterized in that The device also includes: A first acquisition module, configured to acquire a plurality of sample signal segments and a usage status corresponding to each of the plurality of sample signal segments; The first training module is used to use the multiple sample signal segments as input of the initial state recognition network, use the usage state corresponding to each sample signal segment in the multiple sample signal segments as output of the initial state recognition network, train the initial state recognition network, and obtain the state recognition network.

24. The device according to claim 21 or 22, characterized in that The device further includes a second training module, wherein the second training module is specifically configured to: A location recognition network is selected from the location recognition networks included in the first corresponding relationship as the first location recognition network, and the first location recognition network is obtained by training through the following operations until each location recognition network included in the first corresponding relationship is obtained by training: Acquire multiple sample feature vectors and a position change amount corresponding to each of the multiple sample feature vectors, wherein the multiple sample feature vectors refer to feature vectors corresponding to an IMU signal segment acquired by a mobile device in a first state, wherein the first state refers to a usage state corresponding to the first location identification network; The multiple sample feature vectors are used as inputs of an initial position recognition network, and the position change corresponding to each sample feature vector in the multiple sample feature vectors is used as output of the initial position recognition network. The initial position recognition network is trained to obtain the first position recognition network.

25. The device according to claim 20, characterized in that The second determining module is specifically used for: The first IMU signal is used as an input of the state recognition network to determine the usage state of the mobile device in multiple time windows, and the multiple time windows are used to divide the first IMU signal.

26. The device according to claim 25, characterized in that The third determination module is specifically used for: Select one time window from the multiple time windows as a reference time window, and perform the following operations on the reference time window until the following operations are performed on each of the multiple time windows: Determine, according to the first corresponding relationship, a location identification network corresponding to the usage state of the mobile device in the reference time window, and use the determined location identification network as a reference location identification network; The signal segment in the first IMU signal that is within the reference time window is used as the input of the reference position recognition network to determine the position change corresponding to the reference time window.

27. The device according to claim 25 or 26, characterized in that The device also includes: A second acquisition module is used to acquire a plurality of sample IMU signals and a usage status of each of the plurality of sample IMU signals in a plurality of time windows; The third training module is used to use the multiple sample IMU signals as the input of the initial state recognition network, and use the usage state of each sample IMU signal in the multiple time windows as the output of the initial state recognition network to train the initial state recognition network to obtain the state recognition network.

28. The device according to claim 25 or 26, characterized in that The device further includes a fourth training module, wherein the fourth training module is specifically configured to: A location recognition network is selected from the location recognition networks included in the first corresponding relationship as the first location recognition network, and the first location recognition network is obtained by training through the following operations until each location recognition network included in the first corresponding relationship is obtained by training: Acquire a plurality of sample signal segments and a position change amount corresponding to each of the plurality of sample signal segments, wherein the plurality of sample signal segments refer to IMU signal segments acquired by a mobile device in a first usage state, wherein the first state refers to a usage state corresponding to the first location identification network; The multiple sample signal segments are used as inputs of an initial position recognition network, and the position change corresponding to each sample signal segment in the multiple sample signal segments is used as output of the initial position recognition network. The initial position recognition network is trained to obtain the first position recognition network.

29. The device according to claim 20, characterized in that The device also includes: a fifth determination module, configured to use the first IMU signal as an input of a behavior recognition network to determine a current user behavior state; The trigger module is used to trigger the first determination module to perform an operation of using the first IMU signal as an input of a state recognition network to determine the use state of the mobile device if the determined user behavior state is a walking state.

30. The device according to claim 20, characterized in that The fourth determination module comprises: A third acquisition module is used to acquire first location information, where the first location information refers to location information of a starting location of the mobile device; The superposition module is used to add the first position information to the position change to obtain the current indoor position of the mobile device.

31. The device according to claim 30, characterized in that The device also includes: A fourth acquisition module is used to acquire second location information, where the second location information refers to location information of a destination location to be reached by the mobile device; a sixth determining module, configured to determine a navigation path from the starting position to the destination position according to the first position information and the second position information; The first display module is used to display the navigation path and the current indoor position of the mobile device in a navigation interface to perform indoor navigation.

32. The device according to claim 30, characterized in that The device also includes: A fifth acquisition module, used to acquire outdoor positioning information last acquired before the mobile device enters the indoor place; A sixth acquisition module, configured to acquire a trajectory map of the indoor place from a cloud server according to the outdoor positioning information, wherein the trajectory map of the indoor place is generated by the cloud server after fusion of the walking trajectories of multiple users; The second display module is used to display the current indoor location of the mobile device and a track map of the indoor place in the navigation interface to perform indoor navigation.

33. The device according to claim 32, characterized in that The device also includes: A seventh determination module, used to determine a current user walking trajectory according to the indoor location of the mobile device; The first sending module is used to send the current user walking trajectory to the cloud server, so that the cloud server updates the trajectory map of the indoor place.

34. The device according to claim 33, characterized in that The trajectory map of the indoor place is also marked with key landmark information, and the key landmark information is used to indicate the key landmarks in the indoor place; The device also includes: An eighth determination module is used to determine the change of the outdoor positioning signal and the change of the indoor elevation corresponding to each track point on the current user walking track; The second sending module is used to send the changes in the outdoor positioning signal and the changes in the indoor elevation of each track point on the current user's walking track to the cloud server, so that the cloud server can update the key landmark information on the track map of the indoor place.

35. An indoor navigation device, characterized in that: Applied in a cloud server, the device comprises: A first receiving module, used to receive outdoor positioning information acquired by a mobile device; An acquisition module, used to acquire a trajectory map of an indoor place corresponding to the outdoor positioning information, wherein the trajectory map of the indoor place is generated by the cloud server after merging the walking trajectories of multiple users; A sending module is used to send a trajectory map of the indoor place to the mobile device so that the mobile device can navigate in the indoor place according to the current indoor position and the trajectory map of the indoor place, and the current indoor position of the mobile device is determined according to the indoor positioning method described in any one of claims 1-15.

36. The device according to claim 35, characterized in that The device also includes: A second receiving module is used to receive a walking track of a user currently in the indoor place sent by the mobile device; A first generating module, configured to generate a trajectory heat map according to the user walking trajectory sent by the mobile device and the currently stored user walking trajectory in the indoor place; The second generating module is used to regenerate the trajectory map of the indoor place according to the trajectory heat map to achieve the update of the trajectory map.

37. The device according to claim 36, characterized in that The device also includes: A third receiving module is used to receive changes in the outdoor positioning signal and the indoor elevation of each track point on the user's walking track currently in the indoor place sent by the mobile device; The marking module is used to mark the change of the outdoor positioning signal and the change of the indoor elevation of each trajectory point on the trajectory heat map.

38. The device according to claim 37, characterized in that The second generating module is specifically used for: Selecting a plurality of candidate trajectory points from a plurality of trajectory points included in the trajectory heat map; Regenerating a trajectory map of the indoor place according to the plurality of candidate trajectory points; According to the changes in the outdoor positioning signals and the changes in the indoor elevations of the multiple candidate trajectory points, key landmark information is marked on the trajectory map, and the key landmark information is used to indicate the key landmarks in the indoor place.

39. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program to implement the steps of the method according to any one of claims 1 to 19.

40. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 19 are implemented.