A geomagnetic indoor positioning system based on a dilated convolutional neural network

Abstract

The present invention discloses a geomagnetic indoor positioning system based on a dilated convolutional neural network. According to the characteristic that the geomagnetic fingerprint is uniquely associated with the spatial position coordinates, the present invention designs a fingerprint segmentation length evaluation algorithm, and calculates the fingerprint segmentation length based on the richness of the geomagnetic fingerprint in the indoor space. And an overlapping segmentation method is adopted to segment the fingerprint sequence, and each segmented fingerprint is mapped to a position point, which increases the density of the positioning points and makes more full use of the fingerprint features. In order to more accurately match and identify the geomagnetic fingerprint, after converting the geomagnetic fingerprint sequence into a curve difference matrix, the present invention designs a fingerprint classification model based on a dilated convolutional neural network to distinguish each segmented fingerprint. The dilated convolution in the classification model can obtain more effective fingerprint information, and has a high fingerprint matching accuracy after being trained with a large amount of fingerprint data. This geomagnetic indoor positioning system improves the positioning accuracy at a relatively low positioning cost.

Description

Technical Field

[0001] The present invention belongs to the field of indoor positioning, and relates to a geomagnetic indoor positioning system based on deep learning and an implementation method thereof. After learning a large amount of fingerprint data through a neural network model, the system predicts position points, realizing indoor positioning services with excellent accuracy. Background Art

[0002] In recent years, with the development of mobile Internet technology, applications based on location-based service (LBS) have emerged continuously, penetrating into all aspects of people's clothing, food, housing and transportation, greatly improving the convenience of people's daily life. As the technical support of LBS, the mature Global Positioning System (GPS) can achieve high positioning accuracy in outdoor environments. However, due to signal blockage and multipath effects in indoor environments, the positioning effect cannot meet people's needs for indoor location-based service (ILBS). Therefore, indoor positioning technology has developed rapidly, and various signals have been used in indoor positioning technology, such as Wi-Fi (Wireless-Fidelity), Bluetooth, Radio Frequency Identification (RFID), infrared, Ultra Wide Band (UWB), geomagnetism, visible light, ultrasonic, Frequency Modulation (FM), Inertial Navigation System (INS), and images.

[0003] When considering the positioning accuracy of an indoor positioning solution, the deployment and maintenance costs of the solution also need to be considered. Technologies such as RFID, ultrasonic, and UWB have relatively high positioning accuracy, all of which can achieve positioning accuracy at the meter level. However, the deployment costs and energy consumption of the devices are relatively high, resulting in difficulties in implementing the positioning solution and making it difficult to popularize and apply. Sensing technologies such as Wi-Fi and inertial navigation have relatively low costs, but their positioning accuracy is not satisfactory. For example, Wi-Fi is affected by signal fluctuations and multipath effects and cannot effectively locate in complex environments. In contrast, the positioning technology based on the geomagnetic field not only has stable signals and high positioning accuracy but also does not require the deployment of additional devices, gradually becoming the focus of research.

[0004] Existing geomagnetic solutions also have many deficiencies, mainly including: 1) Fingerprint acquisition and fingerprint database construction are time-consuming and laborious: Existing research requires staff to conduct surveys at the positioning site, manually collect fingerprint data in the space, mark the positions, and establish the mapping relationship between fingerprints and positions. 2) Limited positioning accuracy: In most existing geomagnetic positioning solutions, the fingerprint matching accuracy is not high, and there are also problems with sequence alignment during matching. A relatively high positioning walking cost is required to achieve an ideal positioning accuracy, and the positioning effect is not ideal at short distances. 3) Long positioning latency: Traditional fingerprint similarity matching algorithms need to traverse all fingerprints in the fingerprint database, and the algorithm takes a long time; the particle filter algorithm has a large computational amount, and the particles may converge slowly, resulting in a certain latency during positioning. Summary of the Invention

[0005] The object of the present invention is to solve the problems of high fingerprint acquisition cost, limited positioning accuracy, high positioning walking cost, and high positioning latency in existing geomagnetic indoor positioning algorithms. By using deep learning methods to train a fingerprint classification model and establish the mapping relationship between position coordinate points and geomagnetic fingerprints, a low-cost, efficient, and accurate geomagnetic indoor positioning system is realized.

[0006] The core technical idea of the present invention is to achieve positioning by matching and recognizing geomagnetic fingerprints through a deep learning model. The system includes two stages: offline training and online positioning. 1) Offline training stage: Collect geomagnetic fingerprint information in the space, and after processing, train a geomagnetic fingerprint classification model and establish the mapping relationship between geomagnetic fingerprints and positions. 2) Online positioning stage: After receiving the user's positioning request, calculate the positioning result through the fingerprint classification model and related algorithms and feedback it to the user.

[0007] The core technologies for the present invention to solve problems include a fingerprint sequence segmentation length evaluation algorithm, an overlapping fingerprint segmentation method, a curve interpolation matrix, and a geomagnetic fingerprint classification model.

[0008] (1) Fingerprint sequence segmentation length evaluation algorithm. The present invention realizes positioning by solving the classification and matching problem of geomagnetic fingerprint sequences. A fingerprint sequence corresponds to a segment of the main path, and the corresponding positioning point is the coordinate point of the end of the main path segment. Although the longer the length of the geomagnetic fingerprint sequence, the more information it expresses and the higher its uniqueness within a limited space range, the relationship between length and uniqueness is not a linear growth relationship. In order to seek a balance between positioning accuracy (uniqueness) and positioning walking cost (fingerprint sequence length), the present invention proposes a fingerprint sequence segmentation length evaluation algorithm.

[0009] (2) Overlapping fingerprint segmentation method. Since there is a problem of misalignment at the beginning and end of the sequence during actual fingerprint sequence matching, a relatively large fingerprint sequence segmentation length will lead to a high alignment cost and a large error. Therefore, based on the above-mentioned segmented geomagnetic fingerprint acquisition algorithm, the present invention proposes an overlapping segmented geomagnetic fingerprint acquisition algorithm, with a certain overlap between subsequences.

[0010] (3) Curve interpolation matrix. The geomagnetic fingerprint sequence is a one-dimensional spatio-temporal sequence, and there are device differences. The absolute values of the fingerprint sequence cannot be directly used for classification tasks, and only the relative relationships and high-dimensional spatial features existing between the sequence values can be sought. To better describe the relative relationships between the sequence values, the present invention proposes a curve difference matrix, using the differences between sampling points to replace the absolute values to describe the shape of the curve formed by the fingerprint sequence, so that the neural network can better perceive the high-dimensional spatial features of the data.

[0011] (4) Geomagnetic fingerprint classification model. A basic convolutional neural network consists of a convolutional layer, a pooling layer, and a fully connected layer. The convolutional layer uses convolutional kernels to extract features from images, the pooling layer plays a role in downsampling, and finally the fully connected layer combines the information obtained by different neurons, which is the feature of the entire image extracted by the neural network. The curve difference matrix of the geomagnetic fingerprint sequence is different from a general image. Each value of the matrix corresponds to the difference between two sampling points on the fingerprint sequence, which is neither irrelevant nor redundant, but all effective information that contributes to the model output. Therefore, during the feature extraction process of the curve difference matrix, neither max pooling nor average pooling is suitable for extensive use as they will reduce or blur information. On the other hand, pooling can increase the size of the receptive field, enabling each convolutional output to contain information in a larger range. A convolutional neural network with no pooling or very little pooling will lose a certain receptive field and cannot perceive the features of a larger range of the fingerprint sequence. The feature extraction of the geomagnetic fingerprint sequence requires both avoiding information loss and trying to ensure the acquisition of information at different range scales, and dilated convolution just meets the above requirements. Therefore, the present invention proposes a geomagnetic fingerprint classification model based on a dilated convolutional neural network. Description of the Drawings

[0012] Figure 1 It is the structure diagram of the geomagnetic positioning system based on the dilated convolutional neural network of the present invention;

[0013] Figure 2 It is the main path division diagram of the floor related to the present invention;

[0014] Figure 3 It is an example diagram of the fingerprint sequence segmentation length evaluation algorithm related to the present invention;

[0015] Figure 4 It is the overlapping geomagnetic fingerprint sequence segmentation diagram related to the present invention;

[0016] Figure 5 This is the fingerprint classification model diagram related to the present invention;

[0017] Figure 6 This is the fingerprint matching misalignment correction diagram related to the present invention; Detailed implementation manners

[0018] The following further describes the present invention in conjunction with the attached drawings. The system structure is as Figure 1 shown, mainly including four parts: geomagnetic fingerprint acquisition, geomagnetic fingerprint processing, geomagnetic fingerprint classification model training, and real-time position estimation. It should be noted here that the description of these implementation manners is only used to help understand the present invention and does not constitute a limitation to the present invention.

[0019] 1. Geomagnetic fingerprint acquisition

[0020] The real-time geomagnetic field values (geo x , geo y , geo z ) obtained by the geomagnetic field sensor built in the mobile phone are based on the mobile phone coordinate system, and the values will change with the change of the mobile phone attitude, which is not referenceable. To avoid this problem, it is necessary to convert the geomagnetic field values based on the mobile phone coordinate system into geomagnetic field values (g x , g y , g z ) based on the earth coordinate system through a conversion matrix. Also, because the geomagnetic values g x in the east-west direction of the earth coordinate system are almost 0 and are not used as geomagnetic fingerprints, the geomagnetic fingerprint for a single sampling can be finally expressed as G=(g y , g z , g xyz ), where g xyz is the modulus of the three-axis fingerprint vector. However, in a certain space, G cannot uniquely associate with a position coordinate point. Only by aggregating multiple sampling values into a geomagnetic fingerprint sequence S={G1, G2,..., G n} can the unique mapping relationship between the geomagnetic fingerprint and the position point P be ensured. To reduce the workload of collecting geomagnetic fingerprint data, the present invention uses an intelligent vehicle equipped with a mobile phone for collection. The main path of a certain floor is divided, and it can be divided into four thickened lines as Figure 2 shown.

[0021] 2. Geomagnetic fingerprint processing

[0022] An example diagram of the fingerprint sequence segment length evaluation algorithm is as Figure 3 shown. This algorithm first divides all geomagnetic fingerprint sequences according to the direction. Assuming the segment length (segment sampling number) is SSN, SSN traverses within the set maximum and minimum values. For the n-time fingerprint sampling sequence {S1, S2,..., S of the total path in the same direction,n} As a sample, in each loop, the DTW is used to horizontally compare the similarity between n segmented segments in m component segments among various samples. Through experimental statistics, the similarity between the same segments satisfies the chi-square distribution. Calculate the average value μ and the standard deviation σ of all the similarity values between the segments, and set the discrimination threshold θ of the current segment as θ = μ + σ. Then traverse each complete sequence S i , after each sequence is segmented with the current SSN length and labeled, U is the label set of all segments of the sequence. Then, the DTW is used to calculate the similarity between every two of the m segments of the same sequence. If the result is not greater than the previously calculated discrimination threshold, it is considered that the two segments are similar, and the labels of the two segments are removed from the label set. Finally, calculate and find the SSN value when the discrimination rate converges and approaches 1, which is the optimal segmented length of the geomagnetic fingerprint sequence.

[0023] Based on the above segmented geomagnetic fingerprint acquisition algorithm, an overlapping segmented geomagnetic fingerprint acquisition algorithm is adopted, with a certain overlap between subsequences. For the convenience of calculation, an overlap ratio of 50% is selected for the experiment, as Figure 4 shown. On the basis of increasing the sequence segmentation accuracy, the features of the sequence can be utilized more fully.

[0024] The present invention proposes a curve difference matrix, which replaces the absolute values with the differences between sampling points to describe the shape of the curve constituting the fingerprint sequence, so that the neural network can better perceive the high-dimensional space features of the data. Compared with the normalization process, the discrete degree features between sequences are retained. Assume that the segmented geomagnetic fingerprint sequence is S xyz = {g1, g2, …, g n}, and the curve difference matrix D is as follows:

[0025]

[0026] After conversion, a two-dimensional image with richer content is formed. Different types of curves correspond to different curve difference matrix diagrams, and the image categories are distinct. Both the horizontal and vertical coordinates of the matrix represent the sampling point coordinates of the sequence. The differences between sampling points have high-dimensional features in space. Therefore, it meets the conditions for classification using image processing methods, and a convolutional neural network can be used for feature extraction to achieve the classification of geomagnetic fingerprint sequences.

[0027] 3. Training of the geomagnetic fingerprint classification model

[0028] The present invention proposes a segmented fingerprint classification model based on a dilated convolutional neural network, and the model structure is as Figure 5As shown in the figure, where Conv represents the convolutional layer, f×f×c represents the convolutional kernel size of f and the number of channels of c, and d refers to the dilation rate of the dilated convolution. Pool represents the pooling layer. In this paper, a maximum pooling layer with a size of 2×2 is used, and the stride s is 2. FC is a fully connected neural network layer with different numbers of features. The second-to-last FC has auxiliary data input, which is concatenated with the high-dimensional features of the fingerprint sequence extracted previously and then jointly trained. And Dropout is added after the first FC to prevent overfitting and improve the generalization performance of the model. A batch normalization layer (Batch Normalization, BN) is immediately followed after all Convs. Except for the last layer, the rectified linear unit (ReLU) is used as the activation function. Since it is a multi-classification task, the Softmax function is used as the activation function for the last FC. The dilation rate of the dilated convolution selects a sawtooth structure of 1, 2, 5, 1, 2 to meet the HDC design rules, which can not only sense features of different scales but also avoid the occurrence of the grid effect. The input of the model is the curve difference matrix converted from the geomagnetic fingerprint sequences of three dimensions respectively. After passing through the convolutional layer and the fully connected layer, the direction parameter O is input as auxiliary data to jointly train the model. The Keras deep learning framework is selected as the model framework, TensorFlow is used in the backend, the Adam optimization algorithm is selected for the optimization method of the neural network, and the categorical_crossentropy loss function is selected.

[0029] 4. Real-time Location Estimation

[0030] Since the essence of geomagnetic fingerprint sequence matching is the feature matching between the fingerprint sampling sequence and a segmented fingerprint sequence on the path, although the overlapping segmentation method has been adopted in this paper, there is still the problem of sequence misalignment. Assume that the path length corresponding to the fingerprint segmentation length SSN is FSL. When the walking starting point is in the middle of two positioning points, the maximum misalignment length on the path is FSL / 4, and the maximum misalignment length of the fingerprint sequence is SSN / 4. The misalignment during the matching of the segmented fingerprint sequences will cause a certain deviation in the matching result. To solve this problem, this section analyzes the problem and proposes a position correction method.

[0031] First, analyze the worst-case scenario: when the pedestrian's real-time fingerprint sampling sequence is in the middle of two segmented fingerprint sequences divided on the path (such as Figure 6As shown by the sample line segment, the maximum misalignment length appears. In this case, the fingerprint sampling sequence only contains partial subsequences of two segmented fingerprint sequences. If this sampling sequence is fed into the classification model, the resulting result will inevitably show a large deviation. To ensure the effectiveness of sequence matching, it is first necessary to ensure that the fingerprint sampling sequence contains at least the complete segmented fingerprint sequence at a certain positioning point of the path, so that the subsequence of the fingerprint sampling sequence can be matched to this segmented fingerprint sequence, thereby achieving successful positioning. Therefore, this paper solves the problem of insufficient fingerprint information acquisition by increasing the fingerprint sequence sampling length. The increased sampling length is FSL / 4 (as Figure 6 shown by the dashed line segment), and the sampling length L sample (minimum walking distance) requirement is:

[0032]

[0033] After ensuring that the fingerprint sampling sequence contains the segmented fingerprint sequence, the next step is to align to this segmented fingerprint and perform correct matching. This paper uses the sliding window algorithm for searching. The window size is FSL. At the same time, considering the positioning accuracy and calculation cost, the sliding step size slide is set to FSL / 10.

[0034] The final real-time positioning process is as follows: First, perform sliding average filtering preprocessing on the geomagnetic fingerprint sequence and acceleration sequence collected by the pedestrian. Then calculate the transfer threshold of the FSM algorithm. Divide the acceleration sequence at each step, and then calculate the single-step length and walking distance. After obtaining the walking distance, judge whether it meets the shortest distance requirement obtained by the segmented length evaluation algorithm. If it does not meet, exit the algorithm and prompt to continue walking. If it meets, the fingerprint sequence will be segmented by the sliding window algorithm. The segmented segments are converted into a curve difference matrix and then spliced with the direction parameters, and then enter the geomagnetic fingerprint classification model in turn and return the predicted positioning points and corresponding confidence levels of each segmented fingerprint sequence. Finally, obtain the positioning point and index value with the maximum confidence level, and calculate the final positioning point coordinates through position correction.

[0035] An example of the user scenario of the present invention is as follows:

[0036] In some large buildings, there are many narrow and complex walking paths. Pedestrians often get lost in them and need to obtain location-based services in real time, that is, know the location of themselves or other people or objects in the indoor space. In the above scenario, the present invention has achieved good results, and compared with the existing methods, the equipment deployment cost and positioning walking cost are lower, and the positioning delay is also lower.

Claims

1. A geomagnetic indoor positioning system based on a dilated convolutional neural network, characterized in that : The system includes a mobile terminal and a server. After the user initiates a positioning request through the mobile terminal, the server returns the positioning result; The mobile terminal is divided into a fingerprint acquisition module and a real-time positioning module. Before using the positioning system, it is necessary to collect the geomagnetic fingerprint sequence of the main path in the indoor space through the fingerprint acquisition module of the mobile terminal. Then, use the geomagnetic fingerprint segmentation length evaluation algorithm to calculate the segmentation length of the fingerprint sequence. After segmentation using the overlapping segmentation method, each segmented fingerprint is converted into a curve difference matrix; the processed geomagnetic fingerprint is used as a data set to train a geomagnetic fingerprint classification model based on a dilated convolutional neural network on the server, and establish the mapping relationship between the geomagnetic fingerprint and the position coordinates; After the user sends a positioning request through the real-time positioning module of the mobile terminal, the server first processes various sensor data obtained in real time, calculates the walking distance of the user, converts the geomagnetic fingerprint into a curve difference matrix, and then sends it into the geomagnetic fingerprint classification model to predict the most likely positioning position point. Finally, after position correction, the result is returned to the user's mobile terminal; Geomagnetic fingerprint sequence segmentation length evaluation algorithm, specifically: First, divide all geomagnetic fingerprint sequences according to the direction, and the segmentation length is SSN, and SSN traverses within the set maximum and minimum values; use the n - time fingerprint sampling sequences {S1, S2, …, S n} of the total path in the same direction as samples. In each loop, use DTW to horizontally compare the similarity between n segmentation segments in m components among the samples; through experimental statistics, the similarity between the same segments satisfies the chi - square distribution. Calculate the average value and standard deviation of all inter - segment similarity values, and set the discrimination threshold θ of the current segment as θ = μ+σ; then traverse each complete sequence S i . After each sequence is segmented with the current SSN length, it is labeled, and U is the label set of all segments of this sequence; then calculate the similarity between every two of the m segments of the same sequence using DTW. If the result is not greater than the previously calculated discrimination threshold, it is considered that the two segments are similar, and the two segment labels are removed from the label set; finally, calculate and find the SSN value when the discrimination rate converges and approaches 1, which is the optimal segmentation length of the geomagnetic fingerprint sequence; The conversion of the geomagnetic fingerprint into a curve difference matrix specifically means: replacing the absolute value with the difference between sampling points to describe the shape of the curve formed by the fingerprint sequence.

2. The geomagnetic indoor positioning system based on the dilated convolutional neural network according to claim 1, wherein The overlapping segmentation method specifically means: when segmenting the fingerprint sequence, overlapping segmentation is performed with an overlapping ratio of 50%.

3. The geomagnetic indoor positioning system based on the dilated convolutional neural network according to claim 1, wherein The geomagnetic fingerprint classification model based on a dilated convolutional neural network specifically means: the model consists of 6 dilated convolutional layers with dilation rates in the order of {1, 1, 2, 5, 1, 2}, 1 max pooling layer, and 5 fully connected layers. The pooling layer is located between the dilated convolutional layer and the fully connected layer.

Images