A floor recognition method and system based on graph pulse neural network

Abstract

The present invention relates to a floor recognition method and system based on a graph pulse neural network, and belongs to the field of floor positioning technology. The method realizes floor recognition by organizing the signal access point structure in the space into the form of a fingerprint map and combining it with pulse neural network calculation, and specifically includes the following steps: S1: constructing a fingerprint map of the signal access point in the floor space; S2: organizing the input data according to the constructed fingerprint map, and performing pulse encoding; S3: inputting the encoded data into the graph pulse neural network; S4: identifying the user's floor information by voting. Compared with the existing floor recognition method, the present method can automatically adapt to the spatial structure of signal source access points in different environments, and has good performance when generalized to new scenarios; at the same time, using WiFi signals to identify floor information can ensure the universality between various mobile phones, and the calculation requirements are low; and the present method is combined with a pulse neural network, with good calculation efficiency and high recognition accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of floor positioning, in particular to the field of floor positioning methods based on neural networks, and relates to a floor recognition method and system based on graph pulse neural networks. Background Art

[0002] Floor localization refers to identifying the floor position of a user in a multi-story building or environment. It is the basis for a large number of applications and services, such as indoor positioning, recommendation systems, and emergency response and rescue. Common floor localization methods can be divided into fingerprint-based recognition methods and sensor-based methods.

[0003] Fingerprinting is the most common approach to floor positioning. It consists of a training phase and a localization phase. In the training phase, the received signal strength (RSS) from visible cellular towers or WiFi access points (APs) is recorded, along with the corresponding floor information. The RSS vector collected at a certain location is called a fingerprint. In the localization phase, the user sends a floor localization query containing the measured RSS value to the server, which then infers the user's floor information through machine learning methods such as k-nearest neighbor (kNN). An early floor positioning system was SkyLoc, which used cellular signals to identify the user's current floor in a multi-story building. More recent fingerprinting methods use WiFi fingerprints to locate floors, because WiFi-based fingerprinting can achieve higher accuracy than mobile fingerprinting. Although fingerprinting is simple and easy to implement, it requires a large amount of training data, which increases with the number of floors and areas of interest. Collecting training data is usually labor-intensive and time-consuming. In addition, fingerprinting methods require the identification number of the signal source access point, making them dependent on the access point and difficult to extend to new environments. In addition, fingerprinting techniques are also sensitive to changes in access points in the environment (for example, adding or removing certain access points). When the access point changes, it is usually necessary to collect data repeatedly to ensure that the fingerprint is up to date, which results in poor scalability and cannot be well applied to new buildings and environments.

[0004] In order to speed up the on-site inference of traditional fingerprint methods, sensor-based methods are proposed. Sensor-based methods locate the user's floor information by using sensors that are usually built into smartphones, such as accelerometers, gyroscopes, barometers, magnetometers, etc. Various different sensor data are fused through specific methods to reduce the overhead of on-site inference by leveraging crowdsourcing and smartphone perception. Although sensor-based methods can significantly reduce the time and effort of collecting large amounts of training data by additionally using sensors such as accelerometers and barometers, these methods suffer from calibration issues and poor applicability because these sensors are not available on all devices.

[0005] In recent years, deep learning models have been widely used in the field of floor localization due to their strong representation ability and good versatility. The StoryTeller system uses a convolutional neural network for floor localization, first using WiFi signals to generate images, and then inputs the images into a convolutional neural network that predicts the floor. Although StoryTeller is access point independent and building independent, it requires knowledge of the physical building dimensions and the three-dimensional location of the access points, which is not available in some scenarios. The ZeeFI system uses stacked autoencoders to identify floors, which alleviates the consumption of data collection by automatically identifying the ground floor using smartphone sensors. However, ZeeFI only uses two layers of autoencoders, so its representation ability is limited and cannot handle more complex situations well. A recent work integrates an autoencoder and a long short-term memory (LSTM) network to predict the user's floor information. Although these deep learning-based methods have better accuracy than traditional fingerprint recognition methods, they are often computationally expensive and are not suitable for resource-constrained platforms. In addition, some deep learning-based methods require additional information (e.g., the dimensions and locations of signal tower access points) to achieve better scalability.

[0006] As mentioned above, the existing floor recognition methods have one or more problems such as poor generalization ability, poor device adaptability, difficulty in data organization or high computational consumption. Therefore, it is necessary to propose a floor recognition method that can be well applied to different environments, has strong device versatility, high recognition accuracy and high computational efficiency. Summary of the invention

[0007] In view of this, the purpose of the present invention is to provide a floor recognition method and system based on graph pulse neural network. The method and system organize the signal access point structure in the space into the form of a fingerprint map by an automatic method, and combine the efficient computing efficiency of the pulse neural network to achieve easy-to-expand, computationally efficient and device-universal floor recognition.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A floor recognition method based on graph pulse neural network, which realizes floor recognition by organizing the signal access point structure in space into the form of fingerprint spectrum and combining it with pulse neural network calculation. The method specifically includes the following steps:

[0010] S1: Construct fingerprint map of signal access points in floor space;

[0011] S2: Organize the input data according to the constructed fingerprint spectrum and perform pulse encoding;

[0012] S3: input the encoded data into the graph spiking neural network;

[0013] S4: Identify the user's floor information through voting.

[0014] Further, in step S1, an automatic construction method is used when constructing the signal access point fingerprint map, specifically including: constructing the fingerprint map according to the proximity of the access point, taking each visible access point as a node V in the graph i , each edge e connecting two nodes i,j Indicates that the two access points are close to each other; when two access points appear in the same fingerprint, an edge is added to connect them until all fingerprints are traversed.

[0015] Furthermore, the algorithm for constructing the fingerprint map takes the fingerprint feature map D as input and outputs the fingerprint map. The specific steps include: first, generating the number of access points N by calculating the length of any fingerprint vector (such as X0); then, adding each node to the node set of the graph, using the index of the access point that appears in the fingerprint as the node identification number instead of the access point identification number, so it can be expanded to new environments because it reduces the dependence on the access point identification number; then, extracting the index of the access point visible in the fingerprint library into a set C; finally, determining whether the nodes in the set are already connected, if not, adding two edges <V i ,V j > and <V j ,V i >The reason for adding two edges to the edge graph E is to construct an undirected fingerprint graph.

[0016] Furthermore, in step S2, after obtaining the fingerprint of the access point, the input data is pulse-coded. In order to improve the computational efficiency, the traditional schemes such as frequency coding, delay coding and delta coding are not used, because these coding technologies will introduce a longer time step, resulting in a longer inference time of the algorithm. The first layer of the network model is used as the coding layer, and the time step is fixed to 1 to speed up the inference speed of the network model.

[0017] Further, in step S3, the encoded data is input into the graph pulse neural network. By combining the unstructured data processing capabilities of the graph neural network and the efficient computing efficiency of the pulse neural network, the system can efficiently extract features from the fingerprint data.

[0018] Furthermore, in step S4, the floor where the user is located in the floor vector is given the greatest weight through a voting mechanism, so as to identify the floor information of the user.

[0019] The present invention also provides a floor recognition system based on graph pulse neural network.

[0020] The beneficial effects of the present invention are:

[0021] Compared with the existing floor recognition method, the method proposed in the present invention can automatically construct a fingerprint map according to the spatial structure of the signal source access points of different buildings, and efficiently infer the user's floor information. This method has three advantages: 1. It can automatically adapt to the spatial structure of the signal source access points in different environments, and generalize to new scenes with good performance. 2. Using WiFi signals to identify floor information can ensure the universality between various mobile phones, while having low computing requirements. 3. Combined with the pulse neural network, it has good computing efficiency and high recognition accuracy.

[0022] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0024] Figure 1 Comparison of floor recognition accuracy between the present invention and the benchmark method;

[0025] Figure 2 The recognition accuracy comparison between the present invention and the classic KNN on Building B0;

[0026] Figure 3 The recognition accuracy comparison between the present invention and the classic KNN on Building B1;

[0027] Figure 4 The recognition accuracy comparison between the present invention and the classic KNN on Building B2;

[0028] Figure 5 The effect of the amount of training data on the floor positioning accuracy of the present invention in three test scenarios;

[0029] Figure 6 The influence of the access point density on the floor recognition accuracy of the present invention in three test scenarios;

[0030] Figure 7 To use different amounts of data from new buildings to fine-tune the model pre-trained using other buildings for floor localization accuracy;

[0031] Figure 8 It is a schematic diagram of the overall structure of the present invention;

[0032] Fig. 9is the construction diagram of the fingerprint map;

[0033] Fig.10 It is a computational model of spiking neural network;

[0034] Fig.11 A dynamic diagram of a LIF spiking neuron. DETAILED DESCRIPTION

[0035] The technical solution of the present invention is aimed at the following problems existing in the existing floor identification method: 1) poor scalability, that is, it cannot be well extended to new buildings or environments; 2) poor general performance, that is, it does not take into account the differences between different users' mobile phones, resulting in some mobile phone users being unable to use the floor identification method for floor identification; 3) data organization is difficult, that is, the algorithm depends on the physical location of the data access point, resulting in the inability to organize data well in different environments; 4) high computational consumption, that is, the algorithm depends on a large amount of computing power and is not suitable for resource-constrained platforms. The floor identification method that automatically constructs a signal access point fingerprint map by combining graph neural networks and pulse neural networks does not depend on whether the spatial location of the signal access point is known, can be well generalized to new buildings and new environments, and has the characteristics of good computational efficiency and high accuracy.

[0036] The technical solution of the present invention is described in detail below with reference to the accompanying drawings.

[0037] Figure 8 The overall structure of the present invention is shown, which consists of a TAGConv layer, two Deep blocks, a transition layer (also a TAGConv layer), two FC layers, and a voting layer. Each Deep block consists of three Deep layers, which are closely connected. Each TAGConv or FC layer is followed by LIF activation and event-based batch normalization layers, which are not shown in the figure for clarity. Let x denote the fingerprint, x = <rss1,rss2,···,rss M >, where rss i represents the signal strength received from the i-th access point, and M represents the number of access points in the environment. In order to better learn features from fingerprints through graph spiking neural networks, the following representation method is used to describe fingerprints with positive values. The original RSS is then described as a positive value p i , as shown below:

[0038]

[0039] Where τ is the RSS threshold, indicating whether an access point is detected in the fingerprint. β is a constant parameter, which is simply set to the natural constant e. Those access points with RSS lower than τ are considered undetected. Therefore, the fingerprint x can be rewritten as a vector of positive values ​​for all access points in the environment, that is:

[0040] x= <p1,p2,···,p M > (2)

[0041] Then, the floor localization problem of graph spiking neural network is formulated as a problem of learning node embeddings to predict floor labels given a subgraph (corresponding to WiFi fingerprint) of the graph with node attributes. Let G = (V, E) denote the fingerprint graph, node v i ∈V is an access point, and edge e i,j ∈E edge connects two spatially close access points v i and v j The two access points are close to each other in space, which means that the two access points should appear at least once in the feature map of a fingerprint vector. The pseudo code for constructing the fingerprint map is as follows Fig. 9 As shown in the figure, the algorithm takes the fingerprint feature graph D as input and outputs a fingerprint map. It first generates the number of access points N by calculating the length of any fingerprint vector (such as X0). Then, each node is added to the node set of the graph. Using the index of the access point that appears in the fingerprint as the node identification number instead of the access point identification number allows this method to be extended to new environments because it reduces the dependence on the access point identification number. The indexes of the access points visible in the fingerprint library are then extracted to a set C. Finally, it is determined whether the nodes in the set are already connected. If not, two edges are added. <V i ,V j > and <V j ,V i > into the edge graph E. The learning process of node embedding consists of two steps:

[0042] i) Aggregate Messages: The task of this step is to aggregate messages from neighboring access points, which is written as:

[0043]

[0044] in, and represents the node embedding and the message vector of the layer l access point v, f (l) Represents an aggregate function, are the adjacent nodes of v.

[0045] ii) Transform the message: The task of this step is to transform the message to the next layer. Mathematically, this process is described as:

[0046]

[0047] where g (l) is the transformation function of the lth layer. For batch execution, the above formula can be written as:

[0048]

[0049] Among them, f LIF is the LIF activation function, W (l) is a learnable weight matrix for a particular layer, is the normalized matrix of the adjacency matrix A,

[0050]

[0051] Where I is the identity matrix.

[0052] Instead of using topologically popular graph convolution, this paper adopts adaptive graph convolution (TAGConv) because of its excellent performance in terms of accuracy and computational efficiency. It uses a set of fixed-size learnable filters to simultaneously extract node features and the strength of correlation between nodes. Let G c,k is the kth graph filter, then the kth output feature map on the lth layer can be written as:

[0053]

[0054] In the formula, x c is the cth input feature vector of all nodes, C l is the number of input features of each node on the lth layer, b k is a learnable bias vector. In order to make the convolution operation work for arbitrary graph topology, the graph filter needs to be carefully designed. One way is to use the normalized adjacency matrix of the graph To define the filter of the graph,

[0055]

[0056] where g c,f,k are the polynomial coefficients of the graph filter, is the normalized adjacency matrix.

[0057] We use the LIF model as the activation function, which is a popular model for describing the dynamics of spiking neurons. Fig.10 Visualization of the computational model of a spiking neural network. Fig.11 The dynamics of a LIF spiking neuron are shown. In the absence of spikes, the membrane potential of the postsynaptic neuron accumulates input spikes. When the membrane potential reaches a predefined threshold u T When , the postsynaptic neuron will output a pulse and enter a refractory period. Mathematically, the LIF activation function f LIF Described as:

[0058] f LIF (u)=fire a spike&u(t)←uR ,if u(t)≥u T (9)

[0059] Among them, u R and u T are constants, representing the reset value and the pulse emission threshold respectively. The LIF activation function indicates that when the membrane potential of a neuron reaches or exceeds a given threshold u T When it is triggered (i.e., it outputs a pulse), its membrane potential will be reset to u after discharge. R .

[0060] In order to avoid the gradient vanishing and exploding problems, the present invention adopts the batch normalization method, that is, normalizing the presynaptic input along the channel dimension. The input of each neuron is adjusted to a value from 0 to u T In the interval range, u T Represents the predefined peak discharge threshold (0.5 is used in this example). This adjustment can balance the input and neuronal membrane potential to avoid oversaturation of the membrane potential or overexpression of the input information. Mathematically, the normalization process is described as:

[0061]

[0062] in, Represents x l The i-th channel feature map, and Represents x l The expectation and variance over the batch. ∈ is a small constant that avoids division by zero error, and γ i and β i are two learnable parameters.

[0063] During the training process, x i The expectation and variance of can be calculated as,

[0064] E[x i ]=mean(x i ) (12)

[0065] Var[x i ]=mean((x i -E[x i ]) 2 ) (13)

[0066] Estimate E[x i ] and Var[x i], which can be obtained by moving average solution in the training stage. In order to extract richer features, the network is densely linked, as shown in the Deep block in the figure. Here, Deep uses dense links to transfer features. Let x0,x1,...,x l-1 As the input features of the previous layer of the l-th layer, the output of the l-th layer is written as:

[0067] x l =f[x0,x1,...,x l-1 ] (14)

[0068] Where f[] represents the concatenation operation on the feature channel dimension. Each Deep layer here contains two TAGConv, two LIF neurons and two normalization operations. At the top of the model of the present invention are two fully connected layers and a voting layer. The fully connected layers are similar to the layers used in traditional neural networks, namely:

[0069] h=Wx+b (15)

[0070] Among them, x is the input of the previous layer, W is the weight matrix, b is the bias vector, and h is the output feature. The voting layer is used to decode the network output. This operation associates the output vector with each floor label and the final output C in a given time window l is written as:

[0071]

[0072] Among them, U is the constant voting matrix of the decoding peak, O t is the output feature of the last layer at time t. During training, in order to adapt to the back propagation of errors, the mean square error between the average voting result and the label vector y is taken as the loss function.

[0073] L=||yC l || 2 (17)

[0074] Since the impulse function is non-differentiable, the error back propagation method cannot be used directly. In order to solve this problem, the present invention uses a rectangular function to approximate the derivative of the impulse function, that is,

[0075]

[0076] Where a is a width parameter, which is set to 0.5 in this embodiment.

[0077] In this embodiment, PyTorch is used to implement the method proposed in the present invention, and Adam is used to optimize the model on the training data set. The initial learning rate is set to 0.0001, the number of training iterations is set to 100, and the batch size is set to 128. The training and testing procedures are repeated for 5 rounds with different random seeds, and the average accuracy is reported. Here, the proposed method is evaluated by the public floor recognition dataset UJIIndoorLoc, which contains three floors of buildings. The data details are shown in Table 1. The following comparisons and analyses are made during the evaluation process:

[0078] Table 1 UJIIndoorLoc dataset information description

[0079] Building number Number of floors Number of samples B0 4 5785 B1 4 5503 B2 5 9760

[0080] In this embodiment, the floor recognition accuracy of the method proposed in the present invention and the benchmark method on the floor recognition dataset UJIIndoorLoc is compared. Since the benchmark method only reports its recognition accuracy in Building B0 and Building B1, only the recognition accuracy of the method and the benchmark method in Building B0 and Building B1 is compared here. The comparison results are shown in Figure 2. Figure 1 As shown, the present invention is significantly better than all the benchmark methods. Specifically, the present invention achieves floor recognition accuracy rates of about 96% and 82% in Building B0 and Building B1, respectively.

[0081] The recognition accuracy of the proposed method and the classic fingerprint recognition method - kNN (k nearest neighbors) on three test buildings is compared. The k value of the kNN method ranges from 1 to 9 with an interval of 2. Figure 2 , Figure 3 , Figure 4 It shows that compared with the classic kNN, the floor recognition accuracy of the present invention is significantly better than that of the kNN. In Building B0, the accuracy rate increased by about 11.6%, in Building B1 by 3.5%, and in Building B2 by 6.8%. Compared with kNN, which needs to store the identification number of the signal access point and depends on the building, the present method is independent of both the access point and the building, making it easier to expand to new buildings or environments. Figure 5 The figure shows the influence of the amount of training data on the floor positioning accuracy of the present invention in three test scenarios.

[0082] The impact of access point density on the performance of the present invention is analyzed. To this end, a specific access point is randomly deleted from the training data and test data, and the remaining data is used to train the model and test. The access point density represents the ratio of the number of remaining access points to the total number of access points before the deletion operation. Figure 6It shows how different numbers of access points in the training and test data affect the floor recognition accuracy of the present invention. It can be seen that the accuracy decreases as the density of access points used in training decreases. However, it can be observed that even if 50% of the access points are removed, the present invention can achieve relatively high accuracy, with an accuracy of about 89.5% for Building B0, about 66.1% for Building B1, and about 82.6% for Building B2. This means that the present invention is robust to changes in access points.

[0083] The generalization ability of the proposed method is analyzed. Generalization here refers to the degree to which the pre-trained model performs well in new buildings. Specifically, the proposed model is trained on the training data of building B0, and its performance in new buildings (B1 and B2) is evaluated using different amounts of training data. Figure 7 It shows that with only 5% new building training samples, it is possible to achieve about 70.6% accuracy on building B0 and about 90.4% accuracy on building B2. When using about 10% new training data, the floor localization accuracy is close to the baseline training and testing using complete data. Compared with the classic fingerprint recognition method that requires re-collecting the complete fingerprint database, generalization to new buildings here only requires collecting a small amount of training data. This shows that the method has good generalization ability and can be extended to new buildings.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A floor recognition method based on graph pulse neural network, characterized in that: The method realizes floor recognition by organizing the signal access point structure in the space into the form of a fingerprint map and combining it with pulse neural network calculation. The method specifically includes the following steps: S1: Construct fingerprint map of signal access points in floor space; S2: Organize the input data according to the constructed fingerprint spectrum and perform pulse encoding; S3: input the encoded data into the graph spiking neural network; S4: Identify the user's floor information through voting; In step S1, an automatic construction method is used when constructing a signal access point fingerprint map, specifically including: constructing a fingerprint map according to the proximity of the access point, taking each visible access point as a node V in the graph i , each edge e connecting two nodes i,j Indicates that the two access points are close to each other; when two access points appear in the same fingerprint, add an edge to connect them until all fingerprints are traversed; The algorithm for constructing the fingerprint map takes the fingerprint feature map D as input and outputs the fingerprint map. The specific steps include: first, the number of access points N is generated by calculating the length of any fingerprint vector; then, each node is added to the node set of the graph, and the index of the access point that appears in the fingerprint is used as the node identification number instead of the access point identification number, so it can be expanded to new environments; then, the index of the access point visible in the fingerprint library is extracted into a set C; finally, it is determined whether the nodes in the set are connected, and if not, two edges are added <V i ,V j > and <V j ,V i >In the edge graph E, the reason for adding two edges is to construct an undirected fingerprint graph; In step S2, after obtaining the fingerprint of the access point, the input data is pulse-coded, a topology-adaptive graph convolution layer is used as the coding layer, and the time step is fixed to 1 to speed up the inference speed of the network model; In step S3, the encoded data is input into the graph spiking neural network. By combining the unstructured data processing capability of the graph neural network and the high computational efficiency of the spiking neural network, the system can efficiently extract features from the fingerprint data. In step S4, the floor information of the user is identified by giving the greatest weight to the floor vector where the user is located through a voting mechanism.

2. A floor recognition system based on graph pulse neural network, characterized in that: The system adopts the method as claimed in claim 1 to perform floor identification.

Images