Indoor positioning system and method based on deep neural network

Abstract

The present invention provides an indoor positioning system and method based on a deep neural network, which relates to the field of artificial intelligence technology. The present invention uses a neural network chip as the core, and uses a camera, a temperature sensor, a humidity sensor, and a mobile terminal to collect the required data; transmits the required data through the Internet and a router; uses a positioning server to preprocess the data and store the data; and finally, the neural network chip completes the neural network model training, finds the optimal weight value, generates fingerprint information, and determines the positioning point and other tasks. Compared with the traditional indoor positioning system and method, the present invention can be applied to more complex environments and can achieve more accurate positioning, and has a higher practical value.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an indoor positioning system and method based on a deep neural network. Background Art

[0002] With the rapid development of wireless communication networks, mobile users have put forward higher requirements for location-based positioning services. Indoor positioning has always been a difficult problem in location-based services. Due to non-line-of-sight transmission paths, multipath effects, delay distortion and other issues, the accuracy of indoor positioning has always been relatively low. In recent years, neural networks have become a hot topic in the field of artificial intelligence. Neural networks have achieved great success in fields such as image recognition, speech translation, and autonomous driving. More and more researchers have begun to try to use neural networks to achieve breakthroughs in indoor positioning accuracy. Among them, the indoor positioning method based on the neural network fingerprint positioning algorithm has achieved relatively obvious results. The algorithm receives radio wave information such as the signal arrival strength (RSS) or channel information (CSI) of the signal transmission point through radio waves such as Wi-Fi, Bluetooth or geomagnetic signals, and uses deep neural networks to process the above information and generate fingerprint data of the signal transmission point, and then compares the real-time fingerprint data with the fingerprint data of each point in the indoor environment to achieve indoor positioning.

[0003] Traditional neural network-based fingerprint algorithm indoor positioning systems and positioning methods have certain drawbacks. For example, the collected data is not comprehensive enough, and the calculation results are easily affected by indoor temperature and humidity changes, obstacles, and personnel movement. Based on this situation, this patent provides an indoor positioning system and method based on a deep neural network. Summary of the invention

[0004] The purpose of the present invention is to provide an indoor positioning system and method based on a deep neural network to solve the problems raised in the above background technology.

[0005] In order to achieve the above object, the technical solution of the present invention is:

[0006] In the first aspect, an indoor positioning system based on a deep neural network is provided, including a mobile terminal, a temperature sensor, a humidity sensor, a camera, the Internet, a router, a positioning server and a neural network chip.

[0007] The mobile terminal is bidirectionally connected to the router; the output of the temperature sensor is connected to the input of the router; the output of the humidity sensor is connected to the input of the router; the output of the camera is connected to the input of the router; the router is bidirectionally connected to the positioning server; the positioning server is bidirectionally connected to the neural network chip; the mobile terminal is connected to the router, the temperature sensor is connected to the router, the humidity sensor is connected to the router, and the camera is connected to the router via the Internet.

[0008] Furthermore, the neural network chip can be built into a tablet computer, a laptop computer, or a device that combines the two.

[0009] Furthermore, the neural network chip can be made into a computer board form and inserted into personal desktop computers, servers, workstations, and large and medium-sized computers to build a large-scale neural network.

[0010] In a second aspect, an indoor positioning method based on a deep neural network is provided, and the indoor positioning method based on a deep neural network includes an offline training phase and an online positioning phase.

[0011] The offline training phase includes the following steps:

[0012] Step 1: collect image, temperature and humidity data in the indoor positioning environment through a camera, a temperature sensor and a humidity sensor respectively, and perform image preprocessing on the image data collected by the camera;

[0013] Step 2: Input the preprocessed image data into the unsupervised convolutional neural network, extract the feature vectors and classify them, store the classified feature vectors separately, and record the weight values ​​of the unsupervised convolutional neural network;

[0014] Step 3: Add the corresponding temperature and humidity information to the classified and stored feature vector to obtain the final label vector;

[0015] Step 4: Receive channel information data sent by the training mobile terminal through a router, classify the channel information data according to the corresponding labels, input each type of channel information data into the supervised deep neural network respectively, and repeatedly train to obtain multiple groups of fingerprint vectors and corresponding optimal deep neural network weight values.

[0016] The online positioning stage includes the following steps:

[0017] Step 1: collect image, temperature and humidity data in the indoor positioning environment through a camera, a temperature sensor and a humidity sensor respectively, and perform image preprocessing on the image data collected by the camera;

[0018] Step 2: Input the preprocessed image data into the unsupervised convolutional neural network, extract the feature vector and calculate it with the help of the convolutional neural network weight value to obtain the feature vector;

[0019] Step 3: Add the corresponding temperature and humidity information to the calculated feature vector to obtain the final feature vector, compare the feature vector with the label vector, and select the corresponding optimal deep neural network weight value;

[0020] Step 4: Receive channel information data sent by the mobile terminal held by the indoor positioning target user through the router, input the channel information data into the supervised deep neural network, and calculate with the corresponding optimal deep neural network weight value to obtain the fingerprint vector of the indoor position of the mobile terminal held by the user;

[0021] Step 5: Compare the obtained fingerprint vector with the multiple groups of fingerprint vectors obtained in the offline training phase, select the closest fingerprint vector, determine the corresponding positioning position, and feed the position back to the user through the Internet.

[0022] Furthermore, in step one of the offline training stage, the processing method for preprocessing the image data collected by the camera is data enhancement, de-averaging or normalization; in step one of the online positioning stage, the processing method for preprocessing the image data collected by the camera is de-averaging or normalization.

[0023] Furthermore, after the preprocessed image data is input into the unsupervised convolutional neural network, it passes through the two-dimensional convolution layer, activation function layer, normalization layer and pooling layer in sequence, and repeats three rounds. Then the flattening layer is used to convert the data into feature vectors, and finally the activation layer is used for classification.

[0024] Furthermore, the activation function of the activation function layer is a leaky linear rectifier function, and the pooling method used by the pooling layer is a maximum pooling method.

[0025] Furthermore, after the channel information data is input into the supervised convolutional neural network, it passes through the fully connected layer, the activation function layer and the normalization layer in sequence, and is repeated four times, and finally the activation layer is used for classification.

[0026] Furthermore, the activation function of the activation function layer is a leaky linear rectification function.

[0027] Compared with the prior art, the beneficial effects of the present invention are:

[0028] Provided are an indoor positioning system and method based on a deep neural network. Compared with a traditional fingerprint algorithm indoor positioning system and method based on a neural network, the present invention comprises two neural network modules, one is a convolutional neural network module, which is used to process image information obtained by a camera, and the other is a deep neural network module, which is used to process all input data and achieve final indoor positioning.

[0029] The working process of the present invention is divided into two stages, an offline training stage and an online positioning stage. The offline training stage is responsible for collecting indoor radio wave data and temperature, humidity, obstacles and personnel movement photo information corresponding to the radio wave data, and the above information is processed through data preprocessing, convolutional neural network training and deep neural network training to obtain indoor fingerprint data and labels and neural network weight values ​​for various environments. The online positioning stage is responsible for collecting real-time environmental data and radio wave data, and inputting the above data into the convolutional neural network and deep neural network modules after processing to realize indoor positioning.

[0030] The present invention introduces more environmental information parameters as labels into the training of the neural network, obtains the indoor temperature through a temperature sensor, obtains the indoor humidity through a humidity sensor, and obtains indoor obstacles and personnel movement through a camera. Changes in indoor temperature and humidity will change the physical rules followed by radio wave reflection and refraction, and indoor obstacles and personnel movement will directly affect radio wave reflection and refraction. The present invention further subdivides the training data using more data labels, so that the neural network can generate weight values ​​that are closer to the specific actual situation, thereby improving the accuracy of indoor positioning.

[0031] The present invention adds an independent convolutional neural network module, which is used to process indoor photos obtained by the camera and extract hidden features in the photos. The convolutional neural network module belongs to an unsupervised neural network, and sets the neural network output as a set of vectors, and introduces the vectors into the deep neural network training as labels of indoor obstacles and personnel movement. In this way, targeted classification training is carried out for different indoor conditions, so that our neural network module can obtain more accurate weight values. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application and should not be regarded as improper limitations on the present application.

[0033] Figure 1 It is a structural diagram of an indoor positioning system based on a deep neural network in this application.

[0034] Figure 2It is a structural block diagram of the offline training stage in this application.

[0035] Figure 3 It is a structural block diagram of the online positioning stage in this application.

[0036] Figure 4 This is the network structure diagram of the convolutional neural network module in this application.

[0037] Figure 5 This is the network structure diagram of the deep neural network module in this application. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] Reference below Figure 1 The indoor positioning system provided by the present invention is described.

[0040] Embodiment 1:

[0041] An indoor positioning system based on a deep neural network includes a mobile terminal, a temperature sensor, a humidity sensor, a camera, the Internet, a router, a positioning server and a neural network chip, wherein the mobile terminal is bidirectionally connected to the router; the output of the temperature sensor is connected to the input of the router; the output of the humidity sensor is connected to the input of the router; the output of the camera is connected to the input of the router; the router is bidirectionally connected to the positioning server; the positioning server is bidirectionally connected to the neural network chip; the mobile terminal is connected to the router, the temperature sensor is connected to the router, the humidity sensor is connected to the router, and the camera is connected to the router via the Internet. The neural network chip can be built into a tablet computer, a laptop computer, or a device that combines the two; it can also be made into a computer board form and inserted into a personal desktop computer, a server, a workstation, and a large or medium-sized computer to build a large-scale neural network.

[0042] Reference below Figure 2-Figure 5 The indoor positioning method provided by the present invention is described.

[0043] Embodiment 2:

[0044] An indoor positioning method based on a deep neural network, the indoor positioning method based on a deep neural network comprising an offline training phase and an online positioning phase;

[0045] The offline training phase is as follows Figure 2 As shown, the following steps are included:

[0046] Step 1: collect image, temperature and humidity data in the indoor positioning environment through a camera, a temperature sensor and a humidity sensor respectively, and perform image preprocessing on the image data collected by the camera;

[0047] Step 2: Input the preprocessed image data into the unsupervised convolutional neural network, extract the feature vectors and classify them, and store the classified feature vectors separately. The block diagram assumes that the unsupervised convolutional neural network finally divides the images into M groups and records the weight values ​​of the unsupervised convolutional neural network.

[0048] Step 3: Add the corresponding temperature and humidity information to the classified stored feature vector to obtain the final label vector. Assuming there are I temperature values ​​and J humidity values, the final label vector type N is the product of M, I, and J.

[0049] Step 4: Receive channel information data sent by the training mobile terminal through a router, classify the channel information data according to the corresponding labels, input each type of channel information data into the supervised deep neural network, and repeatedly train to obtain N groups of fingerprint vectors and corresponding optimal deep neural network weight values;

[0050] The online positioning stage is as follows Figure 3 As shown, the following steps are included:

[0051] Step 1: collect image, temperature and humidity data in the indoor positioning environment through a camera, a temperature sensor and a humidity sensor respectively, and perform image preprocessing on the image data collected by the camera;

[0052] Step 2: Input the preprocessed image data into the unsupervised convolutional neural network, extract the feature vector and calculate it with the help of the convolutional neural network weight value to obtain the feature vector;

[0053] Step 3: Add the corresponding temperature and humidity information to the calculated feature vector to obtain the final feature vector, compare the feature vector with the label vector, and select the corresponding optimal deep neural network weight value;

[0054] Step 4: Receive channel information data sent by the mobile terminal held by the indoor positioning target user through the router, input the channel information data into the supervised deep neural network, and calculate with the corresponding optimal deep neural network weight value to obtain the fingerprint vector of the indoor position of the mobile terminal held by the user;

[0055] Step 5: Compare the obtained fingerprint vector with the multiple groups of fingerprint vectors obtained in the offline training phase, select the closest fingerprint vector, determine the corresponding positioning position, and feed the position back to the user through the Internet.

[0056] In step one of the offline training stage, the processing method for preprocessing the image data collected by the camera is data augmentation, mean subtraction or normalization; in step one of the online positioning stage, the processing method for preprocessing the image data collected by the camera is mean subtraction or normalization.

[0057] like Figure 4 As shown in the figure, after the preprocessed image data is input into the unsupervised convolutional neural network, it passes through the two-dimensional convolution layer, activation function layer, normalization layer (BatchNormalization) and pooling layer in sequence, repeats three rounds, and then uses the flattening layer (Flatten) to convert the data into feature vectors, and finally uses the activation layer (SoftMax) for classification. The activation function of the activation function layer is the leaky linear rectifier function (Leaky-Relu), and the pooling method used in the pooling layer is the maximum pooling method (Max-Pooling).

[0058] like Figure 5 As shown, after the channel information data is input into the supervised convolutional neural network, it passes through the fully connected layer (Dense), the activation function layer and the normalization layer in sequence, repeats four rounds, and finally uses the activation layer for classification. The activation function of the activation function layer is a leaky linear rectifier function.

[0059] Embodiment 3:

[0060] If the user uses the positioning function continuously, the fingerprint vector obtained in the online positioning phase is compared with the fingerprint vectors of all positions obtained in the offline training phase, and several closest fingerprint vectors are selected, and several corresponding positioning positions are determined. The position closest to the last positioning moment among all selected positions is selected as the current positioning position, and this position is fed back to the user through the Internet.

[0061] Embodiment 4:

[0062] If the user uses the positioning function intermittently, the fingerprint vector obtained in the online positioning phase is compared with the fingerprint vectors of all positions obtained in the offline training phase, the closest fingerprint vector is selected, and the corresponding positioning position is determined, and finally this position is fed back to the user through the Internet.

[0063] It should be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "includes..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0064] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An indoor positioning method based on a deep neural network, characterized in that: The indoor positioning method based on deep neural network includes an offline training phase and an online positioning phase; The offline training phase includes the following steps: Step 1: collect image, temperature and humidity data in the indoor positioning environment through a camera, a temperature sensor and a humidity sensor respectively, and perform image preprocessing on the image data collected by the camera; Step 2: Input the preprocessed image data into the unsupervised convolutional neural network, extract the feature vectors and classify them, store the classified feature vectors separately, and record the weight values ​​of the unsupervised convolutional neural network; Step 3: Add the corresponding temperature and humidity information to the classified and stored feature vector to obtain the final label vector; Step 4: Receive channel information data sent by the training mobile terminal through a router, classify the channel information data according to corresponding labels, input each type of channel information data into the supervised deep neural network, and repeatedly train to obtain multiple groups of fingerprint vectors and corresponding optimal deep neural network weight values; The online positioning stage includes the following steps: Step 1: collect image, temperature and humidity data in the indoor positioning environment through a camera, a temperature sensor and a humidity sensor respectively, and perform image preprocessing on the image data collected by the camera; Step 2: Input the preprocessed image data into the unsupervised convolutional neural network, extract the feature vector and calculate it with the help of the convolutional neural network weight value to obtain the feature vector; Step 3: Add the corresponding temperature and humidity information to the calculated feature vector to obtain the final feature vector, compare the feature vector with the label vector, and select the corresponding optimal deep neural network weight value; Step 4: Receive channel information data sent by the mobile terminal held by the indoor positioning target user through the router, input the channel information data into the supervised deep neural network, and calculate with the corresponding optimal deep neural network weight value to obtain the fingerprint vector of the indoor position of the mobile terminal held by the user; Step 5: Compare the obtained fingerprint vector with the multiple groups of fingerprint vectors obtained in the offline training phase, select the closest fingerprint vector, determine the corresponding positioning position, and feed the position back to the user through the Internet.

2. The indoor positioning method based on deep neural network according to claim 1, characterized in that: In step 1 of the offline training stage, the image data collected by the camera is preprocessed by a processing method of data enhancement, de-averaging or normalization; In step 1 of the online positioning stage, the image data collected by the camera is preprocessed by removing the mean or normalizing.

3. The indoor positioning method based on deep neural network according to claim 1, characterized in that: After the preprocessed image data is input into the unsupervised convolutional neural network, it passes through the two-dimensional convolution layer, activation function layer, normalization layer and pooling layer in sequence, and repeats three rounds. Then the flattening layer is used to convert the data into feature vectors, and finally the activation layer is used for classification.

4. The indoor positioning method based on deep neural network according to claim 3, characterized in that: The activation function of the activation function layer is a leaky linear rectifier function, and the pooling method used by the pooling layer is the maximum pooling method.

5. The indoor positioning method based on deep neural network according to claim 1, characterized in that: After the channel information data is input into the supervised convolutional neural network, it passes through the fully connected layer, the activation function layer and the normalization layer in sequence, and is repeated four times. Finally, the activation layer is used for classification.

6. The indoor positioning method based on deep neural network according to claim 5, characterized in that: The activation function of the activation function layer is a leaky linear rectifier function.

Images