Indoor and outdoor integrated positioning method, system, equipment and medium

Abstract

The invention provides an indoor and outdoor integrated positioning method, system, device and medium, and belongs to the technical field of positioning, and the method comprises the following steps: collecting GNSS outdoor positioning data, IMU indoor positioning data, UWB indoor positioning data and height positioning data of a positioning terminal; constructing a convolutional neural network positioning model comprising two different channels, inputting the collected data into the two different channels of the convolutional neural network positioning model, extracting features of the input data at different nodes and features of the input data at different time sequences respectively, and fusing the extracted features to obtain a convolutional neural network positioning model; and predicting the position of the positioning terminal according to the fused features. According to the method, the features of the input data at different nodes and different time sequences can be extracted and fused, and indoor and outdoor high-precision seamless positioning can be realized through the fused positioning result.

Description

Technical Field

[0001] The present invention belongs to the field of positioning, and in particular relates to an indoor and outdoor integrated positioning method, system, equipment and medium. Background Art

[0002] With the rapid development of the economy and society and the emergence of various technologies, navigation and positioning technology has also developed rapidly, and the demand for navigation and positioning is also growing. At present, satellites in orbit have basically met people's navigation needs in outdoor environments, but in indoor environments, satellite signals in orbit cannot enter indoors, and navigation technology still has great limitations. With the rapid development of unmanned driving, robots, the Internet of Things and other fields, the demand for seamless indoor and outdoor positioning is becoming more and more urgent.

[0003] The existing indoor and outdoor positioning technologies based on GNSS, UWB, etc. use least squares or Kalman filtering technology to estimate the user position based on existing statistical models. Such methods are effective when the indoor environment does not change. However, when the environment changes, the existing statistical models are no longer suitable for data processing, resulting in poor positioning results. Summary of the invention

[0004] In order to overcome the shortcomings of the above-mentioned prior art in that the positioning results are poor, the present invention provides an indoor and outdoor integrated positioning method, comprising the following steps:

[0005] Collect GNSS outdoor positioning data, IMU indoor positioning data, UWB indoor positioning data and altitude positioning data of the positioning terminal;

[0006] A convolutional neural network positioning model containing two different channels is constructed. The GNSS outdoor positioning data, IMU indoor positioning data, UWB indoor positioning data and altitude positioning data are input into two different channels of the convolutional neural network positioning model. The features of the input data at different nodes and the features of different time series are extracted respectively, and then the extracted features are fused. The position of the positioning terminal is predicted based on the fused features.

[0007] Preferably, the first channel of the convolutional neural network positioning model includes 2 CNN convolutional layers, 2 Dropout random discarding layers and 1 Flatten straightening layer, and the second channel includes 1 Flatten straightening layer and 1 fully connected layer. The features output by the straightening layer of the first channel and the features output by the fully connected layer of the second channel are added and then passed through 3 fully connected layers to obtain an output feature map.

[0008] Preferably, the CNN convolutional layer is used to extract features from input data, specifically comprising the following steps:

[0009] Multiply the data in each position window of the convolution layer by the convolution kernel element by element, and then add the results to get an element in the output feature map;

[0010] An activation function is used to perform a nonlinear transformation on an element in the obtained feature map; the output space of the result after the nonlinear transformation is reduced through the pooling layer to obtain a two-dimensional output feature map.

[0011] Preferably, before inputting the GNSS outdoor positioning data, the IMU indoor positioning data, the UWB indoor positioning data and the altitude positioning data into two different channels of the convolutional neural network positioning model, the method further includes training the convolutional neural network positioning model, including the following steps:

[0012] Select several target locations to be located in the area to be located, and collect multiple GNSS, IMU outdoor positioning data, UWB indoor positioning data and altitude positioning data at each target location;

[0013] Combine multiple frames of one-dimensional GNSS data, IMU outdoor positioning data, UWB indoor positioning data and altitude positioning data at the same target location into two-dimensional data;

[0014] The two-dimensional data is input into a convolutional neural network positioning model for training to obtain a trained convolutional neural network positioning model.

[0015] Preferably, the size of the convolution kernel of the CNN convolution layer is 2×2.

[0016] Preferably, the GNSS outdoor positioning data, the IMU indoor positioning data, the UWB indoor positioning data and the altitude positioning data are all acquired through sensors of the data terminal.

[0017] The present invention also provides an indoor and outdoor integrated positioning system, comprising:

[0018] Data collection module, used to collect GNSS outdoor positioning data, IMU indoor positioning data, UWB indoor positioning data and altitude positioning data of the positioning terminal;

[0019] The positioning module is used to build a convolutional neural network positioning model containing two different channels. The GNSS outdoor positioning data, IMU indoor positioning data, UWB indoor positioning data and altitude positioning data are input into two different channels of the convolutional neural network positioning model, and the features of the input data at different nodes and the features of different time series are extracted respectively. The extracted features are then fused, and the position of the positioning terminal is predicted based on the fused features.

[0020] Preferably, the positioning module is arranged on a server platform, and the GNSS outdoor positioning data, IMU indoor positioning data, UWB indoor positioning data and altitude positioning data are transmitted to the server platform via a 4G network.

[0021] The present invention also provides a computer device, including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the indoor and outdoor integrated positioning method.

[0022] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the indoor and outdoor integrated positioning method.

[0023] The indoor and outdoor integrated positioning method, system, device and medium provided by the present invention have the following beneficial effects:

[0024] The present invention collects GNSS outdoor positioning data, IMU indoor positioning data, UWB indoor positioning data and altitude positioning data, and inputs them into two different channels of a convolutional neural network positioning model. The features of the input data at different nodes and the features of different time series can be extracted through the two different channels, and the process can obtain the characteristics of indoor environmental changes. By fusing the obtained features of different nodes and the features of different time series, the fused positioning parameters can be dynamically adjusted in real time, thereby achieving high-precision seamless positioning indoors and outdoors under environmental changes, and providing precise location services for the Internet of Things and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiment of the present invention and its design scheme, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A flowchart of an indoor and outdoor integrated positioning method according to an embodiment of the present invention;

[0027] Figure 2 Structural diagram of the convolutional neural network positioning model;

[0028] Figure 3 Schematic diagram of data convolution after fusion;

[0029] Figure 4 It is the overall architecture of the BeiDou / UWB / IMU seamless high-precision space-time service platform for indoor and outdoor use;

[0030] Figure 5For the server platform architecture. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the technical solution of the present invention and implement it, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention.

[0032] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the technical solutions of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0033] In addition, the terms "first", "second", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. In the description of the present invention, it should be noted that, unless otherwise clearly specified or limited, the terms "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. In the description of the present invention, unless otherwise specified, "plurality" means two or more, which will not be described in detail here.

[0034] Example

[0035] The present invention provides an indoor and outdoor integrated positioning method, such as Figure 1 As shown, the following steps are included:

[0036] Step 1: Collect GNSS outdoor positioning data, IMU indoor positioning data, UWB indoor positioning data and altitude positioning data of the positioning terminal.

[0037] The indoor positioning part includes UWB / IMU / barometer. After the UWB ranging signal is smoothed, iterative least squares is used to obtain the indoor position result; the barometer obtains the altitude information (altitude positioning data) based on the differential pressure method. The outdoor positioning mode adopts the GNSS positioning mode and performs quality control on the positioning results.

[0038] Step 2: Construct a convolutional neural network positioning model containing two different channels. Input the GNSS outdoor positioning data (data obtained by the global navigation satellite system), IMU indoor positioning data (data obtained by the built-in inertial sensor IMU of personal wearable devices), UWB indoor positioning data (data obtained by ultra-wideband) and altitude positioning data (data obtained by the barometer) into two different channels of the convolutional neural network positioning model, extract the features of the input data at different nodes and the features of different time series, fuse the extracted features, and predict the position of the positioning terminal based on the fused features.

[0039] After the acquired data is uniformly input into the convolutional neural network positioning model, a unified spatiotemporal benchmark is first established, and then combined with the convolutional neural network positioning model to output stable indoor and outdoor seamless positioning results. The specific process is as follows:

[0040] 1. Training the convolutional neural network positioning model includes the following steps:

[0041] (1) Data collection: Select several target locations to be located in the positioning area, collect multiple GNSS outdoor positioning data, IMU indoor positioning data, and UWB indoor positioning data at each target location. The data collected for the i-th time is x i .

[0042] (2) Random time series data fusion: Multiple frames of one-dimensional data x at the same position are fused i Combined into two-dimensional data, recorded as X κ , X κ is the k-th data of the input data matrix X, and the corresponding output data is y κ .

[0043] (3) Model training: First, the data X κ The data is sent to the convolutional neural network positioning model, and the convolutional neural network positioning model is used to extract data features to obtain signal features. Finally, the fully connected neural network of the convolutional neural network positioning model is used for prediction, and the output is the location of the grid.

[0044] (4) Output the trained positioning model.

[0045] Compared with the conventional convolutional neural network, the convolutional neural network positioning model of the present invention uses two different channels to extract features of the input data. The convolutional layer is used to extract the connection between the output positions of sensors such as GNSS, IMU and UWB, and the fully connected layer is used to map the low-dimensional features to the high-dimensional feature space, and finally add them together to obtain the comprehensive features.

[0046] 2. Feature extraction and positioning.

[0047] The convolutional neural network positioning model used in the present invention is as follows: Figure 2 As shown in the figure, the input data passes through two different channels. The first channel includes 2 CNN layers (the size of the convolution kernel is 2×2), 2 Dropout layers and 1 straightening layer (Flatten), and the second channel is 1 straightening layer and 1 fully connected layer (Fully connected layers, FC). The outputs of the two channels are added and classified through 3 fully connected layers to obtain the category to which the input data belongs; the second channel directly connects the output from the first layer to the input of the second layer, thereby reducing the number of parameters of the channel. The fewer the number of parameters, the less likely it is to overfit.

[0048] The convolution layer in the convolutional neural network is the core part. It extracts the features of the input data through the convolution operation and outputs a two-dimensional feature map. First, the data is sent to the convolution layer. The convolution operation can be regarded as a sliding window. The data in each position window is multiplied element by element by the convolution kernel, and then the results are added to obtain an element in the output feature map. The mathematical expression of the two-dimensional convolution operation is:

[0049]

[0050] In the formula, h i,j is an element in the output feature map, w m,n is the weight of the convolution kernel, x i+m,j+n is the corresponding element in the input data, b is the bias term, A and B both represent the size of the convolution kernel, and i and j are the row and column indexes of the calculation results. Then, the calculation results of the convolution layer are transformed nonlinearly through the activation layer to obtain Where σ is the activation function, is the calculation result of the activation layer. Finally, The pooling layer is used to reduce the spatial size of the convolutional layer output and obtain the calculation result. This is the output feature map of the convolutional layer.

[0051] Four groups of data are selected for fusion, each group of data has 12 eigenvalues ​​( Figure 3 The data in the dashed box is a group), that is, the original data is X∈R 4×12 The fused data is The convolution process is as follows Figure 3 As shown in the figure, red represents GNSS data, green represents IMU, and blue represents UWB. The size of the convolution kernel is selected as 2×2. The convolutional neural network positioning model can not only extract the connection between different nodes, but also extract the connection between different time series. For example, the value of (1,1) is the value of the first group of data. θ 11 , as well as The value of (2,4) is the value of the first group of data. θ 21 , as well as It is obtained by multiplying the convolution kernel, which contains the relationship between different time series data.

[0052] The present invention also provides an indoor and outdoor integrated positioning system, including a data acquisition module and a positioning module. The data acquisition module is used to collect GNSS outdoor positioning data, IMU indoor positioning data, UWB indoor positioning data and height positioning data of the positioning terminal. The positioning module is used to construct a convolutional neural network positioning model containing two different channels, and the GNSS outdoor positioning data, IMU indoor positioning data, UWB indoor positioning data and height positioning data are all input into two different channels of the convolutional neural network positioning model, and the features of the input data at different nodes and the features of different time series are extracted respectively, and then the extracted features are fused, and the position of the positioning terminal is predicted according to the fused features. In this embodiment, the positioning module mainly includes three parts: Beidou high-precision positioning algorithm, UWB indoor high-precision positioning algorithm and combined indoor and outdoor seamless high-precision positioning algorithm based on deep learning. Among them, the Beidou high-precision positioning algorithm is mainly used for high-precision positioning in outdoor open environments, the UWB indoor high-precision positioning is mainly used for indoor environments lacking satellite navigation signals, and the deep learning indoor and outdoor seamless high-precision positioning is mainly used for the transition from indoor to outdoor or from outdoor to indoor, and finally provides indoor and outdoor seamless high-precision positioning results for different industries.

[0053] Specifically, the indoor and outdoor integrated positioning system architecture of the present invention is as follows: Figure 4 As shown, satellite data and UWB base station data are received in real time through Beidou GNSS, UWB and IMU to obtain position and distance information, which are transmitted to the server platform in real time through 4G using TCP protocol. The server platform receives terminal data in real time, calls the positioning module, obtains the results of fusion positioning in real time, and publishes them to the terminal user (data terminal).

[0054] The data terminal and the server platform establish data transmission through 4G. The data transmission protocol adopts TCP. The platform opens multiple ports, one device corresponds to one port, to achieve real-time data reception. The transmitted data includes GPGGA data of GNSS positioning, UWB ranging data and barometer data (altitude positioning data), and the data format can be in plain text format.

[0055] The server platform receives data from the data terminal in real time, calls the UWB base station coordinate data, and calls the positioning module. The positioning module can fuse and locate multi-source heterogeneous data, solve in real time to obtain the fused positioning result, and output the positioning to the main program. The main program publishes the positioning result in real time through TCP.

[0056] like Figure 5 As shown, the data acquisition module is deployed on the server platform, receives sensor data uploaded by multiple personnel positioning devices in real time, and forwards it to the positioning module. At the same time, it can store the data uploaded by the device to files for later testing and evaluation. The software supports TCP protocol.

[0057] The positioning module mainly includes data reception and analysis, GNSS / IMU / UWB / barometer sensor data processing submodule, and positioning module; after the GNSS / IMU / UWB / barometer data processing submodule processes the data, it transmits the result to the positioning module, and the positioning module outputs the positioning result to the user position display subsystem.

[0058] The user location display subsystem is deployed on the server platform and is used to receive user location information in real time and display it in real time.

[0059] The present invention also provides a computer device, including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the indoor and outdoor integrated positioning method.

[0060] The present invention also provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded by a processor to execute the indoor and outdoor integrated positioning method.

[0061] The embodiments described above are only preferred specific implementation modes of the present invention, and the protection scope of the present invention is not limited thereto. Any simple changes or equivalent replacements of the technical solutions that can be obviously obtained by any technician familiar with the field within the technical scope disclosed in the present invention belong to the protection scope of the present invention.

Claims

1. An indoor and outdoor integrated positioning method, characterized in that: The steps include: Collect GNSS outdoor positioning data, IMU indoor positioning data, UWB indoor positioning data and altitude positioning data of the positioning terminal; A convolutional neural network positioning model containing two different channels is constructed. The GNSS outdoor positioning data, IMU indoor positioning data, UWB indoor positioning data and altitude positioning data are input into two different channels of the convolutional neural network positioning model. The features of the input data at different nodes and the features of different time series are extracted respectively, and then the extracted features are fused. The position of the positioning terminal is predicted based on the fused features.

2. The indoor and outdoor integrated positioning method according to claim 1, characterized in that: The first channel of the convolutional neural network positioning model includes 2 CNN convolutional layers, 2 Dropout random discarding layers and 1 Flatten straightening layer, and the second channel includes 1 Flatten straightening layer and 1 fully connected layer. The features output by the straightening layer of the first channel and the features output by the fully connected layer of the second channel are added and then passed through 3 fully connected layers to obtain an output feature map.

3. The indoor and outdoor integrated positioning method according to claim 2, characterized in that: The CNN convolutional layer is used to extract features from input data, specifically including the following steps: Multiply the data in each position window of the convolution layer by the convolution kernel element by element, and then add the results to get an element in the output feature map; Use the activation function to perform a nonlinear transformation on an element in the obtained feature map; The output space of the result after nonlinear transformation is reduced by the pooling layer to obtain a two-dimensional output feature map.

4. The indoor and outdoor integrated positioning method according to claim 2, characterized in that: Before inputting the GNSS outdoor positioning data, the IMU indoor positioning data, the UWB indoor positioning data and the height positioning data into two different channels of the convolutional neural network positioning model, the convolutional neural network positioning model is trained, including the following steps: Select several target locations to be located in the area to be located, and collect multiple GNSS, IMU outdoor positioning data, UWB indoor positioning data and altitude positioning data at each target location; Combine multiple frames of one-dimensional GNSS data, IMU outdoor positioning data, UWB indoor positioning data and altitude positioning data at the same target location into two-dimensional data; The two-dimensional data is input into a convolutional neural network positioning model for training to obtain a trained convolutional neural network positioning model.

5. The indoor and outdoor integrated positioning method according to claim 2, characterized in that: The size of the convolution kernel of the CNN convolutional layer is 2×2.

6. The indoor and outdoor integrated positioning method according to claim 1, characterized in that: The GNSS outdoor positioning data, IMU indoor positioning data, UWB indoor positioning data and altitude positioning data are all acquired through sensors of the data terminal.

7. An indoor and outdoor integrated positioning system, characterized in that: include: Data collection module, used to collect GNSS outdoor positioning data, IMU indoor positioning data, UWB indoor positioning data and altitude positioning data of the positioning terminal; The positioning module is used to build a convolutional neural network positioning model containing two different channels. The GNSS outdoor positioning data, IMU indoor positioning data, UWB indoor positioning data and altitude positioning data are input into two different channels of the convolutional neural network positioning model, and the features of the input data at different nodes and the features of different time series are extracted respectively. The extracted features are then fused, and the position of the positioning terminal is predicted based on the fused features.

8. The indoor and outdoor integrated positioning system according to claim 7, characterized in that: The positioning module is arranged on a server platform, and the GNSS outdoor positioning data, the IMU indoor positioning data, the UWB indoor positioning data and the altitude positioning data are transmitted to the server platform via a 4G network.

9. A computer device, characterized in that: It comprises a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the indoor and outdoor integrated positioning method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the indoor and outdoor integrated positioning method according to any one of claims 1 to 6.

Images