Passive fusion positioning method and system in indoor environment and electronic device

Abstract

The application provides a kind of indoor environment under passive fusion positioning method, system and electronic equipment, it is related to indoor positioning technical field.The method comprises constructing fusion weight, using the WKNN algorithm based on fusion weight, the magnetic field matching positioning result of target carrier is determined;According to real-time motion magnetic field data and real-time motion inertial sensor data, using adaptive window evaluation mean filter algorithm and long distance step model, the PDR positioning result of target carrier is determined;According to real-time motion two-dimensional vision data and two-dimensional vision fingerprint database construction data, using semantic VSLAM auxiliary matching algorithm, the VSLAM matching positioning result of target carrier is determined;According to magnetic field matching positioning result, PDR positioning result and VSLAM matching positioning result, the fusion positioning result of target carrier is determined.The application can improve indoor positioning accuracy by fusing magnetic field matching positioning result, PDR positioning result and VSLAM matching positioning result of target carrier.

Description

Technical Field

[0001] This invention relates to the field of indoor positioning technology, and in particular to a passive fusion positioning method, system and electronic device for indoor environments. Background Technology

[0002] In the wave of digitalization, location-based services (LBS) are a fundamental force driving the development of the Internet of Things (IoT). Currently, the Global Navigation Satellite System (GNSS) provides high-precision location information in outdoor environments and is widely used in social production. However, due to interference from non-line-of-sight (NLOS) and multipath effects, GNSS positioning technology is difficult to utilize effectively in large indoor spaces such as shopping malls, hospitals, and underground parking garages, leaving a gap in indoor location services. With the development of the IoT, intelligent products such as smartphones, smart homes, and smart supermarkets, as important components of the IoT, have placed higher demands on the accuracy of indoor location services. To achieve interconnectivity between people, machines, and things in time and space, a low-cost, high-precision indoor positioning technology is needed.

[0003] Passive positioning technologies such as magnetic field matching and Pedestrian Dead Reckoning (PDR) are characterized by the absence of additional signal transmitters, low initial construction costs, and strong versatility. Benefiting from multi-source fusion technology and the complementary nature of these two methods, geomagnetic / PDR fusion positioning technology shows promising prospects in indoor positioning research. However, these technologies rely on sampling information from the target positioning device's sensors for localization. Due to the low sampling accuracy of commercially available micro-electromechanical systems (MEMS) sensors, their accuracy is generally low in smart device applications. Furthermore, positioning technologies based on passive information such as magnetic field / PDR suffer from low feature dimensionality, high rates of anomalous points with similar features, and difficulty in initialization, thus lacking large-scale positioning capabilities. Visual Simultaneous Localization and Mapping (VSLAM), based on visual sensors to acquire information about the carrier environment, provides high-dimensional image and point cloud information, becoming one of the important technologies for positioning and mapping. However, VSLAM is sensitive to environmental changes, has poor robustness, and its information computation and storage are complex, consuming a lot of system resources. Therefore, its expected value as a standalone positioning technology is low. Summary of the Invention

[0004] The purpose of this invention is to provide a passive fusion positioning method, system, and electronic device for indoor environments, which can improve indoor positioning accuracy.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A passive fusion positioning method for indoor environments includes:

[0007] The system acquires real-time motion data, fingerprint database construction data, and long-distance step length model training data of the target carrier in an indoor environment. These data are obtained from various sensors mounted on the target carrier. The real-time motion data includes real-time motion magnetic field data, real-time motion inertial sensor data, and real-time motion two-dimensional visual data. The fingerprint database construction data includes magnetic field fingerprint database construction data and two-dimensional visual fingerprint database construction data.

[0008] A fusion weight is constructed based on the real-time motion magnetic field data and the magnetic field fingerprint database, and the magnetic field matching and positioning result of the target carrier is determined using the WKNN algorithm based on the fusion weight.

[0009] Based on the real-time motion magnetic field data and the real-time motion inertial sensor data, the PDR heading information of the target carrier is determined using an adaptive window evaluation mean filtering algorithm.

[0010] Using training data from the long-range step size model, a basic deep learning model, a fully connected neural network model, is trained to obtain the long-range step size model.

[0011] Based on the real-time motion sensor data, the PDR heading information, and the long-distance step size model, the PDR positioning result of the target vehicle is determined;

[0012] Based on the real-time motion 2D visual data and the 2D visual fingerprint database, the VSLAM matching and localization results of the target carrier are determined using the semantic VSLAM-assisted matching algorithm.

[0013] Based on the magnetic field matching positioning results, the PDR positioning results, and the VSLAM matching positioning results, the fusion positioning results of the target carrier are constrained.

[0014] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0015] This invention provides a passive fusion positioning method, system, and electronic device for indoor environments, relating to the field of indoor positioning technology. The method includes: constructing fusion weights based on real-time motion magnetic field data and magnetic field fingerprint database data; determining the magnetic field matching positioning result of a target vehicle using a WKNN algorithm based on the fusion weights; determining the PDR positioning result of the target vehicle using an adaptive window evaluation mean filtering algorithm and a long-distance step size model based on real-time motion magnetic field data and real-time motion inertial sensor data; determining the VSLAM matching positioning result of the target vehicle using a semantic VSLAM-assisted matching algorithm based on real-time motion two-dimensional visual data and two-dimensional visual fingerprint database data; and constraining the fusion positioning result based on the magnetic field matching positioning result, PDR positioning result, and VSLAM matching positioning result. This invention improves indoor positioning accuracy by fusing the magnetic field matching positioning result, PDR positioning result, and VSLAM matching positioning result of the target vehicle. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the passive fusion positioning method for indoor environments provided by the present invention;

[0018] Figure 2 This is a schematic diagram of the passive fusion positioning method for indoor environments provided by the present invention.

[0019] Figure 3 A logic diagram illustrating the passive fusion positioning method for indoor environments provided by this invention;

[0020] Figure 4 A schematic diagram illustrating the extraction of inertial data feature values ​​from the training data of the long-distance step-length model provided by this invention;

[0021] Figure 5 This is a schematic diagram of the fully connected neural network provided by the present invention;

[0022] Figure 6 A schematic diagram illustrating the construction of the step size constraint factor provided by this invention;

[0023] Figure 7 This is a schematic diagram of semantic map construction provided by the present invention;

[0024] Figure 8 This is a diagram of the passive fusion positioning system architecture for indoor environments provided by the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] The purpose of this invention is to provide a passive fusion positioning method, system, and electronic device for indoor environments, which can improve indoor positioning accuracy.

[0027] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] Example 1

[0029] like Figure 1 As shown, this embodiment provides a passive fusion positioning method for indoor environments, including:

[0030] Step 1: Acquire real-time motion data, fingerprint database construction data, and long-distance step-length model training data of the target vehicle in an indoor environment. The real-time motion data, fingerprint database construction data, and long-distance step-length model training data are all acquired by various sensors mounted on the target vehicle. Real-time motion data includes real-time motion magnetic field data, real-time motion inertial sensor data, and real-time motion 2D visual data. Fingerprint database construction data includes magnetic field fingerprint database construction data and 2D visual fingerprint database construction data.

[0031] Step 2: Construct fusion weights based on real-time motion magnetic field data and magnetic field fingerprint database, and use the WKNN algorithm based on fusion weights to determine the magnetic field matching and positioning results of the target carrier.

[0032] Step 3: Based on real-time motion magnetic field data and real-time motion inertial sensor data, the PDR heading information of the target vehicle is determined using an adaptive window evaluation mean filtering algorithm.

[0033] Step 4: Use the long-distance step size model training data to train the basic deep learning model, the fully connected neural network model, to obtain the long-distance step size model;

[0034] Step 5: Determine the PDR positioning result of the target vehicle based on real-time motion sensor data, PDR heading information and long-distance step size model.

[0035] Step 6: Based on real-time motion 2D visual data and 2D visual fingerprint database, construct data and use semantic VSLAM-assisted matching algorithm to determine the VSLAM matching and localization result of the target carrier.

[0036] Step 7: Based on the magnetic field matching positioning results, PDR positioning results, and VSLAM matching positioning results, constrain the fusion positioning results of the target carrier.

[0037] Step 2 includes:

[0038] Step 2-1: Determine candidate magnetic field fingerprint points based on the real-time motion magnetic field data trend and the data trend constructed from the magnetic field fingerprint database.

[0039] Step 2-2: Determine the magnetic similarity weight of each magnetic candidate fingerprint point based on the Euclidean distance between the magnetic field characteristics of the target carrier and the magnetic field characteristics of the magnetic field candidate fingerprint points.

[0040] Steps 2-3: Determine the inverse spatial distance weight of each magnetic field candidate fingerprint point based on the actual distance between the initial position of the target carrier and the position of the magnetic field candidate fingerprint point.

[0041] Steps 2-4: Determine the product of the magnetic field similarity weight and the anti-spatial distance weight of the same magnetic field candidate fingerprint point as the magnetic field candidate fingerprint point fusion weight.

[0042] Steps 2-5: Based on the fusion weights of the candidate magnetic fingerprint points and the coordinates of multiple candidate magnetic fingerprint points, the WKNN algorithm is used to determine the magnetic field matching and positioning results of the target carrier.

[0043] Step 3 includes:

[0044] Step 3-1: Determine the original heading angle sequence of the target carrier based on real-time motion magnetic field data and real-time motion inertial sensor data.

[0045] Step 3-2: The original heading angle sequence of the target vehicle is logically judged using an adaptive window evaluation mean filtering algorithm, and adaptive window mean filtering is performed to obtain the PDR heading information. The PDR heading information is the mean of multiple original heading angles within the adaptive window.

[0046] Step 4 includes:

[0047] Step 4-1: Construct a basic deep learning model, a fully connected neural network model.

[0048] Step 4-2: Using historical step frequency, historical mean acceleration, historical maximum acceleration, historical minimum acceleration, historical acceleration difference, and historical acceleration variance as inputs, and historical step length as output, train the basic deep learning model, the fully connected neural network model, to obtain the long-distance step length model.

[0049] Step 5 includes:

[0050] Step 5-1: Based on the real-time motion inertial sensor data, determine the target vehicle's real-time step frequency, real-time average acceleration, real-time maximum acceleration, real-time minimum acceleration, real-time acceleration difference, and real-time acceleration variance.

[0051] Step 5-2: Input the real-time step frequency, real-time average acceleration, real-time maximum acceleration, real-time minimum acceleration, real-time acceleration difference, and real-time acceleration variance of the target vehicle into the long-distance step size model to obtain the PDR step size.

[0052] Step 5-3: Determine the step size constraint factor for fingerprint points within a circular region centered on the initial position of the target carrier and with the PDR step size as the radius, and determine the selection range of the magnetic field fingerprint database.

[0053] Step 5-4: Determine the PDR positioning result of the target vehicle based on the PDR step size, PDR heading information and the fused positioning result of the previous moment.

[0054] Step 6 includes:

[0055] Step 6-1: Use the trained convolutional neural network to extract multidimensional semantic features from each real-time motion 2D visual data.

[0056] Step 6-2: Determine VSLAM candidate fingerprint points based on multidimensional semantic features.

[0057] Step 6-3: Determine the image similarity weight of each VSLAM candidate fingerprint point based on the multidimensional semantic features corresponding to the VSLAM candidate fingerprint points.

[0058] Step 6-4: Based on the image similarity weight and coordinates of each VSLAM candidate fingerprint point, use the semantic VSLAM-assisted matching algorithm to determine the VSLAM matching and localization result of the target carrier.

[0059] Step 7 includes:

[0060] Step 7-1: Using the magnetic field matching positioning result as the positioning truth, obtain the magnetic field position deviation value between the candidate fingerprint point and the positioning truth at the current moment.

[0061] Step 7-2: Determine the weighted average of the magnetic field position deviation values ​​as the magnetic field positioning error.

[0062] Step 7-3: Using the VSLAM matching and localization result as the localization ground truth, obtain the VSLAM position deviation value between the VSLAM candidate fingerprint point and the localization ground truth at the current time.

[0063] Step 7-4: Determine the weighted average of the VSLAM position deviation values ​​as the VSLAM positioning error.

[0064] Step 7-5: Determine the real-time step size estimation error of the PDR positioning result and the estimation error of the long-distance step size model.

[0065] Step 7-6: Determine the maximum value of the real-time step size estimation error and the estimation error of the training long-distance step size model as the PDR positioning error.

[0066] Step 7-7: Using the Kalman filter algorithm, with the PDR positioning result as the state information and the magnetic field matching positioning result and VSLAM matching positioning result as the measurement information, construct the first Kalman filter system model.

[0067] Steps 7-8: Based on the magnetic field positioning error, VSLAM positioning error and PDR positioning error, solve the first Kalman filter system model to obtain the fusion positioning result of the target carrier.

[0068] Alternatively, step 7 includes:

[0069] Steps 7-9: Construct measurement posterior factors based on the magnetic field matching positioning results and VSLAM matching positioning results.

[0070] Steps 7-10: Using the Kalman filter algorithm, with the PDR positioning result as the state information and the magnetic field matching positioning result and VSLAM matching positioning result as the measurement information, the step size information and heading information in the PDR positioning result are added to the state equation, and the step size measurement value and heading vector measurement value of the measurement posterior factor are added to the measurement equation as measurement information to construct the second Kalman filter system model.

[0071] Steps 7-11: Solve the second Kalman filter system model to obtain the fusion localization result of the target carrier.

[0072] The passive fusion positioning method for indoor environments provided by this invention will be described in detail below, such as... Figures 2-3 This embodiment includes:

[0073] Step 101: Using the sensors mounted on the target vehicle, acquire real-time motion data of the target vehicle in an indoor environment, fingerprint database construction data, and long-distance step length model training data.

[0074] In practical applications, the real-time motion data of the target carrier specifically includes magnetic field data, two-dimensional visual data, and inertial sensor data of the target carrier in motion state in three-dimensional space. The tester carries the smart mobile device, which serves as the target carrier, and moves freely within the positioning area. With the help of sensors such as the visual sensor, magnetometer, and accelerometer mounted on the smart mobile device, the tester collects two-dimensional visual data, magnetic field data, and inertial sensor data of the target carrier in motion in three-dimensional space, respectively, as the real-time motion data of the target carrier in the positioning method of this invention.

[0075] In practical applications, the fingerprint database construction data specifically includes magnetic field fingerprint database construction data within a three-dimensional spatial positioning area and two-dimensional visual fingerprint database construction data. When collecting magnetic field fingerprint database construction data, considering that the grid spacing is equivalent to the magnetic field fingerprint database point density, and is positively correlated with the accuracy of the magnetic field matching algorithm but negatively correlated with construction costs, and that excessively large spacing can easily cause frequent anomalies in the magnetic field matching algorithm, this embodiment uses a grid with X and Y spacing of 0.80m to divide the indoor positioning area. The tester uses the same smart mobile device to collect magnetic field data at each grid vertex in the positioning area at 60-second intervals, while simultaneously recording the local coordinate information of the positioning area where the collection point is located, which is recorded as the collection result for that point.

[0076] The data collection was performed twice. The second data collection was used to verify that the magnetic field data of each grid vertex collected in the first collection was normal. If there was a significant difference between the two magnetic field data for a certain vertex, it was necessary to collect magnetic field data for that vertex and remove the abnormal data.

[0077] When collecting data for the construction of the 2D visual fingerprint database, indoor positioning areas are divided at 2.00m intervals along the road travel direction. Data collection points for the 2D visual fingerprint database are placed at the center of each interval, perpendicular to the travel direction. Using the same smart mobile device, testers take multiple sets of photos at each collection point at 9:00, 12:00, 18:00, and 22:00 on the same day, with the forward and reverse travel directions as the shooting directions and shooting heights of 1.80m, 1.70m, 1.60m, and 1.50m respectively. Simultaneously, the local coordinate information of the positioning area where the collection point is located is recorded, and this is recorded as the collection result for that collection point. This fingerprint database data does not require verification, and the collection process is performed only once.

[0078] It should be noted that the implementation environment of the embodiment includes some balconies and light-transmitting corridors. In a fully enclosed indoor environment where stable lighting is provided by lighting equipment, there is no need to consider the sampling time period. In areas where the lighting changes drastically, there is no need to collect data.

[0079] In practical applications, when collecting training data for the long-distance stride length model, considering the significant noise in stride length when the movement distance is too short, and the individual and device heterogeneity in the application of the stride length model, this embodiment uses different smart mobile devices to collect inertial sensor data from different testers. For any combination of testers and devices, inertial sensor data for different movement states such as slow walking, normal walking, fast walking, and running are collected for movement distances greater than 100m. Inertial data feature values ​​are extracted, and the actual displacement distance, time spent, number of steps taken, corresponding smart mobile device manufacturer, tester height, and other information are marked. The specific method for extracting inertial data feature values ​​from the long-distance stride length model training data is as follows:

[0080] like Figure 4 As shown, the motion step frequency is calculated from the time difference between two peak values, and the calculation formula is as follows:

[0081]

[0082] Among them, t fre Indicates step frequency, This represents the time of the i-th threshold detection point.

[0083] The acceleration difference is calculated from the difference between the maximum and minimum acceleration values ​​within one period, using the following formula:

[0084] a diff =a max -a min (2)

[0085] Among them, a diff This represents the difference in acceleration, a. max a min These represent the maximum and minimum values ​​of acceleration within the same period, respectively.

[0086] The mean acceleration is obtained by dividing the sum of the acceleration norms within one period by the number of accelerations within the period, as shown in the following formula:

[0087]

[0088] Among them, a mean This represents the average acceleration, and n represents the number of acceleration data points within the same period.

[0089] The acceleration variance is calculated from the acceleration variance over one period, as shown in the following formula:

[0090]

[0091] in, This represents the variance of acceleration.

[0092] It should be noted that in the long-distance step length model, each set of data collection results is the average step length, average step frequency, average maximum acceleration, average minimum acceleration, average mean acceleration and the difference between average acceleration within a distance of approximately 100m. For any combination of tester and smart mobile device, at least 5 sets of data collection should be guaranteed.

[0093] Step 102: Construct a magnetic field fingerprint database based on the data from the magnetic field fingerprint database, determine candidate fingerprint points by combining real-time motion magnetic field data and construct fusion weights, and use the WKNN algorithm based on the fusion weights to determine the magnetic field matching and positioning results of the target carrier.

[0094] In practical applications, step 102 specifically includes:

[0095] 1) Construct a magnetic field fingerprint database based on the magnetic field fingerprint database construction data in the fingerprint database construction data.

[0096] 2) Based on the changing trend of magnetic field data in the real-time motion data of the target carrier, candidate magnetic field fingerprints that conform to the gradient changing trend are selected in one step.

[0097] 3) Calculate the Euclidean distance between the magnetic field characteristics of the target carrier and the magnetic field characteristics of the screened fingerprint points to obtain the magnetic field similarity weight of the candidate fingerprint points. The calculation methods for the Euclidean distance between the magnetic field characteristics of the target carrier and the magnetic field characteristics of the screened fingerprint points, and the calculation methods for the magnetic field similarity weight of the candidate fingerprint points are shown in formulas (5) and (6), respectively.

[0098] The Euclidean distance between the magnetic field characteristics of the target carrier and the magnetic field characteristics of the screened fingerprint points is calculated as follows:

[0099]

[0100] Where, d magi mag represents the Euclidean distance of the i-th magnetic field fingerprint point. j MAG ij Let represent the j-th dimension feature of the measured magnetic field data and the j-th dimension feature of the ith magnetic field fingerprint point, respectively, and m represent the total number of magnetic field features.

[0101] The formula for calculating the magnetic field similarity weight of candidate fingerprint points is as follows:

[0102]

[0103] in, Denotes the magnetic field similarity weight of the i-th magnetic field fingerprint point, min(d mag ) represents the minimum Euclidean distance among the magnetic fingerprint points.

[0104] 4) Calculate the actual distance between the approximate location of the target carrier and the location of the candidate fingerprint point to obtain the inverse spatial distance weight of the candidate fingerprint point. The calculation methods for the actual distance between the approximate location of the target carrier and the location of the candidate fingerprint point, and the calculation methods for the inverse spatial distance weight of the candidate fingerprint point are shown in formulas (7) and (8), respectively.

[0105] The formula for calculating the actual distance between the approximate location of the target carrier and the location of the candidate fingerprint point is as follows:

[0106]

[0107] Where, d i This represents the actual distance between the candidate fingerprint point and the approximate location of the target carrier, (pos x pos y ), (x i y i ) represent the approximate position coordinates and the coordinates of the i-th candidate fingerprint point, respectively.

[0108] The formula for calculating the inverse spatial distance weight of candidate fingerprint points is as follows:

[0109]

[0110] in Let represent the inverse spatial distance weight of the i-th candidate fingerprint point, and min(d) represent the minimum spatial distance among the candidate fingerprint points.

[0111] 5) Calculate the product of the magnetic field similarity weight and the anti-spatial distance weight to determine the final weight for each candidate fingerprint point, and determine the magnetic field matching and localization result based on the WKNN algorithm. The method for calculating the fusion weight of candidate fingerprint points is shown in formula (9), and the WKNN algorithm is shown in formula (10).

[0112] Formula for calculating the fusion weight of candidate fingerprint points:

[0113]

[0114] Among them, w i This represents the fusion weight of candidate fingerprint points.

[0115] Based on the fingerprint fusion weight results, the WKNN algorithm is used to determine the magnetic field matching and positioning results. The WKNN algorithm formula is as follows:

[0116]

[0117] in, This indicates the magnetic field matching and positioning result, and represents the number of candidate points for the N magnetic field.

[0118] It is worth noting that, based on the changing trend of magnetic field data in the real-time motion data of the target carrier, the method of screening candidate magnetic field fingerprint points that conform to the gradient change trend should be executed at the beginning of the second magnetic field matching algorithm. Based on the fusion weight result of the previous time step, the weighted magnetic field features are interpolated to obtain the gradient points of the magnetic field fingerprint database at the previous time step. The calculation formula is as follows:

[0119]

[0120] in, Let j represent the j-th dimension feature of the gradient point of the magnetic field fingerprint database at the previous time step.

[0121] Based on the changing trend of the magnetic field data of the target carrier, candidate fingerprint points that match the changing trend of the gradient points in the magnetic field fingerprint database at the previous moment are selected. The selection logic is as follows:

[0122]

[0123] Among them, mag last mag curr These represent the magnetic field data of the target carrier at the previous and current moments, respectively.

[0124] Step 103: Calculate the original heading angle of the carrier based on the real-time motion magnetic field data and the real-time motion inertial sensor data, and use the adaptive window evaluation mean filtering algorithm to determine the PDR heading information of the target carrier.

[0125] In practical applications, step 103 specifically includes:

[0126] 1) Calculate the original heading angle of the carrier

[0127] The calculation of the original heading angle of the carrier requires obtaining the magnetic field data and inertial sensor data from the real-time motion data of the target carrier. The calculation of the original heading angle is shown in formula (23). The specific calculation steps include:

[0128] When at rest, the accelerometer readings satisfy the following relationship:

[0129]

[0130] And it satisfies the following relationship:

[0131]

[0132] in, This represents the three-axis components of acceleration in the carrier coordinate system. This represents the transformation matrix from the carrier coordinate system to the navigation coordinate system, where g represents the gravitational acceleration.

[0133] In this embodiment, the selected carrier coordinate system is front, right, and bottom. This can be represented as:

[0134]

[0135] Where γ, θ, and ψ represent the roll angle, pitch angle, and yaw angle, respectively.

[0136] Similarly, we get:

[0137]

[0138] The accelerometer readings on all three axes due to gravity. It can be represented as:

[0139]

[0140] The initial roll angle, pitch angle, and heading angles γ0 and θ0 caused by gravity can be expressed as:

[0141]

[0142]

[0143] Find the rotation matrix of the target vehicle coordinate system to rotate to the horizontal plane. have:

[0144]

[0145] Triaxial readings of magnetic field under horizontal plane The following relationship must be satisfied:

[0146]

[0147] The heading angle ψ below the horizontal plane can be expressed as:

[0148]

[0149] After compensating for the geographic magnetic declination Δψ, the original heading angle It can be represented as:

[0150]

[0151] 2) Construct an adaptive window evaluation mean filtering algorithm

[0152] Given an initial adaptive window size ≥ 3, fill the adaptive window with the first three raw heading angles to ensure sufficient data support for the initial evaluation within the adaptive window. When obtaining the fourth raw heading angle, estimate the standard error of the raw heading angles within the adaptive window, where σ is the standard error of the raw heading angles. angle The calculation formula is as follows:

[0153]

[0154] Where M represents the total number of original heading angles within the adaptive window, and A i , Let i and n represent the i-th original heading angle and the mean heading angle within the adaptive window, respectively.

[0155] Based on the mean square error of the original heading angle within the adaptive window, three regions—acceptable region, danger region, and extreme region—are defined, with one and three times the mean square error as boundaries, respectively. The corresponding region decision is executed based on the region where the heading angle data for the next moment is located. The corresponding processing methods are: allowing entry into the window, allowing entry into the window and counting, and prohibiting entry into the window and counting. Resetting the adaptive window when the count reaches a certain amount reduces the impact of the heading angle filter on the steering data. The heading information of the PDR algorithm is used as the mean filtering result of the adaptive window. The logical expression is as follows:

[0156]

[0157] Where Size represents the number of original heading angles within the adaptive window, and A new This indicates the heading angle data for the next moment. Count represents the count of large-fluctuation heading angle data. In indicates that heading angle data is allowed to enter the adaptive window. Out indicates that heading angle data is not allowed to enter the adaptive window. Reset indicates that the adaptive window is reset to its initial state.

[0158] 3) Determine PDR heading information

[0159] Based on the adaptive window evaluation mean filtering algorithm, after obtaining the original heading angle, regardless of the state of the adaptive window, the mean heading angle within the adaptive window is used as the PDR heading information. PDR heading information A PDR The calculation formula is as follows:

[0160]

[0161] Step 104: Based on the training data of the long-distance step size model, train the basic deep learning model fully connected neural network model to obtain the long-distance step size model, and construct the step size constraint factor based on the real-time motion sensor data and the long-distance step size model, and determine the PDR positioning result of the target vehicle by combining the PDR heading information.

[0162] In practical applications, step 104 specifically includes:

[0163] 1) Training a long-distance stride model

[0164] Based on the training data of the long-range step size model, a deep learning step size model is trained using a basic deep learning model, a fully connected neural network model. For example... Figure 5 As shown, there are N sets of training datasets, with N-1 sets used as the training dataset and the remaining set as the test dataset. The input features include six features in one step motion cycle: step frequency, mean acceleration, maximum acceleration, minimum acceleration, acceleration difference, and acceleration variance. The output feature is the step length.

[0165] In model construction, three fully connected layers are used, containing 16, 8, and 1 neurons respectively. One layer amplifies data features for machine learning extraction. ReLU is used as the activation function for inter-layer transfer mapping. The mean squared error loss function is used, and the Adam gradient descent optimization method is selected. During testing, an additional test set missing a certain feature should be added to verify whether the model has irrelevant features. If so, the irrelevant features are removed, and the long-range stride model is retrained, resulting in the following long-range stride model:

[0166]

[0167] 2) Determine the PDR step frequency and PDR step size

[0168] Based on inertial sensor data from the real-time motion data of the target carrier, according to Figure 4 and Figure 5 The diagram shows the inertial data feature extraction method, which obtains feature values ​​of the carrier, including step frequency, maximum acceleration, minimum acceleration, acceleration difference, average acceleration, and acceleration variance. The PDR step size is determined based on the feature values ​​and the long-distance step size model.

[0169] 3) Construct step size constraint factors

[0170] Depend on Figure 6 As shown, based on the PDR step size, a circle is drawn with the approximate position of the target carrier as the center and the PDR step size as the radius to construct the step size constraint factor, which constrains the selection range of the fingerprint database for the magnetic field matching algorithm. Formula (7) satisfies the following conditions:

[0171]

[0172] in, This represents the approximate position at time i. Let represent the estimated step size at time i.

[0173] 4) Determine the PDR positioning results

[0174] Based on the PDR step frequency, step length, and heading information, the PDR positioning result is calculated using the following formula:

[0175]

[0176] Step 105: Construct a two-dimensional visual fingerprint database by training a convolutional neural network based on the data from the two-dimensional visual fingerprint database, and determine the VSLAM matching and localization result of the target carrier by using a semantic VSLAM-assisted matching algorithm based on real-time motion two-dimensional visual data.

[0177] In practical applications, step 105 specifically includes:

[0178] 1) Construct a two-dimensional visual fingerprint database

[0179] like Figure 7 As shown, if the camera pixel count of the intelligent shooting device is poor, noise reduction processing is required on the two-dimensional visual fingerprint database construction data in the fingerprint database construction data to improve image clarity.

[0180] Convolutional neural networks are used to extract semantic features from images. These semantic features can be divided into visual, object, and conceptual layers: the visual layer generally refers to the image's color, texture, and shape; the object layer typically contains semantic features of things that frequently appear or exist in the data collection location; and the conceptual layer generally refers to the area where the image is located, such as a corridor or hallway. In this embodiment, the semantic features of the object layer are mainly divided into fixed landmarks that can serve as location features and randomly appearing objects that interfere with location judgment. The former mainly includes walls, indoor support columns, and tables and chairs that generally do not move, while the latter mainly includes constantly moving pedestrians.

[0181] During training, the model is first trained to extract semantic features at the object layer. Given images of pedestrians, walls, tables, etc., a convolutional neural network is used for convolution and pooling to gradually extract high-dimensional features from the images. The final output is then connected to a Softmax classification layer. Object recognition supervision is used to determine whether the classification result corresponds to the semantic information of the corresponding image in the training data. After the object recognition classification training results are satisfactory and verified on a test dataset, achieving an accuracy rate higher than 90%, the object recognition classification result is then connected to another Softmax classification layer to extract the concept layer, with concept recognition supervision performed again. After extracting the semantic features of the image object layer and concept layer, the traditional VSLAM matching method is used to segment the image based on its color, texture, and shape features. A fingerprint point can be obtained from the three layers of semantic features of each image and the corresponding local coordinate information. This method is used to construct a two-dimensional visual fingerprint database.

[0182] 2) Constructing a semantic VSLAM matching algorithm

[0183] The semantic VSLAM matching algorithm serves as an auxiliary localization source, requiring only the input of real-time two-dimensional visual data of the target carrier to determine the semantic VSLAM matching localization result. Before localization begins, the two-dimensional visual fingerprint database is read. During the initial localization process and when additional localization sources are needed, the semantic features in the real-time image are extracted using a model trained by a convolutional neural network based on the two-dimensional visual data in the real-time motion data of the target carrier. In the semantic matching process, candidate points matching the real-time semantic concept layer are selected in one step, and candidate points matching the real-time semantic object layer are further selected from the candidate points. Finally, the Euclidean distance of the real-time image visual layer is calculated using formula (30) to obtain the image similarity weight, and the VSLAM matching localization result is calculated using formula (31).

[0184] Image similarity calculation formula:

[0185]

[0186] in, This represents the similarity weight of the i-th image. VSLAM represents the Euclidean distance of the i-th candidate fingerprint point. j VSLAM ij Let represent the j-th dimension features of the real-time image visual layer and the candidate fingerprint point image visual layer, respectively.

[0187] VSLAM matching and localization result calculation formula:

[0188]

[0189] in, This indicates the VSLAM matching and localization results. This indicates the coordinates of the candidate fingerprint points.

[0190] Step 106: Based on the magnetic field matching positioning results, PDR positioning results, and VSLAM matching positioning results, construct a noise evaluation factor, perform filtering anomaly decision judgment, and determine the fusion positioning result of the target carrier.

[0191] In practical applications, step 106 specifically includes:

[0192] 1) Constructing a noise evaluation factor for the magnetic field matching algorithm

[0193] Using the magnetic field matching positioning result as the true positioning value, the positional deviation between all candidate points selected by the magnetic field matching algorithm at the current moment and the true positioning value is calculated. The weighted result of the deviation values ​​is used as the positioning error of the magnetic field matching algorithm. The specific formula is as follows:

[0194]

[0195]

[0196] in, These represent the positioning errors in the x-direction and y-direction of the magnetic field matching algorithm, respectively.

[0197] 2) Constructing a noise evaluation factor for the semantic VSLAM-assisted matching algorithm

[0198] Using the VSLAM matching localization result as the true localization value, the positional deviation between all candidate points selected by the VSLAM matching algorithm at the current time and the true localization value is calculated. The weighted result of the deviation values ​​is used as the localization error of the VSLAM matching algorithm. The specific formula is as follows:

[0199]

[0200] in, These represent the positioning errors in the x-direction and y-direction of the magnetic field matching algorithm, respectively.

[0201] 3) Constructing the noise evaluation factor for the PDR algorithm

[0202] Based on the characteristic of PDR positioning error accumulating over time, it can be divided into errors caused by heading estimation, errors caused by step frequency detection, and errors caused by step length estimation. In this embodiment, the error caused by step frequency detection is negligible due to the high accuracy of the step frequency detection method and the difficulty in estimating the step frequency detection accuracy in real time; the error caused by heading estimation is obtained by an adaptive window evaluation mean filtering algorithm.

[0203] For the error caused by step size estimation, based on the mean squared error loss function given by the training model of the long-distance step size model, the long-distance step size model estimation error σ is... tstep The calculation method can be expressed as:

[0204]

[0205] Among them, L i , Let represent the actual step size and the estimated step size of the i-th step, respectively, and Test represent the number of test datasets.

[0206] In real-time estimation, if at some point the long-range step size model estimation result is greater than 3σ... tstep If the step size result at that time step is discarded, the step size result at the previous time step is used instead, and the real-time step size estimation error σ rstep The calculation method can be expressed as:

[0207]

[0208] The maximum value between the real-time step size estimation error and the training distance step size model estimation error is taken as the PDR step size estimation error σ.step The calculation method can be expressed as:

[0209] σ step =max(σ tstep , σ rstep (37)

[0210] Without measurement updates, continuous PDR positioning suffers from error accumulation. Based on the number of steps recorded by gait frequency detection and the measurement correction status of the Kalman filter, the cumulative error of the PDR algorithm is estimated. Based on the PDR heading estimation error, PDR step size estimation error, and PDR continuous positioning count, the cumulative variance of the PDR algorithm can be expressed as:

[0211]

[0212] Here, count represents the number of consecutive localizations achieved by the PDR algorithm without correction.

[0213] 4) Filtering anomaly decision judgment

[0214] Special cases such as fingerprint databases of different densities or heterogeneous fingerprint database construction and positioning devices can easily lead to algorithm anomalies. Specifically, this may manifest as no magnetic field matching results or small differences in magnetic field matching positioning results. Based on these anomalies, corresponding processing decisions are set, and the logical expression is as follows:

[0215]

[0216] Here, limit represents the threshold for judging significant changes in the magnetic field.

[0217] When the target carrier stops moving, the trend of the magnetic field data will gradually disappear. The limit should be a sufficiently small value to determine whether the results of the magnetic field matching algorithm have not changed. In this invention example, the limit is set as follows:

[0218]

[0219] 5) Construct a Kalman filter system to determine the target carrier fusion positioning result.

[0220] Using the Kalman filter algorithm, with the PDR algorithm positioning result as the state information and the magnetic field matching positioning result / semantic VSLAM matching positioning result as the measurement information, taking the magnetic field matching positioning result as an example, the Kalman filter system is modeled as follows:

[0221]

[0222] in, These represent the posterior state estimates at time k-1 and time k, respectively, i.e., the fused filtering position results: P represents the prior state estimate at time k, i.e., the localization result of the PDR algorithm; k-1 P k These represent the posterior estimated covariances at time k-1 and time k, respectively, i.e., the results of the fusion filter weights. z represents the prior estimate covariance at time k, i.e., the covariance of the localization result from the PDR algorithm; k This represents the measured value, i.e., the result of magnetic field matching and positioning; K k Let A and B represent the filter gain matrix at time k; A and B represent the state coefficient matrices, i.e., the mathematical model of the PDR algorithm; H represents the measurement coefficient matrix, i.e., the mathematical model of the magnetic field matching algorithm; Q represents the process excitation noise covariance, i.e., the positioning error of the PDR algorithm; and R represents the measurement noise covariance, i.e., the positioning error of the magnetic field matching algorithm.

[0223] The initial values ​​of the x0 and P0 matrices for the Kalman filter are defined as follows, and the update of the matrix z is defined as follows:

[0224]

[0225] The specific calculation method for the A, B, H, Q, and R matrices of the Kalman filter is expressed as follows:

[0226]

[0227] Step 107: Based on multiple sets of magnetic field matching positioning results and VSLAM matching positioning results, construct a measurement posterior factor, obtain the carrier motion state and attitude measurement values, and construct a Kalman filter algorithm containing step length and flight vector measurements to constrain and fuse the positioning results.

[0228] In practical applications, step 107 specifically includes:

[0229] 1) Constructing the posterior factor of the measurement

[0230] The recorded measurement matching results are processed using a polynomial fitting method to construct the measurement posterior factor. Specifically, this includes obtaining the carrier's flight vector measurement value by differentiating the slope of the regression curve or the approximate position; and obtaining the carrier's step length measurement value based on the cumulative value of the displacement distance of the measurement matching results and the step frequency detection results.

[0231] To ensure the reliability of the polynomial fitting results, given that the regression analysis results have good reliability when the number of measurement records Num ≥ 15, the positional information of the measurement results at the first Num time steps is recorded. The recorded positional information of the measurement results is then fitted using a polynomial fitting method to obtain the fitting curve. If the fitting curve is significantly linear with a first-order term, the flight vector measurement value is obtained by calculating the arctangent function from the forward direction and the slope. If the fitting curve is significantly quadratic or higher, the flight vector measurement value is obtained by calculating the arctangent function by differentiating the forward direction and the approximate position of the vehicle as the tangent point. The specific logic can be expressed as follows:

[0232]

[0233] Where f represents the fitted curve, n represents the degree of the polynomial of the fitted curve, and A regress This represents the measured value of the flight vector, which needs to be compensated by ±π based on the direction of travel.

[0234] It is important to note that the selection of independent and dependent variables must be considered in polynomial fitting. To avoid outlier, the selection of the independent variable Indep is determined by the coordinate elements that show the most significant changes in the measurement results. In this embodiment, the specific logic can be expressed as follows:

[0235]

[0236] In this example, the original heading angle is calculated by calculating the angle between the forward direction and the positive x-axis. When the independent variable is y, the angle of the fitted curve needs to be compensated to be based on the positive x-axis as the starting direction, as follows:

[0237]

[0238] The cumulative displacement distance of the measurement matching results is obtained by summing the distances between two points at adjacent moments in the measurement matching results. The step length measurement value is allocated according to the inverse proportion of the step frequency corresponding to each step in the total movement time in the step frequency detection results, that is, the faster the step frequency, the longer the step length, and the slower the step frequency, the shorter the step length. The step length measurement value can be calculated as follows:

[0239]

[0240] Among them, L regress W represents the step size measurement value at step i. i D represents the corresponding inverse step frequency weight. regress This represents the cumulative value of the displacement distance.

[0241] It should be noted that the measurement posterior factor is not suitable for irregular turning movements. The current record should be appropriately reduced or even reset based on the impact of the newly recorded measurement results on the fitted curve. In this embodiment, a confidence level of 0.0027 is used; if any dependent variable has a residual... i For measurement results exceeding the confidence interval, the current record is reset. The specific logic can be expressed as follows:

[0242]

[0243] Among them, Indep i Depen i Let i and j represent the i-th independent variable and dependent variable in the measurement results, respectively.

[0244] 2) Construct a Kalman filter algorithm that includes step size and course vector measurements to determine the target vehicle's fusion positioning result.

[0245] In constructing a Kalman filter algorithm containing step size and heading measurements, the step size and heading information estimated by PDR are added to the state equation, and the step size and heading measurements provided by the measurement posterior factor are added to the measurement equation as measurement information, resulting in a Kalman filter system containing step size and heading measurements. At this point, the Kalman filter definition matrices x, P, z are:

[0246]

[0247] The specific calculation method for the A, B, H, Q, and R matrices of the Kalman filter is expressed as follows:

[0248]

[0249] in, This represents the variance of the step size check value. The variance of the heading check value can be expressed as follows:

[0250]

[0251]

[0252] in, This represents the step size check value calculated by Num-1 and the predicted step size value for each step obtained from the Num step frequency. This represents the step size in the Num calculation. This represents the heading check value calculated from Num-1. This represents the heading check value calculated from Num.

[0253] When a measurement posterior factor exists, the Kalman filter system constructed in step 106 is replaced with the form in step 107: Substitute formulas (49) and (50) into formula (41), and the filtered output result is the target carrier positioning result.

[0254] This invention leverages the advantages of readily available and environmentally adaptable passive information such as magnetic field and inertial information to achieve a geomagnetic / PDR passive fusion positioning technology based on the Kalman filter algorithm. It utilizes the absolute positioning capability of magnetic field matching to eliminate accumulated errors in PDR positioning technology and adds semantic VSLAM matching measurements to expand the feature dimension, reducing the impact of insufficient feature dimensions and accelerating positioning initialization. Furthermore, it estimates the positioning errors of the magnetic field matching algorithm, semantic VSLAM matching algorithm, and PDR algorithm in real time, determines the Q and R noise matrices in the Kalman filter system, and provides heading and step size measurements to make the model more consistent with the actual problem. Simultaneously, it employs various anomaly decision-making methods to enhance system robustness. This invention provides a low-cost, high-precision, and highly robust passive positioning technology for indoor environments by utilizing multiple sensors in intelligent devices to increase information sources, improving existing positioning algorithms, and supplementing them with various anomaly decision-making methods, without significantly increasing positioning costs.

[0255] Example 2

[0256] In order to perform the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a passive fusion positioning system for indoor environments is provided below, including:

[0257] The data acquisition module is used to acquire real-time motion data, fingerprint database construction data, and long-distance step length model training data of the target carrier in an indoor environment. The real-time motion data, fingerprint database construction data, and long-distance step length model training data are all acquired by various sensors mounted on the target carrier. The real-time motion data includes real-time motion magnetic field data, real-time motion inertial sensor data, and real-time motion two-dimensional visual data. The fingerprint database construction data includes magnetic field fingerprint database construction data and two-dimensional visual fingerprint database construction data.

[0258] The magnetic field matching and positioning result determination module is used to construct fusion weights based on real-time motion magnetic field data and magnetic field fingerprint database data, and use the WKNN algorithm based on the fusion weights to determine the magnetic field matching and positioning result of the target carrier.

[0259] The PDR heading information determination module is used to determine the PDR heading information of the target vehicle based on real-time motion magnetic field data and real-time motion inertial sensor data, using an adaptive window evaluation mean filtering algorithm.

[0260] The long-distance step size model training module is used to train the basic deep learning model, the fully connected neural network model, using long-distance step size model training data to obtain the long-distance step size model.

[0261] The PDR positioning result determination module is used to determine the PDR positioning result of the target vehicle based on real-time motion sensor data, PDR heading information and long-distance step model.

[0262] The VSLAM matching and localization result determination module is used to construct data based on real-time motion 2D visual data and a 2D visual fingerprint database, and to determine the VSLAM matching and localization result of the target carrier using a semantic VSLAM-assisted matching algorithm.

[0263] The fusion positioning result determination module is used to constrain the fusion positioning result of the target carrier based on the magnetic field matching positioning result, PDR positioning result, and VSLAM matching positioning result.

[0264] like Figure 8 The fusion positioning system provided in this embodiment includes:

[0265] The multi-source data integration module is used to acquire real-time motion data of the target carrier in an indoor environment, fingerprint database construction data, and long-distance step length model training data using the sensors mounted on the target carrier.

[0266] The magnetic field matching algorithm module is used to construct a magnetic field fingerprint database using data from the magnetic field fingerprint database, determine candidate fingerprint points by combining real-time motion magnetic field data and construct fusion weights, and use the WKNN algorithm based on the fusion weights to determine the magnetic field matching and positioning results of the target carrier.

[0267] The heading angle filtering module is used to calculate the original heading angle of the carrier using real-time motion magnetic field data and real-time motion inertial sensor data, and to determine the PDR heading information of the target carrier using an adaptive window evaluation mean filtering algorithm.

[0268] The PDR algorithm module is used to train the basic deep learning model, the fully connected neural network model, using long-distance step size model training data to obtain the long-distance step size model. It also constructs step size constraint factors based on real-time motion sensor data and the long-distance step size model, and determines the PDR positioning result of the target vehicle by combining PDR heading information.

[0269] The VSLAM matching algorithm module is used to train a convolutional neural network to construct a two-dimensional visual fingerprint database using data from a two-dimensional visual fingerprint database, and to determine the VSLAM matching and localization result of the target carrier based on real-time motion two-dimensional visual data using a semantic VSLAM-assisted matching algorithm.

[0270] The Kalman filter module is used to construct a noise assessment factor using the magnetic field matching positioning results, PDR positioning results, and VSLAM matching positioning results, and to perform filtering anomaly decision judgment to determine the fusion positioning result of the target carrier.

[0271] The measurement posterior module is used to construct measurement posterior factors using multiple sets of magnetic field matching positioning results and VSLAM matching positioning results, obtain the carrier motion state and attitude measurement values, and construct a Kalman filter algorithm containing step size and flight vector measurements to constrain and fuse the positioning results.

[0272] Step 701, the multi-source data integration module, specifically includes:

[0273] The signal receiving chip and sensor integration unit are used to receive signals from active environmental devices and acquire observation data such as visual and magnetic field data of the surrounding environment of the target carrier and the carrier's own inertial data.

[0274] The data storage unit is used to store the data acquired by the signal receiving chip and the sensor integration unit, as well as the calculation results of the algorithm units of other modules in the system.

[0275] In practical applications, the signal receiving chip and sensor integration unit and the data storage unit can be integrated into one smart device, or into two smart devices capable of network information interaction. The signal receiving chip and sensor integration unit is generally mounted on the target carrier that needs to obtain location services, including a series of portable or wearable smart devices such as smartphones, tablets, and smartwatches, to acquire observation data of the user and the surrounding environment. The data storage unit can be located in a mobile smart device with sufficient storage capacity, or it can be a cloud server, local area network server, or local server with certain computing and storage capabilities, capable of real-time data interaction with the user's smart device, to store the data acquired by the signal receiving chip and sensor integration unit and the calculation results of the algorithm units of other modules in the system.

[0276] Specifically, step 702, the magnetic field matching algorithm module, includes:

[0277] The magnetic field gradient screening unit is used to screen candidate magnetic field fingerprints that conform to the gradient change trend in one step based on the changing trend of magnetic field data in the real-time motion data of the target carrier.

[0278] The magnetic field similarity weighting unit is used to calculate the Euclidean distance between the magnetic field characteristics of the target carrier and the magnetic field characteristics of the screened fingerprint points, and to obtain the magnetic field similarity weight of the candidate fingerprint points.

[0279] The inverse spatial distance weighting unit is used to calculate the actual distance between the approximate position of the target carrier and the position of the candidate fingerprint point, and to obtain the inverse spatial distance weight of the candidate fingerprint point.

[0280] The magnetic field matching unit is used to calculate the product of the magnetic field similarity weight and the inverse spatial distance weight, to finally determine the weight of each candidate fingerprint point, and to determine the magnetic field matching localization result based on the WKNN algorithm.

[0281] Specifically, step 703, the heading angle filtering module, includes:

[0282] The original heading angle calculation unit is used to calculate the original heading angle of the carrier based on the magnetic field data and inertial sensor data in the real-time motion data of the target carrier;

[0283] The adaptive window evaluation unit is used to evaluate the data dispersion within the adaptive window. It divides the window into three regions: the acceptance region, the danger region, and the extreme region, using one time error and three times error as boundaries. It then performs corresponding region decisions based on the region where the heading angle data is located at the next moment, and adds, deletes, and resets the adaptive window.

[0284] The mean filtering unit is used to evaluate the filtering algorithm based on the adaptive window and determine the PDR heading information based on the mean heading angle within the adaptive window.

[0285] Specifically, step 704, the PDR algorithm module, includes:

[0286] The model training unit is used to train a basic deep learning model, a fully connected neural network model, based on long-distance step size model training data, to obtain a step size model for long-distance motion.

[0287] The inertial feature extraction unit is used to obtain feature values ​​of the target carrier, including step frequency, maximum acceleration, minimum acceleration, acceleration difference, acceleration mean, and acceleration variance, based on inertial sensor data in the real-time motion data of the target carrier. The PDR step size is determined based on the feature values ​​and the long-distance step size model.

[0288] The step size constraint factor construction unit is used to construct a circle with the approximate position of the target carrier as the center and the PDR step size as the radius, and to constrain the selection range of the fingerprint database for the magnetic field matching algorithm.

[0289] The PDR positioning unit is used to determine the PDR positioning result based on the PDR step frequency, step length, and heading information.

[0290] Specifically, step 705, the VSLAM matching algorithm module, includes:

[0291] The semantic extraction training unit is used to extract semantic features from the visual layer, object layer and concept layer of an image using a convolutional neural network, and to build a two-dimensional visual fingerprint database.

[0292] The semantic VSLAM matching unit is used to extract semantic features from real-time images based on two-dimensional visual data in real-time motion data of the target carrier, using a model trained by a convolutional neural network, to screen candidate fingerprint points that match the real-time semantic concept layer and object layer, and to calculate the Euclidean distance between the candidate fingerprint points and the real-time image visual layer to obtain image similarity weights. The VSLAM matching and localization results are then determined by the WKNN algorithm.

[0293] In practical applications, the VSLAM algorithm matching module is an auxiliary observation source used for system initialization and positioning periods requiring additional observation sources. Smart device users provide two-dimensional visual data to the data storage unit by taking and uploading photos, and then the data storage unit transmits the data to the VSLAM matching module. After the module calculates the candidate point matching results, it sends the results back to the user's smart device.

[0294] In practical applications, the VSLAM matching module transmits location results in real time based on the uploaded 2D visual data. Users can choose whether to continue taking photos and uploading or end VSLAM-assisted positioning by observing the location information displayed on the visual map. Therefore, unlike the PDR algorithm module and the magnetic field matching algorithm module, the VSLAM matching algorithm module is a passive positioning module. It is activated and provides positioning results by whether the user uploads 2D visual data, and does not require the smart device's visual sensor to be in an active state for a long time.

[0295] Specifically, step 706, the Kalman filter module, includes:

[0296] The magnetic field matching error estimation unit is used to calculate the position deviation between all candidate points selected by the magnetic field matching algorithm and the positioning truth value at the current moment, using the magnetic field matching positioning result as the positioning truth value, and to estimate the positioning error of the magnetic field matching algorithm by the weighted result of the deviation value.

[0297] The VSLAM matching error estimation unit is used to calculate the position deviation between all candidate points selected by the VSLAM matching algorithm at the current time and the true positioning value, using the VSLAM matching positioning result as the true positioning value, and to estimate the VSLAM matching algorithm positioning error by the weighted result of the deviation value.

[0298] The PDR positioning error estimation unit is used to determine the PDR heading estimation error by evaluating the mean filtering algorithm with an adaptive window, to determine the PDR step size estimation error by the training model loss function and the real-time step size estimation error, and to determine the PDR algorithm accumulation by the PDR heading estimation error, the PDR step size estimation error and the PDR continuous positioning count.

[0299] Anomaly decision unit is used to take into account the heterogeneity of fingerprint database and device heterogeneity, and to set corresponding processing decisions for abnormal results such as no magnetic field matching results or small differences in magnetic field matching positioning results;

[0300] The Kalman filter unit is used to construct a Kalman filter system by using the PDR algorithm positioning results as state information and the magnetic field matching positioning results / semantic VSLAM matching positioning results as measurement information, and to determine the target carrier fusion positioning result.

[0301] Step 707, the measurement post-test module, specifically includes:

[0302] The measurement posterior factor construction unit is used to perform polynomial fitting on the recorded measurement matching results using a polynomial fitting method to construct the measurement posterior factor. The carrier aeronautical vector measurement value is obtained by differentiating the slope of the regression curve or the approximate position. The carrier step length measurement value is obtained based on the cumulative value of the displacement distance of the measurement matching results and the step frequency detection results.

[0303] The measurement posterior Kalman filter unit is used to add the step size and heading information estimated by PDR into the state equation. The step size measurement value and heading vector measurement value provided by the measurement posterior factor are added into the measurement equation as measurement information to construct a Kalman filter algorithm containing step size and heading vector measurement to determine the target vehicle fusion positioning result.

[0304] In practical applications, the magnetic field matching algorithm module, heading angle filtering module, PDR algorithm module, VSLAM matching module, measurement posterior module, and Kalman filter module are generally integrated with the data storage unit in the same electronic device. This facilitates real-time and rapid data interaction between the data storage unit and the algorithm units of other modules in the system. The electronic device can be a personal computer, laptop, smartphone, PAD, or other device with certain computing and processing capabilities. Over short distances, data can be transmitted between the positioning carrier and the storage and computing unit via wired data or local area network connections. For remote devices such as cloud servers, a device with high mobile internet performance is required to interact with the target carrier in real time, reducing the lag in positioning results caused by data transmission congestion and delays.

[0305] Example 3

[0306] This embodiment provides an electronic device, including a memory and a processor. The memory is used to store computer programs, and the processor runs the computer programs to enable the electronic device to execute a passive fusion positioning method in an indoor environment according to Embodiment 1.

[0307] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0308] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A passive fusion positioning method for indoor environments, characterized in that, include: Acquire real-time motion data of the target carrier in an indoor environment, fingerprint database construction data, and long-distance stride model training data; The real-time motion data, fingerprint database construction data, and long-distance step length model training data are all acquired by various sensors mounted on the target carrier; the real-time motion data includes real-time motion magnetic field data, real-time motion inertial sensor data, and real-time motion two-dimensional visual data; the fingerprint database construction data includes magnetic field fingerprint database construction data and two-dimensional visual fingerprint database construction data. A fusion weight is constructed based on the real-time motion magnetic field data and the magnetic field fingerprint database, and the magnetic field matching and positioning result of the target carrier is determined using the WKNN algorithm based on the fusion weight. Based on the real-time motion magnetic field data and the real-time motion inertial sensor data, the PDR heading information of the target carrier is determined using an adaptive window evaluation mean filtering algorithm. Using training data from the long-range step size model, a basic deep learning model, a fully connected neural network model, is trained to obtain the long-range step size model. Based on the real-time motion inertial sensor data, the PDR heading information, and the long-distance step size model, the PDR positioning result of the target vehicle is determined; Based on the real-time motion 2D visual data and the 2D visual fingerprint database, the VSLAM matching and localization results of the target carrier are determined using the semantic VSLAM-assisted matching algorithm. Based on the magnetic field matching positioning results, the PDR positioning results, and the VSLAM matching positioning results, the constrained target carrier's fusion positioning results include: Using the magnetic field matching and positioning result as the positioning truth, the magnetic field position deviation value between the current magnetic field candidate fingerprint point and the positioning truth is obtained; The weighted average of the magnetic field position deviation values ​​is determined as the magnetic field positioning error; Using the VSLAM matching and localization result as the localization truth, the VSLAM position deviation value between the VSLAM candidate fingerprint point and the localization truth value at the current time is obtained; The weighted average of the VSLAM position deviation values ​​is determined as the VSLAM positioning error; Determine the real-time step size estimation error and the long-distance step size model estimation error of the PDR positioning results; The maximum value of the real-time step size estimation error and the long-distance step size model estimation error is determined to be the PDR positioning error; Using the Kalman filter algorithm, with the PDR positioning result as the state information and the magnetic field matching positioning result and the VSLAM matching positioning result as the measurement information, a first Kalman filter system model is constructed. Based on the magnetic field positioning error, the VSLAM positioning error, and the PDR positioning error, the first Kalman filter system model is solved to obtain the fusion positioning result of the target carrier.

2. The passive fusion positioning method for indoor environments according to claim 1, characterized in that, Based on the real-time motion magnetic field data and the magnetic field fingerprint database, a fusion weight is constructed, and the WKNN algorithm based on the fusion weight is used to determine the magnetic field matching and positioning result of the target carrier, including: Based on the real-time motion magnetic field data trend and the data trend constructed from the magnetic field fingerprint database, candidate magnetic field fingerprint points are determined; The magnetic similarity weight of each magnetic candidate fingerprint point is determined based on the Euclidean distance between the magnetic field characteristics of the target carrier and the magnetic field characteristics of the magnetic field candidate fingerprint points. The inverse spatial distance weight of each magnetic field candidate fingerprint point is determined based on the actual distance between the initial position of the target carrier and the position of the magnetic field candidate fingerprint point. The product of the magnetic field similarity weight and the anti-spatial distance weight of the same magnetic field candidate fingerprint point is determined as the fusion weight of the magnetic field candidate fingerprint point; Based on the fusion weights of the candidate magnetic field fingerprints and the coordinates of multiple candidate magnetic field fingerprints, the WKNN algorithm is used to determine the magnetic field matching and positioning results of the target carrier.

3. The passive fusion positioning method for indoor environments according to claim 1, characterized in that, Based on the real-time motion magnetic field data and the real-time motion inertial sensor data, the PDR heading information of the target vehicle is determined using an adaptive window evaluation mean filtering algorithm, including: Based on the real-time motion magnetic field data and the real-time motion inertial sensor data, the original heading angle sequence of the target carrier is determined; The original heading angle sequence of the target vehicle is logically judged using an adaptive window evaluation mean filtering algorithm, and adaptive window mean filtering is performed to obtain PDR heading information; the PDR heading information is the mean of multiple original heading angles within the adaptive window.

4. The passive fusion positioning method for indoor environments according to claim 1, characterized in that, Using training data from the long-range stride model, a basic deep learning model, a fully connected neural network, is trained to obtain the long-range stride model, which includes: Construct a basic deep learning model: a fully connected neural network model; Using historical step frequency, historical mean acceleration, historical maximum acceleration, historical minimum acceleration, historical acceleration difference, and historical acceleration variance as inputs, and historical step length as output, a basic deep learning model, a fully connected neural network model, is trained to obtain a long-distance step length model.

5. The passive fusion positioning method for indoor environments according to claim 1, characterized in that, Based on the real-time motion inertial sensor data, the PDR heading information, and the long-distance step size model, the PDR positioning result of the target vehicle is determined, including: Based on real-time motion inertial sensor data, determine the target vehicle's real-time step frequency, real-time mean acceleration, real-time maximum acceleration, real-time minimum acceleration, real-time acceleration difference, and real-time acceleration variance. The real-time step frequency, real-time average acceleration, real-time maximum acceleration, real-time minimum acceleration, real-time acceleration difference, and real-time acceleration variance of the target vehicle are all input into the long-distance step size model to obtain the PDR step size. The step size constraint factor is constructed by determining the fingerprint points within a circular area centered on the initial position of the target carrier and with the PDR step size as the radius, and the selection range of the magnetic field fingerprint database is determined. Based on the PDR step size, the PDR heading information, and the fused positioning result of the previous moment, the PDR positioning result of the target vehicle is determined.

6. The passive fusion positioning method for indoor environments according to claim 1, characterized in that, Based on the real-time motion 2D visual data and the 2D visual fingerprint database, the VSLAM matching and localization results of the target carrier are determined using a semantic VSLAM-assisted matching algorithm, including: The trained convolutional neural network is used to extract multidimensional semantic features from each real-time motion 2D visual data. Based on the multidimensional semantic features, VSLAM candidate fingerprint points are determined; Based on the multidimensional semantic features corresponding to the VSLAM candidate fingerprint points, determine the image similarity weight of each VSLAM candidate fingerprint point; Based on the image similarity weights and coordinates of each VSLAM candidate fingerprint point, the VSLAM matching and localization results of the target carrier are determined using a semantic VSLAM-assisted matching algorithm.

7. The passive fusion positioning method for indoor environments according to claim 1, characterized in that, Based on the magnetic field matching positioning results, the PDR positioning results, and the VSLAM matching positioning results, the constrained target carrier's fusion positioning results include: Based on the magnetic field matching positioning results and the VSLAM matching positioning results, a measurement posterior factor is constructed. Using the Kalman filter algorithm, with the PDR positioning result as the state information and the magnetic field matching positioning result and the VSLAM matching positioning result as the measurement information, the step size information and heading information in the PDR positioning result are added to the state equation, and the step size measurement value and heading vector measurement value of the measurement posterior factor are added to the measurement equation as measurement information to construct the second Kalman filter system model. Solve the second Kalman filter system model to obtain the fusion localization result of the target carrier.

8. A passive fusion positioning system for indoor environments, characterized in that, The system applies the passive fusion positioning method for indoor environments as described in any one of claims 1-7, and the system includes: The data acquisition module is used to acquire real-time motion data, fingerprint database construction data, and long-distance step length model training data of the target carrier in an indoor environment. The real-time motion data, fingerprint database construction data, and long-distance step length model training data are all acquired by various sensors mounted on the target carrier. The real-time motion data includes real-time motion magnetic field data, real-time motion inertial sensor data, and real-time motion two-dimensional visual data. The fingerprint database construction data includes magnetic field fingerprint database construction data and two-dimensional visual fingerprint database construction data. The magnetic field matching and positioning result determination module is used to construct fusion weights based on the real-time motion magnetic field data and the magnetic field fingerprint database, and to determine the magnetic field matching and positioning result of the target carrier using the WKNN algorithm based on the fusion weights. The PDR heading information determination module is used to determine the PDR heading information of the target carrier based on the real-time motion magnetic field data and the real-time motion inertial sensor data, using an adaptive window evaluation mean filtering algorithm. The long-distance step size model training module is used to train the basic deep learning model, the fully connected neural network model, using long-distance step size model training data to obtain the long-distance step size model. The PDR positioning result determination module is used to determine the PDR positioning result of the target vehicle based on the real-time motion inertial sensor data, the PDR heading information and the long-distance step model. The VSLAM matching and localization result determination module is used to determine the VSLAM matching and localization result of the target carrier based on the real-time motion two-dimensional visual data and the two-dimensional visual fingerprint database, using the semantic VSLAM assisted matching algorithm. The fusion positioning result determination module is used to constrain the fusion positioning result of the target carrier based on the magnetic field matching positioning result, the PDR positioning result, and the VSLAM matching positioning result.

9. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program and the processor runs the computer program to enable the electronic device to perform a passive fusion positioning method for an indoor environment as described in any one of claims 1 to 7.

Images