Intelligent terminal indoor multi-scene positioning method, device, equipment and medium

Abstract

The invention provides an intelligent terminal indoor multi-scene positioning method, device and equipment and a medium, and the method comprises the steps: when a target element exists in a current indoor scene where an intelligent terminal (a smartphone, a tablet computer and the like) is located, according to the visual positioning information of the intelligent terminal and the Bluetooth positioning information of the intelligent terminal, the target element is located in the current indoor scene; obtaining fusion positioning information according to the PDR positioning information of the intelligent terminal; and finally determining real-time position information of the intelligent terminal according to the fused positioning information and the building map positioning anchor point data set of the constraint area. According to the invention, the real-time positioning of the indoor multi-scene intelligent terminal can be realized, the stability of long-time positioning during the continuous change of the scene is improved, the interference of indoor environment factors on wireless signals is reduced, and the purpose of low-cost and stable universal indoor positioning is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of indoor positioning, and particularly relates to a method, device, equipment and medium for indoor multi-scenario positioning of intelligent terminals. Background Art

[0002] Location information has greatly improved the convenience of people's lives and the efficiency of social operation. With the development of smartphones, Internet of Things technology and smart cities, the indoor location positioning service industry based on intelligent terminals (such as mobile phones, tablets) has developed rapidly, and people's demand for Location Based Services (LBS) is becoming increasingly strong.

[0003] Nowadays, intelligent terminals have become indispensable devices in people's lives. They are integrated with rich sensors such as accelerometers, magnetometers, gyroscopes, Bluetooth, Wi-Fi, cameras, etc., and can be used as positioning sources for pedestrians to perform indoor positioning. Currently, many indoor positioning technologies have been explored based on intelligent terminals, mainly including Bluetooth Low Energy (BLE), Wi-Fi, magnetic field, acoustics, Ultra Wide Band (UWB), visible light, vision, inertial navigation, etc. The Global Navigation Satellite System (GNSS) is widely used for outdoor positioning, which can provide relatively long-term and stable meter-level location services. However, affected by the complex indoor environment, GNSS signals will have multi-path effects, occlusion and attenuation, making it difficult to provide reliable positioning services indoors, so that pedestrians are difficult to accurately obtain their current positions in buildings. In addition, general technologies like GNSS are lacking in indoor environments. Each indoor positioning method based on intelligent terminals has its own advantages and disadvantages, and no single technology can dominate in all actual scenarios in terms of accuracy, power consumption and portability.

[0004] With the in-depth research and development of multi-source fusion positioning solutions, a variety of integrated indoor positioning technologies have emerged. Through multi-source fusion technology, the advantages of multiple positioning sources can be integrated to achieve the complementary advantages of different technologies. However, the actual indoor environment is complex and diverse, resulting in poor stability of indoor positioning algorithms during long-term positioning in continuously changing scenarios, and the radio signals emitted by various electronic devices deployed indoors will also interfere with the wireless signals of the indoor positioning system, affecting the overall positioning accuracy. In addition, how to control the system deployment cost while ensuring sufficient positioning accuracy is a practical problem that a highly available indoor positioning system needs to carefully consider. Therefore, a low-cost, highly stable and highly accurate indoor positioning method based on intelligent terminals that can perform long-term positioning in multiple indoor environments is a practical problem that urgently needs to be solved in the field of indoor positioning. Summary of the Invention

[0005] In view of the problems existing in the prior art, the present invention provides an indoor multi-scenario positioning method, device, equipment and medium for intelligent terminals.

[0006] The present invention provides an indoor multi-scenario positioning method for intelligent terminals, including: When there are target elements in the current scenario where the intelligent terminal is located, determine the relative distance between the target elements and the intelligent terminal, the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal; According to the relative distance between the target elements and the intelligent terminal, determine a constraint area in the current scenario, and screen out the building map positioning anchor point dataset corresponding to the constraint area from the building map positioning anchor point dataset corresponding to the current scenario; Obtain fusion positioning information according to the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal; Determine the real-time position information of the intelligent terminal according to the fusion positioning information and the building map positioning anchor point dataset of the constraint area.

[0007] According to the indoor multi-scenario positioning method for intelligent terminals provided by the present invention, the method further includes: when there are no target elements in the current scenario where the intelligent terminal is located, determine the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal; Obtain fusion positioning information according to the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal; Determine the real-time position information of the intelligent terminal according to the fusion positioning information and the building map positioning anchor point dataset of the current scenario.

[0008] An indoor multi-scenario positioning method for an intelligent terminal provided by the present invention. The building map positioning anchor point dataset contains anchor points corresponding to different sensor signal change characteristics. Accordingly, the method further includes: while determining PDR positioning information based on the sensor signals collected by the intelligent terminal, extracting the current signal change characteristics according to the sensor signals; the current signal change characteristics represent the characteristic information of the behavior change type of the user carrying the intelligent terminal. Match the signal change characteristics of the built-in sensors of the current intelligent terminal with the anchor points in the building map positioning anchor point dataset of the current scene. After successfully matching the anchor points, use the coordinates of the matched anchor points as the real-time position information of the intelligent terminal.

[0009] An indoor multi-scenario positioning method for an intelligent terminal provided by the present invention. The building map positioning anchor point dataset contains anchor points corresponding to elements in different scenarios. Accordingly, when there are target elements in the current scene where the intelligent terminal is located, determining the visual positioning information of the intelligent terminal includes: Match the target elements existing in the current scene with the anchor points in the building map positioning anchor point dataset of the current scene. After successfully matching the anchor points, perform intelligent terminal spatial position calculation based on the coordinates of the matched anchor points and the relative distance between the target elements and the intelligent terminal to determine the visual positioning information of the intelligent terminal.

[0010] An indoor multi-scenario positioning method for an intelligent terminal provided by the present invention. Determining the Bluetooth positioning information of the intelligent terminal includes: Select a preset number of Bluetooth beacons according to the signal strength of the Bluetooth beacons around the intelligent terminal. Obtain the coordinate information of the selected Bluetooth beacons in the building map positioning anchor point dataset of the current scene. Determine the Euclidean distance between the Bluetooth beacon and the intelligent terminal according to the signal strength of the selected Bluetooth beacon. Determine the weight of the selected Bluetooth beacon according to the Euclidean distance. Determine the Bluetooth positioning information of the intelligent terminal according to the coordinate information and the weight of the selected beacons.

[0011] An indoor multi-scenario positioning method for an intelligent terminal provided by the present invention. The determining the Euclidean distance between the Bluetooth beacon and the intelligent terminal according to the signal strength of the selected Bluetooth beacon includes: Match the corresponding Euclidean distance calculation strategy according to the signal strength of the Bluetooth beacon. Determine the Euclidean distance between the Bluetooth beacon and the intelligent terminal according to the signal strength of the Bluetooth beacon and the Euclidean distance calculation strategy.

[0012] An indoor multi-scenario positioning method for an intelligent terminal provided by the present invention to determine the relative distance between the target element and the intelligent terminal includes: Obtain the attribute information of the target element in the current scene in the building map positioning anchor point dataset of the current scene, and determine the actual height of the target element according to the attribute information; Obtain the current video frame captured by the intelligent terminal, and determine the height of the detection frame of the target element according to the current video frame; Obtain the device parameters when the intelligent terminal takes a picture; Determine the relative distance between the target element and the intelligent terminal according to the actual height of the target element, the height of the detection frame, and the device parameters.

[0013] The present invention also provides an indoor multi-scenario positioning device for an intelligent terminal, including: A determination module, configured to determine the relative distance between the target element and the intelligent terminal, the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal when there is a target element in the current scene where the intelligent terminal is located; A selection module, configured to determine a constraint area in the current scene according to the relative distance between the target element and the intelligent terminal, and screen out the building map positioning anchor point dataset corresponding to the constraint area in the building map positioning anchor point dataset corresponding to the current scene; A fusion module, configured to obtain fusion positioning information according to the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal; A positioning module, configured to determine the real-time position information of the intelligent terminal according to the fusion positioning information and the building map positioning anchor point dataset of the constraint area.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements any of the above indoor multi-scenario positioning methods for an intelligent terminal when executing the program.

[0015] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and the computer program implements any of the above indoor multi-scenario positioning methods for an intelligent terminal when executed by a processor.

[0016] The present invention also provides a computer program product, including a computer program, and the computer program implements any of the above indoor multi-scenario positioning methods for an intelligent terminal when executed by a processor.

[0017] An intelligent terminal indoor multi-scenario positioning method, device, equipment and medium provided by the present invention obtain fused positioning information by using the visual positioning information, Bluetooth positioning information and PDR positioning information of the intelligent terminal, and determine the real-time position information of the intelligent terminal according to the fused positioning information and the building map positioning anchor point data set of the constraint area, which can realize the real-time positioning of indoor multi-scenarios, improve the stability of long-term positioning during continuous scene changes, reduce the interference of indoor environmental factors on wireless signals, and achieve a low-cost, stable and available universal indoor positioning effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 is a schematic flowchart of the intelligent terminal indoor multi-scenario positioning method provided by the present invention.

[0020] Figure 2 is a schematic structural diagram of the intelligent terminal indoor multi-scenario positioning device provided by the present invention.

[0021] Figure 3 is a schematic structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] To make the objectives, technical solutions and advantages of the present invention clearer, the following clearly and completely describes the technical solutions in the present invention with reference to the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in the present invention belong to the scope of protection of the present invention.

[0023] The following combines Figures 1 - 3 to describe the intelligent terminal indoor multi-scenario positioning method, device, equipment and medium of the present invention.

[0024] Figure 1 shows a schematic flowchart of an intelligent terminal indoor multi-scenario positioning method provided by the present invention. Refer to Figure 1 , the method includes the following steps: Step 11. When there is a target element in the current scene where the intelligent terminal is located, determine the relative distance between the target element and the intelligent terminal, the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal.

[0025] Step 12. According to the relative distance between the target element and the intelligent terminal, determine a constraint area in the current scene, and screen out the building map positioning anchor point dataset corresponding to the constraint area from the building map positioning anchor point dataset corresponding to the current scene.

[0026] Step 13. Obtain fused positioning information based on the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal.

[0027] Step 14. Determine the real-time position information of the intelligent terminal according to the fused positioning information and the building map positioning anchor point dataset of the constraint area.

[0028] Regarding Steps 11 to 14, it should be noted that location information has greatly improved the convenience of people's lives and the efficiency of social operations. With the development of smartphones, Internet of Things technology, and smart cities, the indoor location positioning service industry based on intelligent terminals (such as mobile phones and tablets) has developed rapidly, and people's demand for location-based services (LBS) has become increasingly strong.

[0029] Today, intelligent terminals have become an indispensable device in people's lives. They are internally integrated with rich sensors such as accelerometers, magnetometers, gyroscopes, Bluetooth, Wi-Fi, cameras, etc., and can be used as positioning sources for pedestrians to locate indoors.

[0030] Based on the positioning correction and matching of the intelligent terminal camera, built-in inertial sensors (accelerometer, gyroscope, magnetometer), Bluetooth sensors, and combined with the constructed building map positioning anchor points (MLA), the present invention proposes an indoor positioning method for intelligent terminals, aiming to improve the comprehensive positioning performance of BLE, PDR, vision, and building map information.

[0031] For this reason, in the present invention, when a user carrying an intelligent terminal in a building (such as a shopping mall or an office building) needs indoor positioning, the camera is turned on to capture the video image in front of the camera. The video image will display the images of various indoor elements in the current scene. Most of these indoor elements are universal elements, including doorplates, doors, fire alarms, emergency exits, etc. Therefore, by performing element recognition on the video image, it is possible to identify whether there are elements or no elements in the current image.

[0032] In the present invention, a dataset of building map location anchor points is constructed. The MapLocation Anchor (MLA) is a way to represent feature points in an indoor space. By extracting key location points with geometric or semantic features in the building environment, a dataset of spatial location anchor points with multi-dimensional attribute information is constructed. Inside a building, considering the basic structure of the building and the generally recognizable element features, constructing an MLA system based on the building entity-network-voxel hybrid map model (ENV) means that even if the indoor environment changes, these anchor points do not need to be re-maintained or adjusted, and the entire positioning process can maintain its accuracy and reliability.

[0033] The NEV model discretizes the indoor space into the smallest spatial units (voxels) through the newly added voxelization technology, achieving three-dimensional improvements: First, the voxelization process breaks through the linear limitation of the network model on the spatial topological structure. Non-network associated feature points such as wall corners and equipment aggregation areas can be extracted through voxel clustering analysis, significantly expanding the spatial coverage density of the anchor points. Second, the three-dimensional spatial coordinates (x, y, z) carried by the voxel unit can accurately represent the spatial distribution in the vertical direction, improving the three-dimensional position matching accuracy of the MLA. Finally, since each voxel unit can integrate multi-modal data such as material attributes, electromagnetic fingerprints, and visual features, the MLA can not only be used for position matching in indoor location services but also has environmental perception capabilities.

[0034] In the present invention, the MLA includes anchor points corresponding to the sensor signal change characteristics for multi-source sensor fusion indoor positioning for intelligent terminals, simply referred to as MLA(S), and anchor points corresponding to recognizable target elements for indoor scene visual positioning for intelligent terminals, simply referred to as MLA(C). MLA(S) is a dataset composed of spatial feature points with different sensor change characteristics. These feature points include walking nodes where pedestrians walk (such as walking nodes in corridors), turning nodes when pedestrians turn (such as corridor corners, nodes where the corridor turns into a room, etc.), and connection nodes where pedestrians switch indoor scenes (such as stairwells, elevator entrances, etc.). By matching the corresponding anchor points according to the sensor signal changes during walking, real-time position constraints and cumulative error correction can be performed during the indoor positioning and navigation process. MLA(C) is a dataset composed of the central coordinates of general and recognizable elements in different indoor scenes. These indoor elements include doorplates, doors, fire alarms, emergency exits, etc. By identifying these elements with an intelligent terminal and matching the position information of the MLA(C), position coordinates can be provided in visual positioning.

[0035] The model definition of the MLA is as follows: It is an anchor point subset corresponding to the change characteristics of the sensor signal, storing the coordinate information of the anchor points that match the change characteristics of the sensor signal inside the intelligent terminal. and represent the position of the anchor point on the horizontal plane. represents the height information of the anchor point. represents the set of real numbers. It is an anchor point subset for the elements in the visually recognizable scene, storing the coordinate information and attribute information of the anchor points corresponding to the elements in the indoor scene image captured by the intelligent terminal in real time.

[0036] ; ; represents the th type of sensor of the anchor point, represents the sensor signal characteristics associated with the anchor point, , , and respectively represent the signal change characteristics of the accelerometer, gyroscope, magnetometer, and barometer sensors. represents the th type of element of the recognizable element anchor point (such as doorplate, doorplate, electrical box, etc.), represents the th attribute information of the recognizable element anchor point (such as doorplate number, device number, etc.). The MLA dataset will be used for position matching and position constraint in the positioning algorithm based on multi-source sensor fusion and the visual positioning algorithm for element recognition in the indoor scene.

[0037] Due to the complex and diverse scenarios of indoor positioning and navigation (rooms, corridors, stairs, halls, etc.), different scenario switches will affect the accuracy and stability of indoor positioning. Therefore, the present invention constructs the MLA dataset based on the basic structure of the building and the generally recognizable features, collects the signal changes of the intelligent terminal sensors of pedestrians in different motion states (such as walking, turning, going up and down stairs, taking the elevator, etc.), analyzes the sensor signal change characteristics when the motion state of the pedestrian changes, and binds them with the MLA S for binding. Collect the position coordinate information and semantic information of the recognizable elements in the building map scene and bind them with the MLA C for position matching during visual positioning.

[0038] In the present invention, when identifying elements in the video image captured by the smart terminal, if it is determined that there is one or more elements in the current scene where the smart terminal is located, the relative distance between the element and the smart terminal can be determined at this time (since it is the image captured by the smart terminal camera, in fact, the depth of the element on the image is obtained). This relative distance can delimit a pedestrian position range buffer centered on the position coordinates of the visual element to constrain the position of the pedestrian, facilitating the reduction of data volume when locating and matching the anchor point, avoiding redundant calculations, and at the same time controlling the error of position estimation within an acceptable range. Therefore, according to the relative distance between the element and the smart terminal, a constraint area is determined in the current scene, and the building map location anchor point data set corresponding to the constraint area is screened out from the building map location anchor point data set corresponding to the current scene.

[0039] In a further invention, obtaining the relative distance between the target element and the smart terminal specifically includes: obtaining the coordinate information and attribute information of the target element in the current scene in the building map location anchor point data set of the current scene, determining the actual position coordinates of the target element according to the coordinate information, and determining the actual height of the target element according to the attribute information.

[0040] Obtain the current video image captured by the smart terminal, and determine the detection frame height of the target element according to the current video image.

[0041] Obtain the device parameters when the smart terminal is shooting, such as including the vertical pitch angle and the focal length of the camera.

[0042] According to the actual height, the detection frame height, and the device parameters, determine the relative distance between the target element and the smart terminal.

[0043] More specifically, in the present invention, the following calculation formula can be used to obtain the relative distance between the target element and the smart terminal.

[0044] Among them, represents the actual height of the i th element (obtained from the MLA information), represents the detection frame height of the i th element, is the vertical pitch angle, f represents the focal length of the rear camera. The pitch angle is used to reduce the error generated when the camera shoots at an inclined angle. With the relative distance as the radius and the position coordinates of the target element as the center, a pedestrian position buffer, that is, a constraint area, is set. This constraint area is a spatial area. If multiple target elements are identified, the intersection area of multiple areas can be optimized and extended to obtain one area.

[0045] In the present invention, when positioning the smart terminal (i.e., indoor pedestrian) by combining the building map positioning anchor point dataset, the visual positioning information of the smart terminal, the Bluetooth positioning information of the smart terminal, and the PDR positioning information of the smart terminal are also required. The visual positioning information is the position where the pedestrian is located determined by identifying the universal elements in the video frame. The Bluetooth positioning information is the positional relationship between the Bluetooth sensor and the Bluetooth beacons in the building. PDR is a relative positioning technology that uses the measurement values of the IMU sensor built in the smart terminal to estimate the current position based on the previous position by calculating the walking distance and the heading angle.

[0046] In the present invention, the visual positioning information of the smart terminal, the Bluetooth positioning information of the smart terminal, and the PDR positioning information of the smart terminal are fused to obtain the fused positioning information. This fused positioning information avoids the reduction in accuracy caused by the interference of single positioning information, and can make multiple positioning information mutually constrained and corrected to ensure high-precision positioning information.

[0047] Finally, based on the fused positioning information and the building map positioning anchor point dataset in the constraint area, the real-time position information of the smart terminal is determined. During the real-time positioning of the pedestrian, the fused position coordinates are used to match the anchor point positions in the MLA database, and after the geometric constraint and position correction of the MLA, the anchor point positions are obtained and output as the final positioning result, realizing the real-time positioning in multiple indoor scenarios.

[0048] The smart terminal indoor positioning method provided by the present invention can realize the real-time positioning in multiple indoor scenarios, improve the stability of long-term positioning during continuous scene changes, reduce the interference of indoor environmental factors on wireless signals, and achieve the purpose of low-cost and stable universal indoor positioning by obtaining the fused positioning information based on the visual positioning information of the smart terminal, the Bluetooth positioning information of the smart terminal, and the PDR positioning information of the smart terminal, and determining the real-time position information of the smart terminal based on the fused positioning information and the building map positioning anchor point dataset in the constraint area.

[0049] In a further implementation of the above method, when identifying the elements in the video frame, it can be recognized whether there are target elements or not in the current frame. Therefore, when there are no target elements in the current scene where the smart terminal is located, at this time, it is not necessary to determine the visual positioning information of the smart terminal, and only the Bluetooth positioning information of the smart terminal and the PDR positioning information of the smart terminal need to be determined.

[0050] Next, based on the Bluetooth positioning information of the intelligent terminal and the PDR positioning information of the intelligent terminal, the fused positioning information is obtained. Finally, based on the fused positioning information and the building map positioning anchor point dataset of the current scene, the real-time position information of the intelligent terminal is determined. The fusion of positioning information and the matching of the fused positioning are described above and will not be elaborated here.

[0051] In a further method of the above method, the building map positioning anchor point dataset contains anchor points corresponding to different signal change characteristics. After detecting the sensor signal change characteristics during walking and matching the corresponding anchor points, real-time positioning can be directly performed during indoor positioning and navigation, and the position matched according to the fused positioning information can be constrained and error-corrected. That is to say, in this special case, the position of the anchor point directly identified and bound to the sensor signal change has a higher priority as the output of the real-time position.

[0052] Therefore, in the present invention, while determining the PDR positioning information based on the sensor signals collected by the intelligent terminal, the current signal change characteristics are extracted from the sensor signals. The collected sensor signals are the signals of the accelerometer, gyroscope, magnetometer, and barometer sensors. By performing signal analysis on the sensor signals at different times before and after, the signal change characteristics can be obtained. The current signal change characteristics represent the characteristic information of the behavior change type of the user carrying the intelligent terminal. The signal changes of the intelligent terminal sensors of pedestrians in different motion states (such as walking, turning, going up and down stairs, taking the elevator, etc.) are collected, and the signal change characteristics when the pedestrian's motion state changes are analyzed.

[0053] The current signal change characteristics are matched with the anchor points in the building map positioning anchor point dataset of the current scene. When the anchor point matching is successful, the coordinates of the matched anchor point are used as the real-time position information of the intelligent terminal.

[0054] In a further method of the above method, the building map positioning anchor point dataset contains anchor points corresponding to different target elements. Accordingly, when there are target elements in the current scene where the intelligent terminal is located, the visual positioning information of the intelligent terminal is determined, including: The target elements existing in the current scene are matched with the anchor points in the building map positioning anchor point dataset of the current scene. When the anchor point matching is successful, based on the coordinates of the matched anchor point and the relative distance between the target element and the intelligent terminal, the spatial position of the intelligent terminal is calculated to determine the visual positioning information of the intelligent terminal.

[0055] Specifically: According to the C anchor point coordinates corresponding to each target element stored in the MLA, the relative distance between the target element and the intelligent terminal, and the current state information of the intelligent terminal, the visual positioning information of the intelligent terminal is calculated through the spherical coordinate system conversion algorithm.

[0056] It should be noted that when the number of target elements identified is greater than 1, the weighted average value is calculated and output as the visual positioning information. In the subsequent positioning process, the visual positioning information is fused with the Bluetooth positioning information and the PDR positioning information, and is constrained by the pedestrian position buffer to match the position in the dataset.

[0057] In a further method of the above method, the process of determining the Bluetooth positioning information of the smart terminal is mainly explained as follows: Select a preset number of Bluetooth beacons according to the signal strength of the Bluetooth beacons around the smart terminal; Obtain the coordinate information of the selected Bluetooth beacons in the building map positioning anchor point dataset of the current scene; Determine the Euclidean distance between the Bluetooth beacon and the smart terminal according to the signal strength of the selected Bluetooth beacon; Determine the weight of the selected Bluetooth beacon according to the Euclidean distance; Determine the Bluetooth positioning information of the smart terminal according to the coordinate information of the selected beacon and the weight.

[0058] In this regard, it should be noted that in the present invention, the Bluetooth sensor of the smart terminal is called to obtain the signal strength value of the surrounding Bluetooth beacons, and according to the signal strength of the Bluetooth beacons, the largest several beacons are selected, and the coordinate data of these beacons are obtained through the MLA database .

[0059] In signal propagation, the signal strength decays along with the distance from the signal transmitter. The attenuation of the signal strength usually has a certain mathematical relationship with the propagation distance, which can be represented by a path loss model. The accuracy, efficiency and positioning accuracy of BLE RSS ranging largely depend on the measured RSSI value and the stability of the path loss model. The research on the path loss model shows that when the geometric distance between the transmitter and the receiver exceeds a certain range, the accuracy of RSSI will be difficult to reflect the change of distance. Therefore, in one embodiment, 10m is selected as the maximum distance for Bluetooth ranging according to the Bluetooth beacon deployment density. In order to reduce the influence of the indoor environment on the BLE ranging accuracy, the present invention proposes an improved piecewise linear path loss model based on MLA. On the basis of the piecewise linear path loss model, the BLE RSS signal data is processed in real time through the EKF filtering algorithm to reduce signal fluctuations and increase the position constraint of MLA, so as to improve the stability and accuracy of RSS ranging. In order to enable the path loss model to cover the rapid change area (1 - 3m) and the gentle attenuation area (5 - 10m) of signal attenuation and ensure the adaptability of the model to different distances.

[0060] Divide the overall distance into multiple intervals (such as 1m, 2m, 3m, 5m, 10m), and use independent linear functions to calibrate parameters for each interval, which is more in line with the actual attenuation curve and reduces systematic deviation. In addition, the signal fluctuation is relatively stable within a short interval, and segmented calibration can reduce the impact of abnormal single RSSI value on the overall ranging. As shown in the following formula: Among them, is the Euclidean distance, and rssi represents the BLE rssi after preprocessing by the filtering algorithm. Rssi1, Rssi2, Rssi3, Rssi5, and Rssi10 are calibration parameters, which are the actually measured RSSI values at distances of 1 m, 2 m, 3 m, 5 m, and 10 m respectively. The RSSI thresholds for each distance interval can be adjusted according to the actual measurement data to make the estimation result closer to the true value. By setting different thresholds, it can be optimized for specific scenarios, and to a certain extent, reduce the impact of the fluctuation of a single RSSI value on the Euclidean distance estimation.

[0061] In order to reduce the influence of the indoor environment, the present invention uses a weighted spatial quadrilateral positioning algorithm to calculate the position coordinates of pedestrians, evaluates and optimizes the geometric layout through spatial dispersion, assigns different weights to beacons at different distances, reduces the sensitivity of the positioning result to the beacon layout, can effectively reduce the influence of noise and interference, and improves the stability and accuracy of positioning. As shown in the following formula: ; Among them is the weight of the th beacon, is the spatial coordinate of the th beacon, is the Euclidean distance between the pedestrian and the th beacon calculated by the piecewise linear loss path algorithm based on MLA. Then, the least squares method is used to calculate the optimal spatial coordinates by minimizing the sum of squared residuals. Finally, the Bluetooth positioning coordinates of the pedestrian are obtained. As shown in the following formula: .

[0062] In the present invention, the overall process of indoor positioning of intelligent terminals is described as follows: (1) Initial positioning. First, identify the current scene category, dynamically load MLA data and limit the range of the pedestrian's feasible area in this scene. Synchronously search for nearby deployed Bluetooth beacons and obtain the coordinate data of these beacons through the MLA database . Calculate the Bluetooth positioning result using the above formula for Bluetooth positioning coordinates , and determine the floor where the pedestrian is located, and further restrict the feasible area of the pedestrian to the corresponding scene on a certain floor. Secondly, according to the element recognition model of the indoor scene, the recognizable elements in the video frame captured by the rear camera of the smart terminal are recognized in real time. According to and the attitude of the smart terminal, map matching is performed with the corresponding records in the MLA database to obtain the position coordinates and attribute information of each element. The visual positioning result is calculated using the above visual positioning algorithm . Then, use EKF fusion and . After that, a constraint judgment is performed with the pedestrian position buffer, and the initial positioning result is projected onto the buffer boundary. Finally, the positioning result is matched to the MLA walking node with the closest distance, and the position coordinates of this node are used as the final initial positioning result , as the starting point of the indoor positioning process.

[0063] (2) Pedestrian position update. During the movement of the pedestrian, the PDR algorithm is used to calculate the position update data of the pedestrian, and the position coordinates at the current moment are calculated based on the position coordinates at the previous moment (initially ). If recognizable elements are recognized by the visual positioning module during the movement, then use EKF fusion , and . After that, the position buffer judgment is also performed. After position matching with the MLA, the fused positioning result is matched to the walking node with the closest distance to reduce the cumulative error of the PDR, and the position coordinates of this node are used as the final real-time positioning result , and is used as the input for the next PDR position update.

[0064] (3) MLA position matching. During the movement of the pedestrian, when the sensors built into the smart terminal detect specific signal features, the MLA(S) record of the sensor with the closest distance in the MLA database will be searched through the current position, and the position coordinates of this MLA(S) will be directly assigned to the pedestrian position coordinates at the current moment , further reducing the cumulative error of the PDR. For example, the gyroscope detects the signal feature of turning, the accelerometer detects the signal feature of opening or closing the door, and the barometer detects the air pressure change feature when walking up or down the stairs or taking the elevator. ​(4) Scene switching. When the position matching of the triggered MLA(S) occurs or the pedestrian position matches the walking node of other scenes, the environmental perception model will be called within a certain time to identify whether the scene category where the pedestrian is located has changed. After the scene changes, the MLA data of this scene will be dynamically loaded and the positioning algorithm of this scene will be called. For example, the adaptive step model parameters of PDR will be adjusted according to the scene. In the staircase scene, an additional height correction will be added to the coordinate calculation of each step. If the change in the barometer measurement value indicates going upstairs, the additional height will be increased; otherwise, it will be decreased. If it is detected that the change in the barometer exceeds the threshold, the Bluetooth positioning module and the environmental perception module will be called to detect the floor change.

[0065] (5) Error recognition and outlier handling. During the positioning process, if an incorrect feature category is recognized, resulting in a large difference between the positioning result and the position at the previous moment or the positioning is outside the feasible area of the pedestrian, the abnormal positioning result will be removed, and only the PDR positioning result will be used to match the nearest MLA walking node to reduce the interference of error recognition and outliers on the positioning result. In addition, when the holding angle of the smart terminal is close to 90 degrees, it will affect the calculation of the PDR heading. Therefore, if it is detected that the tilt angle of the smart terminal exceeds 80 degrees, the pedestrian will be reminded that a too large tilt angle will increase the error of heading calculation, and the pedestrian is requested to reduce the tilt angle of the smart terminal to about 60 degrees (the angle between the back of the mobile phone and the ground).

[0066] The intelligent terminal indoor multi-scene positioning device provided by the present invention will be described below. The intelligent terminal indoor multi-scene positioning device described below can be correspondingly referred to the intelligent terminal indoor multi-scene positioning method described above.

[0067] Figure 2 The structural schematic diagram of an intelligent terminal indoor multi-scene positioning device provided by the present invention is shown. Refer to Figure 2 , the device includes a determination module 21, a selection module 22, a fusion module 23, and a positioning module 24, where: The determination module is used to determine the relative distance between the target element and the intelligent terminal, the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal when there is a target element in the current scene where the intelligent terminal is located; The selection module is used to determine a constraint area in the current scene according to the relative distance between the target element and the intelligent terminal, and screen out the building map positioning anchor point dataset corresponding to the constraint area from the building map positioning anchor point dataset corresponding to the current scene; The fusion module is used to obtain fusion positioning information according to the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal; A positioning module, configured to locate an anchor point dataset according to the fused positioning information and the building map of the constraint area, and determine the real-time position information of the intelligent terminal.

[0068] Since the device in the embodiment of the present invention has the same principle as the method in the above embodiment, the more detailed explanation content will not be repeated here.

[0069] It should be noted that in the embodiment of the present invention, the relevant functional modules can be implemented by a hardware processor.

[0070] The intelligent terminal indoor multi-scenario positioning device provided by the present invention obtains the fused positioning information according to the visual positioning information, Bluetooth positioning information, and PDR positioning information of the intelligent terminal, and locates the anchor point dataset according to the fused positioning information and the building map of the constraint area, and determines the real-time position information of the intelligent terminal, which can realize the real-time positioning of indoor multi-scenarios, improve the stability of long-time positioning during continuous scene changes, reduce the interference of indoor environmental factors on wireless signals, and achieve a low-cost and stable and usable universal indoor positioning effect.

[0071] Figure 3 An example of a schematic physical structure diagram of an electronic device is shown as Figure 3 As shown, the electronic device may include: a processor 31 (processor), a communication interface 32 (Communications Interface), a memory 33 (memory), and a communication bus 34. Among them, the processor 31, the communication interface 32, and the memory 33 communicate with each other through the communication bus 34. The processor 31 can call the logical instructions in the memory 33 to execute the intelligent terminal indoor multi-scenario positioning method, and the method includes: when there is a target element in the current scene where the intelligent terminal is located, determining the relative distance between the target element and the intelligent terminal, the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal, determining a constraint area in the current scene according to the relative distance between the target element and the intelligent terminal, and screening out the building map anchor point dataset corresponding to the constraint area from the building map anchor point dataset corresponding to the current scene, obtaining the fused positioning information according to the visual positioning information, Bluetooth positioning information, and PDR positioning information of the intelligent terminal, and determining the real-time position information of the intelligent terminal according to the fused positioning information and the building map anchor point dataset of the constraint area.

[0072] In addition, when the logical instructions in the above-mentioned memory 33 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0073] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the intelligent terminal indoor multi-scenario positioning method provided by the above-mentioned various methods. The method includes: when there is a target element in the current scene where the intelligent terminal is located, determining the relative distance between the target element and the intelligent terminal, the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal. According to the relative distance between the target element and the intelligent terminal, determining a constraint area in the current scene, and screening out the building map positioning anchor point data set corresponding to the constraint area from the building map positioning anchor point data set corresponding to the current scene. Obtaining fused positioning information according to the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal. Determining the real-time position information of the intelligent terminal according to the fused positioning information and the building map positioning anchor point data set of the constraint area.

[0074] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements an intelligent terminal indoor multi-scenario positioning method provided by the above-mentioned various methods. The method includes: when there is a target element in the current scenario where the intelligent terminal is located, determining the relative distance between the target element and the intelligent terminal, the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal. According to the relative distance between the target element and the intelligent terminal, a constraint area is determined in the current scenario, and a building map positioning anchor point data set corresponding to the constraint area is screened out from the building map positioning anchor point data set corresponding to the current scenario. Fusion positioning information is obtained according to the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal. According to the fusion positioning information and the building map positioning anchor point data set of the constraint area, the real-time position information of the intelligent terminal is determined.

[0075] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0076] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. And these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An indoor multi-scenario positioning method for intelligent terminals, characterized in that Including: When there is a target element in the current scene where the intelligent terminal is located, determining the relative distance between the target element and the intelligent terminal, the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal; According to the relative distance between the target element and the intelligent terminal, determining a constraint area in the current scene, and screening out the building map positioning anchor point dataset corresponding to the constraint area from the building map positioning anchor point dataset corresponding to the current scene; Obtaining fusion positioning information according to the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal; Determining the real-time position information of the intelligent terminal according to the fusion positioning information and the building map positioning anchor point dataset of the constraint area.

2. The indoor multi-scenario positioning method for an intelligent terminal according to claim 1, wherein The method further includes: when there is no target element in the current scene where the intelligent terminal is located, determining the Bluetooth positioning information of the intelligent terminal and the PDR positioning information of the intelligent terminal; Obtaining fusion positioning information according to the Bluetooth positioning information of the intelligent terminal and the PDR positioning information of the intelligent terminal; Determining the real-time position information of the intelligent terminal according to the fusion positioning information and the building map positioning anchor point dataset of the current scene.

3. The indoor multi-scenario positioning method for an intelligent terminal according to claim 1, characterized in that, The building map positioning anchor point dataset contains anchors corresponding to different sensor signal change characteristics. Correspondingly, the method further includes: while determining the PDR positioning information according to the sensor signals collected by the intelligent terminal, extracting the current signal change characteristics according to the sensor signals; the current signal change characteristics carry the characteristic information of the behavior change type of the user of the intelligent terminal; Performing anchor point matching on the signal change characteristics of the sensors built in the current intelligent terminal in the building map positioning anchor point dataset of the current scene. When the anchor point matching is successful, using the coordinates of the matched anchor point as the real-time position information of the intelligent terminal.

4. The indoor multi-scenario positioning method for an intelligent terminal according to claim 1, wherein, The building map positioning anchor point dataset contains anchors corresponding to elements in different scenes. Correspondingly, when there is a target element in the current scene where the intelligent terminal is located, determining the visual positioning information of the intelligent terminal includes: Performing anchor point coordinate matching on the recognition result of the target element existing in the current scene with the corresponding target in the building map positioning anchor point dataset of the current scene. When the anchor point matching is successful, performing intelligent terminal spatial position calculation according to the coordinates of the matched anchor point and the relative distance between the target element and the intelligent terminal, and determining the visual positioning information of the intelligent terminal.

5. The indoor multi-scenario positioning method for an intelligent terminal according to claim 1 or 2, characterized in that, Determining the Bluetooth positioning information of the intelligent terminal includes: Selecting a preset number of Bluetooth beacons according to the signal strength of the Bluetooth beacons around the intelligent terminal; Obtaining the coordinate information of the selected Bluetooth beacons in the building map positioning anchor point dataset of the current scene; Determining the Euclidean distance between the Bluetooth beacon and the intelligent terminal according to the signal strength of the selected Bluetooth beacon; Determining the weight of the selected Bluetooth beacon according to the Euclidean distance; Determine the Bluetooth positioning information of the intelligent terminal according to the coordinate information of the selected beacon and the said weight.

6. The indoor multi-scenario positioning method for an intelligent terminal according to claim 5, characterized in that The determining the Euclidean distance between the Bluetooth beacon and the intelligent terminal according to the signal strength of the selected Bluetooth beacon includes: Match the corresponding Euclidean distance calculation strategy according to the signal strength of the Bluetooth beacon; Determine the Euclidean distance between the Bluetooth beacon and the intelligent terminal according to the signal strength of the Bluetooth beacon and the Euclidean distance calculation strategy.

7. The indoor multi-scenario positioning method for intelligent terminals according to claim 1, characterized in that Determine the relative distance between the target element and the intelligent terminal, including: Obtain the attribute information of the current target element in the building map positioning anchor point dataset of the current scene, and determine the actual height of the target element according to the attribute information; Obtain the current video frame captured by the intelligent terminal, and determine the height of the detection frame of the target element according to the current video frame; Obtain the device parameters when the intelligent terminal takes a picture; Determine the relative distance between the target element and the intelligent terminal according to the actual height of the target element, the height of the detection frame and the device parameters.

8. An indoor multi-scenario positioning device for an intelligent terminal, characterized in that Include: A determination module, configured to determine the relative distance between the target element and the intelligent terminal, the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal when there is a target element in the current scene where the intelligent terminal is located; A selection module, configured to determine a constraint area in the current scene according to the relative distance between the target element and the intelligent terminal, and filter out the building map positioning anchor point dataset corresponding to the constraint area in the building map positioning anchor point dataset corresponding to the current scene; A fusion module, configured to obtain fusion positioning information according to the visual positioning information of the intelligent terminal, the Bluetooth positioning information of the intelligent terminal, and the PDR positioning information of the intelligent terminal; A positioning module, configured to determine the real-time position information of the intelligent terminal according to the fusion positioning information and the building map positioning anchor point dataset of the constraint area.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the intelligent terminal indoor multi-scene positioning method according to any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the intelligent terminal indoor multi-scene positioning method according to any one of claims 1-7.

Images