Indoor positioning method based on multi-sensor information fusion
Through the method of multi-sensor information fusion, combining image and wireless signal information, the human perception mechanism is simulated, and accurate indoor positioning in dense environments of high-rise buildings is achieved, and the problem of difficulty in taking into account accuracy and universality in the existing technology is solved, and the problem of improvement is improved. Positioning efficiency and accuracy.
Patent Information
- Application Number
- CN202411205657.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing indoor positioning technology cannot take into account both accuracy and universality, and cannot provide accurate and convenient indoor positioning services in a dense urban environment with high-rise buildings.
Using multi-sensor information fusion method, by establishing an image database and a wireless signal fingerprint database, combining computer vision technology and wireless signal information, simulate the perception mechanism of human binocular objects, calculate the position of space object points, and limit the intelligent equipment through wireless signal and direction information Perform precise matching within the trapezoidal range to reduce unnecessary image matching overhead.
It realizes the dependence of labeled position information, saves time and effort, enhances the universality of positioning methods, reduces energy consumption and time consumption, and improves positioning accuracy and coverage.
Smart Images

Figure CN118914972A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of indoor positioning, and in particular to an indoor positioning method based on multi-sensor information fusion. Background Art
[0002] With the rapid development of ubiquitous computing, mobile Internet and the Internet of Things, the importance of location information has become increasingly prominent. More and more applications and services have begun to mine and use location information to provide users with services based on geographic location. Against this backdrop, the positioning problem has received widespread attention. The solution to the outdoor positioning problem is already relatively mature, among which the GPS positioning system is widely used in people's daily lives. Research hotspots are mostly focused on how to locate without GPS or with less use of GPS. However, with the rapid development of society, high-rise buildings are lined up in cities, and people will spend more and more time in their daily lives inside buildings. People are in urgent need of accurate and convenient indoor positioning services to provide basic geographic information services for indoor applications such as smart homes, smart care and smart buildings.
[0003] Due to the importance of indoor positioning, it has received extensive attention from academic and industrial researchers in the past decade. Many user location estimation methods have been proposed and verified, such as sound positioning, RFID positioning, UWB positioning, infrared positioning, pressure sensor positioning, image positioning and WiFi positioning. Sound positioning is simple in principle and easy to deploy, but it is easily interfered by background noise in practice; RFID positioning, UWB positioning, infrared positioning and pressure sensor positioning all require additional hardware deployment overhead, so they lose a certain degree of universality; image positioning requires the use of computer vision technology, which has a large computational overhead and requires a line-of-sight path, but has high accuracy; and WiFi positioning has become the most mainstream indoor positioning technology due to its advantages such as easy deployment, low cost and wide coverage, but WiFi signals are affected by multipath effects and dynamic changes in the environment, and the positioning accuracy can only stay at the meter level, which still cannot meet the requirements of all applications.
[0004] In the process of implementing this application, the inventor found that there are at least the following problems in the prior art: although there are many studies on indoor positioning, there is no universally recognized perfect solution. For an indoor positioning technology, accuracy and universality are undoubtedly the two most important points, and there is no indoor positioning solution that can take both points into account at the same time. Summary of the invention
[0005] The purpose of the present invention is to provide an indoor positioning method and device based on multi-sensor information fusion to solve the problem that the existing indoor positioning technology cannot take into account both accuracy and universality.
[0006] To achieve the above-mentioned purpose, the present invention provides the following technical solution: an indoor positioning method based on multi-sensor information fusion, the positioning method comprising the following steps: S1, establishing an image database and a wireless signal fingerprint database, the image database comprising several pairs of indoor images, the wireless signal fingerprint database comprising the correspondence between spatial position and wireless signal strength; S2, determining the position when a pair of indoor images is acquired based on a pair of indoor images; S3, determining the spatial range of the smart device based on the received wireless signal strength of the smart device and the wireless signal fingerprint database; S4, determining the trapezoidal spatial range of the smart device based on the spatial range and the direction information of the smart device; S5, determining a pair of indoor images matching the current photo based on the trapezoidal range of the smart device, the current photo acquired by the smart device and the image database; S6, determining the position of the smart device based on the pair of indoor images matching the current photo.
[0007] Preferably, the above S2 includes the following steps: S21, determining an object point P that exists simultaneously in a pair of indoor images; S22. Determine the position of the object point P in one of a pair of indoor images The position of the object point P in another indoor image ; S23, determine the position of the object point P in the camera coordinate system The position of the object point P in world coordinates ; S24. According to the perspective principle, the following calculation formula is obtained: Where: represents the focal length, B represents the distance between the origins of a pair of indoor images; S25. Calculation method for determining the position of object point P in world coordinates: Where: represents the rotation matrix, represents the translation matrix, is the scaling factor; S26. Determine the position when acquiring a pair of indoor images according to the position of the object point P in the world coordinate system and its position in the camera coordinate system.
[0008] Preferably, in the above S25 The method of determining is as follows: S251, determining the position of the coordinate origin of one of a pair of indoor images in the image plane and the position of the other coordinate origin in the image plane , and determine the coordinates of the object point P in the image plane ; S252: construct the actual distance and The mapping relationship between them is as follows: Where: The angle information when the smart device takes an indoor image. The angle information when taking another indoor image, , , All are error variables; S253, according to the actual distance and The mapping relationship between them builds the optimization function; S254, Solving by optimizing functions and optimization models .
[0009] Preferably, the above S3 comprises the following steps: S31, determining a mapping relationship between wireless signal strength and actual physical space according to the corresponding relationship between wireless signal strength and spatial position; S32: Determine a spatial range of the smart device based on at least three wireless signals.
[0010] Preferably, the method for confirming the mapping relationship between the wireless signal strength and the actual physical space in the above S31 is: Where: X represents the actual physical space dataset, Y represents the wireless signal fingerprint distribution dataset, i and j represent the index of the dataset, A function that represents the mapping of a real physical space dataset to a common low-dimensional manifold, A function that represents the mapping of a wireless signal fingerprint distribution dataset to a common low-dimensional manifold, Represents the neighbor constraint weight relationship of the actual physical space dataset, Represents the neighbor constraint weight relationship of the wireless signal fingerprint distribution dataset, It represents the mapping relationship between the actual physical space dataset and the wireless signal fingerprint distribution dataset at low latitude. is a weight coefficient used to adjust the objective function.
[0011] Preferably, the above S4 comprises the following steps: S41, obtaining direction information according to the smart device; S42. Determine the trapezoidal range where the smart device is located according to the direction information and the spatial range where the smart device is located, and the spatial range is within the trapezoidal range.
[0012] Based on another object of the present invention, the present invention provides the following technical solution: an indoor positioning device based on multi-sensor information fusion, the positioning device using the above-mentioned positioning method, comprising: Database module, used to establish image database and wireless signal fingerprint database; A position acquisition module, used for determining a position when acquiring a pair of indoor images according to a pair of indoor images; A location estimation module is used to determine the spatial range of the smart device based on the wireless signal strength received by the smart device and the wireless signal fingerprint database; An optimization module, used to determine the trapezoidal space range where the smart device is located according to the space range and the direction information of the smart device; A matching module, used to determine a pair of indoor images matching the current photo according to the trapezoidal range where the smart device is located, the current photo acquired by the smart device and the image database; The positioning module is used to determine the location of the smart device based on a pair of indoor images matched with the current photo.
[0013] Based on another purpose of the present invention, the present invention provides the following technical solution: a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the positioning method as described above when executing the computer program.
[0014] Based on another object of the present invention, the present invention provides the following technical solution: a storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned positioning method when executed by a processor.
[0015] Based on another purpose of the present invention, the present invention provides the following technical solution: a chip, the chip includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the positioning method as described above.
[0016] Compared with the prior art, the beneficial effects of the present invention are: by acquiring a pair of indoor images, simulating the mechanism of human eyes to perceive surrounding information, the position of the spatial object point is deduced according to the coordinate deviation information of the corresponding coordinate point in the image, and then the position of the camera relative to the object point is detected according to the computer vision method, so as to obtain the position when a pair of indoor images are acquired. The above positioning method is free from the dependence on the image database marked with position information, does not require manual survey by professionals, saves time and effort, and greatly enhances the universality of the positioning method. By setting the wireless signal information and the direction information of the smart device, the accurate position of the smart device is limited to a trapezoidal range. Only image-based precise matching is required within this limited trapezoidal range to obtain the current position of the smart device and realize the positioning of the smart device. The above positioning method greatly reduces unnecessary image matching overhead, reduces the energy consumption and time required for positioning, and greatly enhances the universality of the positioning method. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0018] Figure 1 A technical roadmap for an indoor positioning method based on multi-sensor information fusion; Figure 2 It is a schematic diagram of the system structure of the present invention; Figure 3 It is a schematic diagram of the principle of obtaining the position of an object point based on a pair of indoor images; Figure 4 It is a schematic diagram of the three-side positioning model; Figure 5 Schematic diagram of the shooting model before and after optimization.
[0019] In the figure: a positioning device 10 , a database module 110 , a position acquisition module 120 , a position estimation module 130 , an optimization module 140 , a matching module 150 , and a positioning module 160 . DETAILED DESCRIPTION
[0020] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0021] In this article, the terms "upper, lower, inside, outside" and the like are established based on the positional relationships shown in the drawings. The corresponding positional relationships may also change depending on the drawings, and therefore cannot be understood as absolute limitations on the scope of protection; moreover, relational terms such as "first" and "second" and the like are merely used to distinguish one component from another with the same name, and do not necessarily require or imply any actual relationship or order between these components.
[0022] Embodiment 1 like Figure 1 As shown, this embodiment provides an indoor positioning method based on multi-sensor information fusion, and the positioning method includes the following steps: S1. Establish an image database and a wireless signal fingerprint database. The image database includes several pairs of indoor images, and the wireless signal fingerprint database includes the corresponding relationship between the spatial position and the wireless signal strength. The positioning method is roughly divided into two stages: the training stage and the execution stage. The above S1 is a step in the training stage. The system staff must build a database through field measurement. Specifically, the wireless signal fingerprint database is collected by the system staff in the field at the signal strength of the wireless signal source (including wifi signal source and Bluetooth signal source) at different locations. The wireless signal strength at the location is the signal fingerprint feature at the location. All signal fingerprint features of the same wireless signal source constitute a set of wireless signal fingerprint data. The wireless signal fingerprint database includes several groups of wireless signal fingerprint data. The image database is obtained by the system staff in the field, and the system staff is asked to take a pair (two) of indoor images at the same location, that is, to take a photo with the left hand straightened, and then to take another photo of the same object with the right hand straightened. And the system staff will continuously take pictures as they move, and obtain several pairs of indoor images to form an image database.
[0023] S2. Determine the position when a pair of indoor images is acquired based on a pair of indoor images; the above S2 is also a step in the training stage. In this embodiment, the method for determining the position when a pair of indoor images is acquired is essentially based on angle information. For example: binocular stereo vision is a mechanism that simulates human eyes to perceive surrounding information, that is, the understanding and recognition of surrounding three-dimensional information is realized through imaging perception equipment. Two images of the same object point (the object point is the geometric center of an object) are acquired by cameras at different positions, and then the position of the spatial object point is calculated based on the coordinate deviation information of the corresponding coordinate points in the image, and then the position of the camera relative to the object point is detected according to the computer vision method, so as to obtain the position when a pair of indoor images is acquired. The above positioning method is free from the dependence on the image database with marked position information, and does not require manual survey by professionals. It saves time and effort, and greatly enhances the universality of the positioning method.
[0024] S3, according to the received wireless signal strength of the smart device and the wireless signal fingerprint database, determine the spatial range of the smart device; S4, according to the spatial range and the direction information of the smart device, determine the trapezoidal spatial range of the smart device; S5, according to the trapezoidal range of the smart device, the current photo acquired by the smart device and the image database, determine a pair of indoor images that match the current photo; In the current positioning method, due to the need for global matching, the image matching cost is high and the positioning time is too long, which is one of the main factors restricting the popularization of the existing image-based indoor positioning method. Especially after the positioning subject is converted from professional equipment to ordinary smart phones, the energy consumption and delay problems become more prominent. In this embodiment, in order to solve the above problems, the wireless signal fingerprint database, the wireless signal strength received by the smart device and the direction information displayed by the compass carried by the smart device are used to determine the trapezoidal spatial range of the smart device, and the image-based precise matching is performed within this limited trapezoidal range to obtain the current position of the smart device, thereby reducing unnecessary image matching overhead. The image matching method in this embodiment is a feature-based matching method. The above feature extraction method first extracts feature points (such as corner points, edges, etc.) from the image and generates descriptors for these feature points. Then, the corresponding feature point pairs are found by comparing the feature descriptors of the two images. This method has good robustness to image scale changes, rotations, and brightness changes. Commonly used feature extraction algorithms include SIFT (Scale Invariant Feature Transform), SURF (Speeded Up Robust Features), and ORB (Oriented FAST and Rotated BRIEF). Of course, other feature extraction algorithms or other precise matching of images can also be used. This only provides an implementable method, not a specific limitation.
[0025] S6. Determine the location of the smart device based on a pair of indoor images that match the current photo. When a pair of indoor images are matched, the location of the smart device that took the current photo can be calculated based on the location when the pair of indoor images were obtained, thereby locating the smart device.
[0026] Specifically, in this embodiment, by acquiring a pair of indoor images, the mechanism of human eyes perceiving surrounding information is simulated, the position of the spatial object point is inferred according to the coordinate deviation information of the corresponding coordinate point in the image, and then the position of the camera relative to the object point is detected according to the computer vision method, so as to obtain the position when a pair of indoor images are acquired. The above positioning method is free from the dependence on the image database with marked position information, does not require manual survey by professionals, saves time and effort, and greatly enhances the universality of the positioning method. By setting the wireless signal information and the direction information of the smart device, the accurate position of the smart device is limited to a trapezoidal range, and only the image-based precise matching is required within this limited trapezoidal range to obtain the current position of the smart device and realize the positioning of the smart device. The above positioning method greatly reduces unnecessary image matching overhead, reduces the energy consumption and time required for positioning, and greatly enhances the universality of the positioning method.
[0027] like Figure 3 As shown, further, the above S2 includes the following steps: S21, determining an object point P that exists simultaneously in a pair of indoor images; the object point is the geometric center of an object; S22. Determine the position of the object point P in one of a pair of indoor images The position of the object point P in another indoor image A pair of indoor images includes an indoor image taken with the left hand and an indoor image taken with the right hand, and the position of the object point P in different images is different.
[0028] S23, determine the position of the object point P in the camera coordinate system The position of the object point P in world coordinates ; The position of object point P in the camera coordinate system It is obtained through computer vision technology. Specifically, it can be obtained by automatically processing indoor images using Context Capture photography software to obtain the MVS point cloud within the range of the indoor image, thereby obtaining the position of the object point P in the camera coordinate system. , the position of the object point P in the world coordinates It is to be determined. In this embodiment, the indoor image obtained does not need to carry location information, avoiding the need for manual survey by professionals.
[0029] S24. According to the perspective principle, the following calculation formula is obtained: Where: represents the focal length, B represents the distance between the origins of a pair of indoor images; S25. Calculation method for determining the position of object point P in world coordinates: Where: represents the rotation matrix, represents the translation matrix, is the scaling factor; S26. Determine the position when a pair of indoor images are acquired based on the position of the object point P in the world coordinate system and its position in the camera coordinate system. Two indoor images of the same object point are acquired by cameras at different positions, and then the position of the object point is inferred based on the coordinate deviation information of the corresponding coordinate points in the indoor image. Then, the position of the camera relative to the object point is detected based on the computer vision method, thereby obtaining the position when a pair of indoor images are acquired. The above positioning method is free from the dependence on the image database marked with position information, and does not require manual survey by professionals. It saves time and effort, and greatly enhances the universality of the positioning method.
[0030] Furthermore, the rotation matrix in the above S25 is , translation matrix and scaling factor All are to be sought. In the above S25 The method of determining is as follows: S251, determining the position of the coordinate origin of one of a pair of indoor images in the image plane and the position of the other coordinate origin in the image plane , and determine the coordinates of the object point P in the image plane ; S252, construct the actual distance and The mapping relationship between them is as follows: Where: The angle information when the smart device takes an indoor image. The angle information when taking another indoor image, , , are error variables; there is a scaling factor in the image plane. The origin of one of the indoor images refers to the position of the camera when the indoor image is taken in the image plane. Similarly, the origin of the other indoor image refers to the position of the camera when the other indoor image is taken in the image plane. The scaling factor can be obtained based on the distance between the two origins in the image plane and the distance in the actual physical space. , and since there may be errors when users take photos with smart devices, the error control variable is introduced . Based on the angle information obtained by the smart device compass and , and according to and Establish as Figure 4 The three-sided positioning model shown in the figure, without loss of generality, sets the coordinates of the photographed object as the origin of the coordinate system, then the physical coordinates of the two cameras can be expressed as and Ideally, the distance between these two coordinates should be equal to the distance between the photos found in the first step. In order to eliminate the error of the compass data of the smart device, a new error variable is introduced for each pair of image data and By minimizing , , , you can get the zoom factor .
[0031] S253, according to the actual distance and The mapping relationship between them is used to construct an optimization function; the calculation formula of the optimization function is as follows: S254, Solving by optimizing functions and optimization models The optimization model is used to minimize , , The specific optimization model is not limited here. In this embodiment, an implementable optimization model is provided, and the optimization model includes the following steps: A1, initialize particle swarm; A2. Determine the global optimal value of each particle; A3. Determine the fitness corresponding to the local optimal value of each particle; A4. Update the individual and global best positions according to fitness; A5. Determine whether the global optimal position meets the minimum limit or reaches the maximum number of iterations; A6. When the global optimal position does not meet the minimum limit and has not reached the maximum number of iterations, repeat step A2; when the global optimal position meets the minimum limit or reaches the maximum number of iterations, output the result.
[0032] There are three parameters to be solved in this model, so it is necessary to search in 3D space. Specifically, the calculation method of the global optimal value of each particle is as follows: Where: represents the current position of particle j, The updated position of particle j, represents the current velocity of particle j, represents the updated velocity of particle j, Indicates the position corresponding to the individual extreme value of particle j at the current iteration number; It represents the global best position corresponding to the iteration of all particles at the current moment. represents the weight coefficient, represents the first speed coefficient, represents the second speed coefficient, preferably, , represents a random function, The corresponding value range is [0,1], which is used to increase the randomness of the search. The current speed of particle j is related to three factors. The first factor is affected by the speed of particle j at the previous moment, and the corresponding weight is affected by the weight coefficient The second factor is affected by the distance between the current position of particle j and the position corresponding to the individual extreme value, through the first speed coefficient The third factor is affected by the distance between the current position of particle j and the global optimal position, through the first speed coefficient control; The corresponding global optimal position in the particle swarm and the position corresponding to the individual extreme value are mainly discriminated and analyzed through the fitness function. Specifically, the fitness calculation method is as follows: Where: is the fitness function. During the iterative update process of the particle swarm, the minimum value of the fitness function under the scope is found. Finally, the optimal .
[0033] Rotation Matrix The specific solution method includes: roughly estimating the rotation direction through the compass data of the smart device. Let and Represent the direction of the i-th photo in the physical world and in the image space, respectively. is the rotation angle we want to obtain, and we introduce Characterize the absolute compass angle measurement error of each image. Then the first step of the rotation solution is transformed into the following optimization problem: According to the above optimization problem, an optimization function is established, and then the solution is obtained through the optimization model. The aforementioned solution process can refer to the aforementioned optimization process, which will not be repeated here.
[0034] In addition, we project the 3D model into the physical world to make the 3D model structure correspond to the physical world structure. Taking the photo of an ordinary room as an example, we find the door of the room in the top view by analyzing the behavior of smart devices and the structure of the 3D model. By matching the door in the 3D model with the door of the store in the actual physical space, we can find the rotation correspondence between the 3D model and the physical world. The above provides a rotation matrix The solution method is as follows: The solution is a solution method, but it should not be understood as a limitation of this embodiment. The rotation matrix The solution is a mature existing technology, and the rotation matrix is solved by other methods. It should not be understood as being different from the present embodiment.
[0035] Translation Matrix The solution methods include: The key to solving the translation vector of the 3D model is to find a corresponding point in the image plane and the actual physical space. Knowing both its coordinates in the image plane and its position in the actual physical space, the translation matrix can be obtained. To this end, we can mine the photo-taking behavior and image structure information of smart devices, and find the corresponding feature points more easily by finding the visual focus of the smart device and the physical features of the corners of the building structure. The solution of is only one solution method, but it should not be understood as a limitation of this embodiment. The translation matrix The solution is a mature existing technology, and the translation matrix is solved by other methods. It should not be understood as being different from the present embodiment.
[0036] Furthermore, the above S3 includes the following steps: S31. According to the corresponding relationship between wireless signal strength and spatial position, determine the mapping relationship between wireless signal strength and actual physical space; the above mapping relationship is obtained by transfer learning. Here is a brief theoretical basis for using transfer learning to roughly estimate the position of smart devices: First, wireless signal strength and spatial position have a strong neighbor constraint relationship: two locations with close distances in the actual physical space should also have similar received wireless signal strengths; conversely, if the wireless signal strengths received at two locations are consistent, then they are very likely to be close in the actual physical space. Secondly, wireless signal strength and spatial position have a common low-dimensional mapping relationship: wireless signals can be collected at a few determined locations to ensure a common low-dimensional mapping relationship. The above mapping relationship can be obtained through a small amount of training data, and then a smart device location range can be obtained through the mapping relationship and the current wireless signal of the smart device.
[0037] S32. Determine the spatial range of the smart device based on at least three wireless signals. Triangulation positioning is achieved through at least three wireless signals to obtain the spatial range of the smart device. Triangulation positioning is a positioning technology based on geometric principles. It determines the position of the target point by measuring the distance or angle between a known reference point and the target point. On a two-dimensional plane, at least two reference points are required to determine the position of a target point, while in a three-dimensional space, at least three reference points are required.
[0038] Furthermore, the method for confirming the mapping relationship between the wireless signal strength and the actual physical space in the above S31 is: Where: X represents the actual physical space dataset, Y represents the wireless signal fingerprint distribution dataset, i and j represent the index of the dataset, A function that represents the mapping of a real physical space dataset to a common low-dimensional manifold, A function that represents the mapping of a wireless signal fingerprint distribution dataset to a common low-dimensional manifold, Represents the neighbor constraint weight relationship of the actual physical space dataset, Represents the neighbor constraint weight relationship of the wireless signal fingerprint distribution dataset, It represents the mapping relationship between the actual physical space dataset and the wireless signal fingerprint distribution dataset at low latitude. is a weight coefficient used to adjust the objective function.
[0039] Furthermore, the above S4 includes the following steps: S41, obtaining direction information according to the smart device; S42, determining the trapezoidal range where the smart device is located according to the direction information and the spatial range where the smart device is located, and the spatial range is within the trapezoidal range. The trapezoidal spatial range where the smart device is located is determined by using the mapping relationship between the wireless signal strength and the actual physical space, the wireless signal strength received by the smart device, and the direction information displayed by the compass carried by the smart device, and performing image-based precise matching within this limited trapezoidal range to obtain the current position of the smart device, thereby reducing unnecessary image matching overhead.
[0040] like Figure 5 As shown, further, in this embodiment, the shooting model of the smart device to obtain the current photo is optimized, with the diameter of the smart device camera as the short side, the viewing angle of the smart device camera as the extension angle of the waists of the two isosceles trapezoids, the direction of the smart device camera as the direction of the isosceles trapezoid height, and the current depth of field of the smart device camera as the length of the isosceles trapezoid height, forming an isosceles trapezoid camera range. Based on the trapezoidal representation of the shooting model, we perform regional matching on the images in the database instead of global matching to reduce the image matching overhead.
[0041] Embodiment 2 The same parts in this embodiment as those in the first embodiment are given the same figure marks, and the same text descriptions are omitted.
[0042] The present embodiment discloses an indoor positioning device based on multi-sensor information fusion. The positioning device 10 uses the positioning method in the first embodiment and includes: a database module 110, which is used to establish an image database and a wireless signal fingerprint database; a position acquisition module 120, which is used to determine the position when a pair of indoor images are acquired based on a pair of indoor images; a position estimation module 130, which is used to determine the spatial range of the smart device based on the received wireless signal strength of the smart device and the wireless signal fingerprint database; an optimization module 140, which is used to determine the trapezoidal spatial range of the smart device based on the spatial range and the direction information of the smart device; a matching module 150, which is used to determine a pair of indoor images matching the current photo based on the trapezoidal range of the smart device, the current photo acquired by the smart device and the image database; a positioning module 160, which is used to determine the position of the smart device based on the pair of indoor images matching the current photo.
[0043] The positioning device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), etc., and the non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), etc., which is not specifically limited in the embodiment of the present application.
[0044] The positioning device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0045] Embodiment 3 The same parts as those in the first and second embodiments of the present embodiment are given the same reference numerals, and the same text descriptions are omitted.
[0046] This embodiment discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the positioning method in the first embodiment is implemented.
[0047] Embodiment 4 The same parts as those in the first, second and third embodiments of the present embodiment are given the same reference numerals, and the same textual descriptions are omitted.
[0048] This embodiment discloses a storage medium on which a computer program is stored. When the computer program is executed by a processor, the positioning method in the first embodiment is implemented. The processor is a processor in the electronic device described in the above embodiment. The storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0049] Embodiment 5 The same parts of this embodiment as those of Embodiment 1, Embodiment 2, Embodiment 3 and Embodiment 4 are given the same figure marks, and the same text descriptions are omitted.
[0050] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned positioning method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0051] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0052] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
Claims
1. An indoor positioning method based on multi-sensor information fusion, characterized in that: The positioning method includes the following steps: S1. Establishing an image database and a wireless signal fingerprint database, wherein the image database includes a plurality of pairs of indoor images, and the wireless signal fingerprint database includes a correspondence between a spatial position and a wireless signal strength; S2. determining, based on the pair of indoor images, a position at which the pair of indoor images are acquired; S3, determining the spatial range of the smart device according to the wireless signal strength received by the smart device and the wireless signal fingerprint database; S4. Determine the trapezoidal spatial range where the smart device is located according to the spatial range and the direction information of the smart device; S5. Determine a pair of the indoor images matching the current photo according to the trapezoidal range in which the smart device is located, the current photo acquired by the smart device, and the image database; S6. Determine the location of the smart device according to a pair of the indoor images that match the current photo.
2. The indoor positioning method based on multi-sensor information fusion according to claim 1, characterized in that: The above S2 includes the following steps: S21, determining an object point P that exists simultaneously in a pair of the indoor images; S22, determining the position of the object point P in one of the pair of indoor images The position of the object point P in another indoor image ; S23, determine the position of the object point P in the camera coordinate system The position of the object point P in world coordinates ; S24. According to the perspective principle, the following calculation formula is obtained: Where: represents the focal length, and B represents the distance between a pair of origins of the indoor images; S25, a calculation method for determining the position of the object point P in the world coordinates: Where: represents the rotation matrix, represents the translation matrix, is the scaling factor; S26. Determine the position for acquiring the pair of indoor images according to the position of the object point P in the world coordinate system and its position in the camera coordinate system.
3. The indoor positioning method based on multi-sensor information fusion according to claim 1, characterized in that: In the above S25 The method of determining is as follows: S251: Determine the position of the coordinate origin of one of the indoor images in the image plane and the position of the other coordinate origin in the image plane , and determine the coordinates of the object point P in the image plane ; S252: constructing a coordinate system of the actual distance and the indoor image coordinate system according to the actual distance between the indoor image coordinate origins. The mapping relationship between them is as follows: Where: The angle information when the smart device takes an indoor image. The angle information when taking another indoor image, , , All are error variables; S253, according to the actual distance and The mapping relationship between them builds the optimization function; S254, Solving by optimizing functions and optimization models .
4. The indoor positioning method based on multi-sensor information fusion according to claim 1, characterized in that: The above S3 includes the following steps: S31, determining a mapping relationship between wireless signal strength and actual physical space according to the corresponding relationship between wireless signal strength and spatial position; S32: Determine a spatial range where the smart device is located according to at least three wireless signals.
5. The indoor positioning method based on multi-sensor information fusion according to claim 4, characterized in that: The method for confirming the mapping relationship between the wireless signal strength and the actual physical space in the above S31 is: Where: X represents the actual physical space dataset, Y represents the wireless signal fingerprint distribution dataset, i and j represent the index of the dataset, A function that represents the mapping of a real physical space dataset to a common low-dimensional manifold, A function that represents the mapping of a wireless signal fingerprint distribution dataset to a common low-dimensional manifold, Represents the neighbor constraint weight relationship of the actual physical space dataset, Represents the neighbor constraint weight relationship of the wireless signal fingerprint distribution dataset, It represents the mapping relationship between the actual physical space dataset and the wireless signal fingerprint distribution dataset at low latitude. is a weight coefficient used to adjust the objective function.
6. The indoor positioning method based on multi-sensor information fusion according to claim 1, characterized in that: The above S4 includes the following steps: S41, obtaining direction information according to the smart device; S42: Determine a trapezoidal range in which the smart device is located according to the direction information and the spatial range in which the smart device is located, wherein the spatial range is within the trapezoidal range.
7. An indoor positioning device based on multi-sensor information fusion, the positioning device using the positioning method according to any one of claims 1 to 6, characterized in that: include: Database module, used to establish image database and wireless signal fingerprint database; A position acquisition module, used for determining a position when acquiring a pair of indoor images according to the pair of indoor images; A location estimation module, used to determine the spatial range of the smart device according to the received wireless signal strength of the smart device and the wireless signal fingerprint database; An optimization module, configured to determine the trapezoidal spatial range where the smart device is located according to the spatial range and the direction information of the smart device; A matching module, configured to determine a pair of the indoor images matching the current photo according to the trapezoidal range in which the smart device is located, the current photo acquired by the smart device and the image database; A positioning module is used to determine the position of the smart device according to a pair of the indoor images matching the current photo.
8. A computer, characterized in that: The method comprises a processor, a memory and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the positioning method as described in claims 1 to 6.
9. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the positioning method according to claims 1-6 are implemented.
10. A chip, characterized in that: The chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the positioning method as described in claims 1-6.
