A method and device for indoor positioning of intelligent mobile terminals based on top-view visual data

Abstract

A method and apparatus for indoor positioning of intelligent mobile terminals based on top-view visual data are disclosed. The method includes: 1) Data acquisition: facing the top with a camera, acquiring visual data from the top view, and transmitting the acquired visual data to a subsequent target coarse extraction module; 2) Target coarse extraction: firstly, using YOLO V7 to detect top anchor points, and roughly extracting the anchor point regions in the image based on the detection results; 3) Target fine extraction: further processing the coarsely extracted regions using edge detection methods to finely extract the anchor points; 4) Inputting the target fine extraction results and prior information into a position calculation module based on top-view data to obtain the location of the intelligent mobile terminal, thus achieving indoor positioning based on top-view data. This invention enables indoor positioning of intelligent mobile terminals based on top-view visual data, providing better location services for other upper-layer applications.

Description

Technical Field

[0001] This invention relates to the field of navigation and positioning technology, and in particular to an indoor positioning method and device for intelligent mobile terminals based on top-view visual data. Background Technology

[0002] Shopping malls, libraries, subway stations, and other indoor areas are primary settings for people's daily lives. High-precision indoor positioning systems can provide reliable location information in GNSS-denied environments, significantly impacting the quality of daily life. Furthermore, indoor positioning plays a crucial role in emergency rescue operations such as search and rescue and firefighting. Therefore, the demand for high-precision indoor positioning is constantly increasing in today's society.

[0003] In recent years, the rapid proliferation of smart mobile devices (such as smartphones, wearable devices, and commercial robots) has brought about significant changes to indoor positioning, while also posing new challenges to indoor positioning systems. With the development of microelectromechanical systems (MEMS), more and more sensors are being integrated into modern smart devices, further driving the development of indoor positioning technology. Currently, there are many methods for achieving indoor positioning, such as Bluetooth, inertial sensors, and received signal strength index (RSSI). Among these, vision-based positioning methods are currently one of the mainstream methods for indoor positioning, utilizing computer vision information to achieve indoor positioning. Visual positioning methods use visual sensors to collect visual data and analyze this data to extract relevant features and determine the target location.

[0004] Currently, vision-based localization methods typically rely on visual data from side-view perspectives. However, side-view visual data is easily affected by the surrounding environment. In real-world scenarios, numerous obstacles typically exist within the side-view field of view, including both dynamic obstacles (such as pedestrians and other moving objects) and static obstacles (such as temporary billboards and posters). These obstacles often obscure target features, increasing the difficulty of image feature extraction and recognition, ultimately affecting localization accuracy. Furthermore, side-view visual data may involve privacy issues, including the privacy of pedestrians collected and scenes that should not be disclosed. Summary of the Invention

[0005] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provide an indoor positioning method for intelligent mobile terminals based on top-view visual data that is real-time, simple and reliable.

[0006] The first aspect of this invention relates to an indoor positioning method for an intelligent mobile terminal based on top-view visual data, comprising the following steps:

[0007] Step 1, Data Acquisition: Point the camera upwards and use the camera to collect visual data from the top viewpoint;

[0008] Step 2, Coarse target extraction: Use the YOLO V7 model to detect anchor points in the top-view visual data, and extract the region where the target anchor points are located in the visual data based on the detection results;

[0009] Step 3, Target Refinement: Exposure compensation is applied to the area extracted in Step 2, and then the anchor points are precisely extracted using the edge detection operator to obtain the size of the anchor points in the pixel coordinate system;

[0010] Step 4, prior information acquisition: Measure the actual size of the anchor point, calibrate the ground projection point of the anchor point's centroid in the real coordinate system, and read the camera's focal length parameters;

[0011] Step 5, Position Calculation Based on Top-View Information: Calculate the scale transformation ratio K between the image coordinate system and the real coordinate system based on the anchor point pixel size and the actual anchor point size; calculate the distance D between the anchor point centroid and the image center point in the pixel coordinate system; combine the scale transformation ratio K, distance D, and the projection position of the anchor point centroid on the ground in the real coordinate system to calculate the initial position of the target using the principle of similar triangles; calculate the height H between the camera and the ceiling using the scale transformation ratio K and the camera focal length f; calculate the attitude θ of the smart device based on the data from the inertial sensor built into the smart mobile device; calculate the position offset using the principle of trigonometric functions based on the height H and the device's attitude θ; correct the initial position using the position offset to obtain the final positioning result.

[0012] In step 2, the YOLOv7 model is used to detect anchor points in the top-view visual data and extract the region where the target anchor point is located in the visual data, as detailed below:

[0013] For the selected anchor points, a YOLO V7 model is trained using a pre-calibrated dataset. The trained YOLO V7 model is then used to detect anchor points in the top-view visual data, obtaining the centroid coordinates of the anchor points and the region where the anchor points are located.

[0014] In step 3, the anchor points are accurately extracted using an edge detection operator to obtain their pixel dimensions, as detailed below:

[0015] Exposure compensation is applied to the area where the anchor point is located; for the selected anchor point, an appropriate edge operator is selected to accurately extract the anchor point and obtain the size of the anchor point in the pixel coordinate system.

[0016] In step 5, the target position is calculated and corrected using the prior information from step 4, the device's pose θ, and the anchor point pixel size, as detailed below:

[0017] First, using the anchor point pixel size and the actual anchor point size, calculate the scale transformation ratio K between the image coordinate system and the real coordinate system;

[0018]

[0019] Where L_real is the actual size of the anchor point, and L_pixel is the pixel size of the anchor point;

[0020] Secondly, the initial position of the target is calculated using the principle of similar triangles;

[0021] (x0, y0) = (x l y l )+(Δx r ,△y r )

[0022] =(x l y l )+K*(Δx p ,△y p (2)

[0023] Among them, (x l y l (Δx) represents the projection of the anchor point's centroid onto the ground in the real coordinate system. r ,△y r () represents the distance between the centroid of the anchor point and the center point of the image in the pixel coordinate system;

[0024] Then, based on the scale conversion ratio K and the camera focal length f, the height H between the camera and the ceiling is calculated;

[0025] H=K*f (3)

[0026] The device's attitude θ is calculated based on data from the built-in inertial sensors of the smart terminal.

[0027] Finally, based on the height H and the device's attitude θ, the position offset is calculated using the principle of trigonometric functions, and the initial position (x0, y0) is corrected to obtain the final positioning result (x1, y1).

[0028] (x1, y1) = (x0, y0) ± (x bias ,y bias )

[0029] =(x0,y0)±H*tan(θ) (4).

[0030] The visual data mentioned in step 1 includes images and videos.

[0031] A second aspect of the present invention relates to an indoor positioning device for an intelligent mobile terminal based on top-view visual data, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the indoor positioning method for an intelligent mobile terminal based on top-view visual data of the present invention.

[0032] A third aspect of the invention relates to a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the intelligent mobile terminal indoor positioning method of the present invention based on top-view visual data.

[0033] Compared with the prior art, the advantages of this invention are: (1) It uses top visual data for positioning, which has low image scene complexity and is easier to implement high-precision, real-time target detection and extraction; (2) It uses top visual data for positioning and selects top objects as anchor points. Since there are relatively few decorations and changes to top objects in real life, their variability is small, and the calibration of prior information is simple and not easy to change; (3) It uses top anchor points for positioning, which does not require additional external nodes, has high portability and low cost; (4) It uses top visual data for positioning, which is more conducive to protecting the privacy of pedestrians and specific scenes; (5) It uses top visual data for positioning, which is more in line with the natural state of pedestrians holding mobile phones when navigating, and has more practical significance; (6) It can achieve high-precision, real-time and convenient indoor positioning of intelligent terminals, which plays an important role in applications such as pedestrian indoor navigation and commercial robots. Attached Figure Description

[0034] Figure 1 This is a framework diagram of the method of the present invention.

[0035] Figure 2 This is a schematic diagram of top-view data acquisition according to the present invention.

[0036] Figure 3 This is a schematic diagram of the YOLO V7 framework for achieving coarse extraction of the target according to the present invention.

[0037] Figure 4 This is a flowchart of the target coarse extraction module of the present invention.

[0038] Figure 5 This is a flowchart of the target fine extraction module of the present invention.

[0039] Figure 6 This is a schematic diagram of the position calculation process based on top-view data of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0042] Example 1

[0043] This embodiment relates to an indoor positioning method for an intelligent mobile terminal based on top-view visual data, including step 1 data acquisition, step 2 coarse target extraction, step 3 fine target extraction, step 4 obtaining prior information, and step 5 position calculation based on top-view data.

[0044] An indoor positioning method for intelligent mobile terminals based on top-view visual data includes the following steps:

[0045] Step 1: Point the camera upwards and use the camera to collect visual data (images or videos) from the top viewpoint, thereby achieving data acquisition 1, and use the data as input for target coarse extraction 2;

[0046] In step 2, coarse target extraction 2, the YOLOv7 model is used to detect anchor points (such as overhead lights) in the top-view visual data. Based on the detection results, the region of the anchor point in the image is roughly extracted, and the coarsely extracted region is used as the input for fine target extraction 3.

[0047] In step 3, target fine extraction 3, firstly, exposure compensation is performed on the coarse extraction area, and then edge detection operators (such as Canny operator, Sobel operator and Laplace operator) are used to finely extract the anchor points, thereby obtaining the size of the anchor points in the pixel coordinate system, and the result and prior information 4 are used as input for position calculation 5 based on top view data;

[0048] Step 5: In the position calculation stage based on top-view data, the scale transformation ratio K between the image coordinate system and the real coordinate system is calculated using the anchor point pixel size and the actual anchor point size (obtained from prior information 4). Based on K and the actual projection position of the anchor point's centroid on the ground (x... l y l (obtained from prior information 4) and the pixel distance (Δx) from the anchor point to the center of the image. r ,Δy rThe initial position of the target is calculated using the principle of similar triangles. Simultaneously, the height H between the camera and the ceiling can be calculated based on the ratio K and the focal length f of the upper camera (obtained from prior information 4). The attitude θ (shooting angle) of the smart device can be calculated using the inertial sensor built into the smart mobile device. Based on the height H and the device's attitude θ, the position offset can be calculated using trigonometric functions, thereby compensating for the initial position, achieving position correction, and obtaining the final positioning result.

[0049] In step 2, YOLOv7 is used to process the top-view data to achieve target detection and roughly extract the region where the target is located in the image.

[0050] In step 3, based on the coarse extraction results, edge detection operators (such as Canny operator, Sobel operator and Laplace operator) are used to refine the anchor points, thereby achieving fine extraction of the target and obtaining the pixel size of the anchor points.

[0051] In step 4, this result and prior information 4 are used as inputs for the position calculation 5 based on the top-view data to realize the position calculation and perform position correction, thereby obtaining the final positioning result, as follows:

[0052] First, using the anchor point pixel size and the actual anchor point size (obtained from prior information 4), the scale transformation ratio K between the image coordinate system and the real coordinate system is calculated.

[0053]

[0054] Where L_real is the actual size of the anchor point, and L_pixel is the pixel size of the anchor point obtained by the target fine extraction 3.

[0055] Then, the initial position of the target is calculated using the principle of similar triangles.

[0056] (x0, y0) = (x l y l )+(Δx r ,△y r )

[0057] =(x l y l )+K*(Δx p ,△y p (2)

[0058] Among them, (x l y l ) represents the projected position of the anchor point's centroid on the ground in real-world coordinates (obtained from prior information 4), (Δx) r ,△y r ) represents the pixel distance from the anchor point to the center of the image in the image coordinate system.

[0059] Next, based on K and the focal length f of the upper camera, the height H between the camera and the ceiling can be calculated.

[0060] H=K*f (3)

[0061] The attitude of the smart terminal, i.e., the shooting angle θ, can be obtained by using the built-in inertial sensor of the smart terminal.

[0062] Finally, based on the height H and the device's attitude θ, the position offset is calculated using trigonometric functions to compensate for the initial position (x0, y0), thereby achieving position correction and obtaining the final positioning result (x1, y1).

[0063] (x1, y1) = (x0, y0) ± (x bias ,y bias )

[0064] = (x0, y0) ± H*tan(θ) (4)

[0065] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0066] Example 2

[0067] This embodiment takes pedestrian navigation as an example. It determines the position of the smartphone (pedestrian) based on the top-view visual data collected by the mobile phone camera. It uses a target detection algorithm to process the top-view data, selects the top light as the target anchor point, extracts the pixel size of the selected top light, and combines the prior information of the top light to realize the position calculation and correction of the smartphone (pedestrian). After the entire process is completed, the above steps are repeated to start a new round of data collection and positioning calculation.

[0068] 1. Framework diagram of an indoor positioning method for intelligent mobile terminals based on top-view visual data

[0069] Framework diagram of an indoor positioning method for intelligent mobile terminals based on top-view visual data, as shown below. Figure 1 As shown, it consists of the following modules: data acquisition 1, target coarse extraction 2, target fine extraction 3, prior information 4, and position calculation based on top-view data 5; data acquisition 1 serves as the input to target coarse extraction 2; the result of target coarse extraction 2 serves as the input to target fine extraction 3; the result of target fine extraction 3 and prior information 4 serve as the input to position calculation based on top-view data 5, ultimately achieving position correction and obtaining the final positioning result;

[0070] Data acquisition 1 utilizes a mobile phone camera, facing upwards to collect visual data (images or videos) from a top-view perspective, serving as input for coarse target extraction 2. Next, the YOLO V7 model is used to detect the top light in the top-view visual data. Based on the detection results, the area of ​​the top light in the image is roughly extracted, and this coarsely extracted area is used as input for fine target extraction 3. Then, exposure compensation is applied to the coarsely extracted area, followed by fine extraction of the top light using edge detection operators (such as Canny, Sobel, and Laplace operators), thereby obtaining the size of the top light in the pixel coordinate system. This result, along with prior information 4, is used as input for position calculation 5 based on the top-view data. Finally, using the pixel size of the top light and its actual size (obtained from prior information 4), the scale transformation ratio K between the image coordinate system and the real coordinate system is calculated. Based on K and the projection position (x, y) of the top light's centroid on the ground in reality... l y l (Obtained from prior information 4) and the pixel distance (Δx) from the centroid of the top light to the center of the image. r ,Δy r The initial position of the target is calculated using the principle of similar triangles. Simultaneously, the height H between the camera and the ceiling can be calculated based on the ratio K and the focal length f of the upper camera (obtained from prior information 4). The attitude θ (shooting angle) of the smart device can be calculated using the inertial sensor built into the smart mobile device. Based on the height H and the device's attitude θ, the position offset can be calculated using trigonometric functions, thereby compensating for the initial position, achieving position correction, and obtaining the final positioning result.

[0071] 2. Top-view data acquisition

[0072] Smart devices, including smartphones, wearable devices, and commercial robots, utilize cameras to acquire top-view visual data. During the acquisition process, the camera is kept plane-upward to capture visual data from a top-view perspective.

[0073] 3. YOLO V7 Framework

[0074] YOLO V7, short for "You Only Look Once," is a real-time, high-precision object detection algorithm in the field of computer vision. YOLO V7 introduces several architectural reforms aimed at improving accuracy while maintaining high detection speed. In terms of architectural reform, YOLO V7 incorporates E-ELAN (Extended Efficient Layer Aggregation Network) into its backbone. Another major architectural reform introduced in YOLO V7 is composite model scaling. This method balances accuracy and detection speed to meet the needs of different applications. YOLO V7 offers the dual advantages of high flexibility and high accuracy.

[0075] 4. Target coarse extraction module

[0076] like Figure 4 As shown, the YOLO V7 model is used to detect the top light in the top-view visual data. Based on the detection results, the area of ​​the top light in the image is roughly extracted, thus obtaining the coarse extraction result of the top light in the image.

[0077] 5. Target Refinement Module

[0078] like Figure 5 As shown, firstly, exposure compensation is applied to the coarsely extracted area, and then edge detection operators (such as Canny operator, Sobel operator and Laplace operator) are used to refine the extraction of the top light, thereby obtaining the size of the top light in the pixel coordinate system.

[0079] 6. Position calculation based on top-view data

[0080] Position calculation methods based on top-view data, such as... Figure 6 As shown, it mainly includes the following steps: proportional calculation, position calculation, position correction and positioning output.

[0081] Because the position of the dome light varies in the top view image captured when the phone's camera is positioned differently below it—for example, when the camera is directly below the dome light, the dome light should be centered in the image—the position of the camera in the actual coordinate system can be deduced based on the pixel distance between the dome light's position and the image center, using the principle of similar triangles.

[0082] First, using the pixel size of the top light and the actual size of the top light (obtained from prior information), calculate the scale transformation ratio K between the image coordinate system and the real coordinate system.

[0083]

[0084] Where L_real is the actual size of the top light, and L_pixel is the pixel size of the top light obtained from the target fine extraction.

[0085] Then, the initial position of the camera is calculated using the principle of similar triangles.

[0086] (x0, y0) = (x l y l )+(Δx r ,Δy r )

[0087] =(x l y l )+K*(Δx p ,Δy p (2)

[0088] Among them, (x l y l ) represents the projected position of the centroid of the overhead light on the ground in real-world coordinates (obtained from prior information), (Δx) r ,Δy r ) represents the pixel distance from the centroid of the top light to the center of the image in the image coordinate system.

[0089] In practical applications, taking smartphone-based pedestrian navigation as an example, the phone camera often cannot maintain a perfectly horizontal position when collecting top-view data. When there is an angle between the phone's surface and the horizontal plane, the field of view captured by the phone camera will be distorted. Figure 6 As shown in the image. The calculated position at this point is not the actual position of the phone, so the initially calculated position needs to be corrected.

[0090] like Figure 6 As shown, to perform position correction, the height H between the camera and the ceiling must first be calculated based on K and the focal length f of the upper camera.

[0091] H=K*f (3)

[0092] The phone's attitude, i.e., its shooting angle θ, can be obtained using the phone's built-in inertial sensors (accelerometer and gyroscope). Finally, based on the height H and the phone's attitude θ, trigonometric functions are used to calculate the position offset, thereby compensating for the initial position (x0, y0), achieving position correction, and obtaining the final positioning result (x1, y1).

[0093] (x1,y1)=(x0,y0)±(x bias ,y bias )

[0094] = (x0, y0) ± H*tan(θ) (4)

[0095] The obtained positioning result, obtained through preliminary position calculation and subsequent position correction, will be used as the final position output. The above steps are repeated throughout the navigation and positioning process to achieve continuous indoor positioning.

[0096] Example 3

[0097] This embodiment relates to an indoor positioning device for an intelligent mobile terminal based on top-view visual data, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the indoor positioning method for an intelligent mobile terminal based on top-view visual data of embodiment 1 or 2.

[0098] Example 4

[0099] This embodiment relates to a computer-readable storage medium storing a program that, when executed by a processor, implements the indoor positioning method for intelligent mobile terminals based on top-view visual data of this embodiment 1 or 2.

Claims

1. An indoor positioning method for intelligent mobile terminals based on top-view visual data, characterized in that, Includes the following steps: Step 1, Data Acquisition: Point the camera upwards and use the camera to acquire visual data (images or videos) from the top viewpoint. Step 2, Coarse target extraction: Use the YOLO V7 model to detect anchor points in the top-view visual data, and extract the region where the target anchor points are located in the visual data based on the detection results; Step 3, Target Refinement: Exposure compensation is applied to the area extracted in Step 2, and then the anchor points are precisely extracted using the edge detection operator to obtain the size of the anchor points in the pixel coordinate system; Step 4, prior information acquisition: Measure the actual size of the anchor point, calibrate the ground projection point of the anchor point's centroid in the real coordinate system, and read the camera's focal length parameters; Step 5, Position calculation based on top-view information: Calculate the scale transformation ratio K between the image coordinate system and the real coordinate system according to the anchor point pixel size and the actual anchor point size; calculate the distance D between the anchor point centroid and the image center point in the pixel coordinate system; Combining the scale transformation ratio K, distance D, and the projection position of the anchor point's centroid on the ground in the real coordinate system, the initial position of the target is calculated using the principle of similar triangles. The height H between the camera and the ceiling is calculated using the scale transformation ratio K and the camera's focal length f. The attitude θ of the smart device is calculated based on data from the inertial sensor built into the smart mobile device. The position offset is calculated using trigonometric functions based on the height H and the device's attitude θ. The initial position is corrected using the position offset to obtain the final positioning result. The details are as follows: First, using the anchor point pixel size and the actual anchor point size, calculate the scale transformation ratio K between the image coordinate system and the real coordinate system; (1) in, The actual dimensions of the anchor point. The anchor point pixel size; Secondly, the initial position of the target is calculated using the principle of similar triangles; (2) in, This represents the projection of the anchor point's centroid onto the ground in the real-world coordinate system. This is the distance between the centroid of the anchor point and the center point of the image in the pixel coordinate system. Then, based on the scale conversion ratio K and the camera focal length f, the height H between the camera and the ceiling is calculated; f(3) The device's attitude θ is calculated based on data from the built-in inertial sensors of the smart terminal. Finally, based on the height H and the equipment's attitude θ, the position offset is calculated using trigonometric functions to determine the initial position. Make corrections to obtain the final positioning result. ; (4) 2. The indoor positioning method for intelligent mobile terminals based on top-view visual data according to claim 1, characterized in that, In step 2, the YOLO V7 model is used to detect anchor points in the top-view visual data, and the region where the target anchor point is located in the visual data is extracted, as follows: For the selected anchor points, a YOLO V7 model is trained using a pre-labeled dataset. The trained YOLO V7 model is then used to detect anchor points in the top-view visual data, obtaining the centroid coordinates of the anchor points and the region where the anchor points are located.

3. The indoor positioning method for intelligent mobile terminals based on top-view visual data according to claim 1, characterized in that, In step 3, the anchor points are accurately extracted using edge detection operators to obtain their pixel dimensions, as detailed below: Exposure compensation is applied to the area where the anchor point is located; for the selected anchor point, an appropriate edge operator is selected to accurately extract the anchor point and obtain the size of the anchor point in the pixel coordinate system.

4. The indoor positioning method for intelligent mobile terminals based on top-view visual data according to claim 1, characterized in that, The visual data mentioned in step 1 includes images and videos.

5. An indoor positioning device for an intelligent mobile terminal based on top-view visual data, characterized in that, The device includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the indoor positioning method for an intelligent mobile terminal based on top-view visual data as described in any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the indoor positioning method for intelligent mobile terminals based on top-view visual data as described in any one of claims 1-4.

Images