Positioning method and system for improving flexibility of visible light positioning system

Abstract

The application discloses a positioning method and system for improving flexibility of a visible light positioning system and belongs to the technical field of communication networks.The method comprises the following steps: on one hand, the application designs an indoor visible light positioning algorithm based on a smartphone in a tilted state, so that the smartphone can still achieve centimeter-level positioning accuracy in a three-dimensional tilted state.On the other hand, the application designs a functional module segmentation method, so that the intelligent terminal can exchange the least bandwidth resources for the most computing resources of the intelligent terminal, and the problems of high requirement on terminal computing capacity and poor system scalability of the visible light positioning system based on the intelligent terminal are solved, and the system flexibility and practicability of the indoor visible light positioning system are greatly improved, and user experience is improved.

Description

Technical Field

[0001] This invention belongs to the field of communication network technology and relates to a positioning algorithm and functional module segmentation method for visible light positioning under three-dimensional tilt state in indoor LED (Light Emitting Diode) lights. Background Technology

[0002] In recent years, with economic and social development, the demand for indoor positioning technology and other services has been rapidly increasing. Common indoor wireless positioning technologies include: Wireless-Fidelity (Wi-Fi), Bluetooth, Infrared (IR), Ultra Wide Band (UWB), Radio Frequency Identification (RFID), ZigBee, and Ultrasound. However, these positioning technologies share a common drawback: they require the deployment of expensive specialized equipment (e.g., Wi-Fi network access point coverage, infrared transmitters, ultrasonic transmitters, etc.), and this specialized equipment is often only used for positioning and has few other applications. This shortcoming greatly limits the development and widespread application of wireless positioning technology.

[0003] Fortunately, the rapid development and increasing popularity of smart lighting devices such as light-emitting diodes (LEDs) have attracted a large number of scholars to conduct research on visible light-based indoor positioning technology, creating a new wave in the field of communications. Emerging LED-based indoor visible light positioning technology has advantages such as low deployment cost, flexible positioning algorithms, and high positioning accuracy, and is expected to become a powerful alternative to indoor wireless positioning technology in the future.

[0004] However, most current research on visible light-based indoor positioning technology focuses on improving indicators such as positioning accuracy, positioning robustness, and positioning real-time performance, with few studies aimed at improving system flexibility.

[0005] Currently, indoor visible light positioning technology based on smart terminals suffers from two main drawbacks in terms of flexibility: 1) Insufficient flexibility in positioning algorithms. The positioning accuracy of smart terminals drops sharply when tilted, requiring specific handholding postures to achieve high-precision positioning; 2) Insufficient system flexibility and scalability. Currently, the computation and processing of indoor visible light positioning systems based on smartphones are concentrated on the mobile terminal, demanding high computing power and resulting in poor system flexibility and scalability. Therefore, designing positioning algorithms and functional module segmentation methods to improve the flexibility of visible light positioning systems is of significant research importance. Summary of the Invention

[0006] This invention aims to solve the problems of the prior art. It proposes a positioning method to improve the flexibility of visible light positioning systems. The technical solution of this invention is as follows:

[0007] A positioning method to improve the flexibility of visible light positioning systems includes the following steps:

[0008] Step 1-1: After capturing the image frame, a lightweight image processing algorithm based on equidistant sampling is used to extract all the regions of interest of the LEDs and obtain the centroid pixel coordinates of all LEDs in the tilted image plane coordinate system;

[0009] Steps 1-2: Decode the images of all extracted LED regions of interest to obtain several LED beacon sequences;

[0010] Steps 1-3: Match the obtained LED beacon sequences with all the pre-set LED beacon sequences in the database to identify specific LEDs and obtain their corresponding world coordinates;

[0011] Steps 1-4: Convert the pixel coordinates of several LEDs obtained in the tilted image plane coordinate system into pixel coordinates in the parallel image plane coordinate system, and combine computer vision and geometric principles to obtain the positioning results of the smart terminal.

[0012] Furthermore, it also includes deploying a cloud computing platform, which divides the image processing, image decoding, LED recognition and positioning computing functions involved in the above steps into parts, so that some functions are deployed on the cloud computing platform and others are deployed on smart terminals.

[0013] Furthermore, the image decoding specifically includes: taking the center line of the LED region of interest, calculating the grayscale value of all points on the center line, and averaging the grayscale value of each point by taking the 10 pixels above and below the center line perpendicular to that point; determining whether the point belongs to a bright stripe or a dark stripe based on the calculated grayscale value, and sorting the x-coordinates of the starting points of all bright and dark stripes from smallest to largest, thus obtaining the x-coordinate value of the starting point of each stripe; traversing the width of each stripe to obtain the bright and dark stripe sequence of the region of interest; and obtaining the bright and dark stripe sequence, i.e., the beacon sequence of the LED, according to the known data packet format.

[0014] Furthermore, steps 1-4 specifically include: point P is the location point to be determined, with coordinates (X, Y) in the world coordinate system; A and B are two LEDs on the ceiling, with coordinates (X, Y) in the world coordinate system respectively. A ,Y A ) and (X B ,Y B The distance between the two is S1. Points P, A, and B correspond to points M, C, and D in the image parallel to the image plane, respectively. The pixel coordinates of the three points are (X, Y, Z, 1, 2, 3, 4, 5, 6, 7, 8, 9, 1, 1, 1, 1, 1, 1, 2 ... mid ,Y mid ), (X a ,Y a ) and (X b ,Y b ), where the pixel distance between points C and D is S2, then we have

[0015]

[0016] The coordinates of P can be calculated using the above geometric relationships, thus enabling positioning under parallel image plane.

[0017] When the smart terminal rotates only on the horizontal plane, a coordinate system is established with the captured LED image as the plane, called the rotated image coordinate system, with the coordinate plane being x. r oy r The plane coordinate system is equivalent to a coordinate system parallel to the image plane, rotated by an angle θ around the image center point. The pixel coordinates of the LED in the rotated image coordinate system are equivalently transformed into pixel coordinates in the parallel image plane coordinate system. The rotation formula is:

[0018]

[0019] Among them, (x r ′,y r (x′, y′) represents the pixel coordinates of the two LEDs in the rotated image coordinate system, and (x′, y′) represents the pixel coordinates of the two LEDs in the parallel image plane coordinate system; the extension of the line connecting the two LEDs in the image intersects the rotated image coordinates x′, y′, and y′. rThe following relationship exists between the angle β between the axes and the angles α and θ between the extension of the line connecting the two LEDs and the x-axis of the world coordinate system plane:

[0020]

[0021] Since both β and α can be determined based on known information, the final result is obtained through (X). A -X) / (X a -X mid ) and (X B -X) / (X b -X mid The consistency of the values ​​determines which rotation method will ultimately determine θ.

[0022] When the smart terminal is in a three-dimensional tilted state, point M is the center point of the convex lens of the smartphone camera, and P is the parallel image plane. t The image plane formed when the smartphone is tilted at an angle γ is called the tilted image plane. In plane P, point a represents the LED. t Mapped to point a, f is the focal length, and point O is the center point parallel to the image plane. t Let C be the center point of the tilted image plane, hc be the isoangular point, and r be the isotropic line. tc and r c It is from point C to point a t The length of the diameter formed by connecting point a and point a. and It is a horizontal angle. According to photography theory, we can obtain:

[0023]

[0024] Establish a polar coordinate system with the isogonal point C as the pole and the isogeographic line hc as the polar axis. Since... radius r tc and r c If they are collinear, then the difference r tc -r c This is the image point displacement value, δ. a The approximate calculation formula is:

[0025]

[0026] The unique unknown γ in the above displacement formula is obtained through the gravity acceleration sensor of the smart terminal.

[0027] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned positioning method for improving the flexibility of a visible light positioning system.

[0028] This invention also provides a positioning system to improve the flexibility of visible light positioning systems, comprising a smart terminal and a cloud computing platform. The system divides the functions into a lightweight image processing algorithm based on equidistant sampling, image decoding, LED recognition, and positioning calculation functions, allowing some functions to be deployed on the cloud computing platform and others on the smart terminal. Capturing continuous image frames is performed solely by the smart terminal. The functional module segmentation employs a performance-optimized scheme that balances data traffic consumption, CPU usage, and memory consumption on the smart terminal. The lightweight image processing algorithm extracts all LED regions of interest and obtains the centroid pixel coordinates of all LEDs in an inclined image plane coordinate system. The image decoding obtains several LED beacon sequences.

[0029] The advantages and beneficial effects of this invention are as follows:

[0030] 1. This invention designs a positioning algorithm to improve the flexibility of visible light positioning systems. Traditional indoor visible light positioning algorithms based on smart terminals require the smart terminal to be as parallel as possible to the world coordinate system plane to achieve high positioning accuracy. Due to the habit of using handheld smart terminals, it is almost impossible to require client users to operate in this way at all times. Therefore, this invention designs a high-precision indoor visible light positioning algorithm for smart terminals in a three-dimensional tilted state, allowing the smartphone plane to be at an angle of ±60° to the ceiling plane, and still achieving centimeter-level positioning results, filling a research gap in this area and improving the system's flexibility.

[0031] 2. This invention designs a functional module segmentation method to improve the flexibility of visible light positioning systems. Traditional indoor visible light positioning systems based on smart terminals centralize the entire process from capturing a complete LED to obtaining the positioning result within the smart terminal, placing high demands on the smart terminal's computing power and hindering system scalability. Therefore, this invention designs a functional module segmentation method to improve the flexibility of visible light positioning systems, providing users with both online and offline positioning service modes. For the offline mode, this invention develops an application on the Android system, implementing the entire process from capturing a complete LED to location calculation to obtain the client user's positioning result. For the online mode, the image processing module, image decoding module, and positioning calculation module are functionally segmented, allowing the algorithms for some modules to be implemented on the smart terminal, while the algorithms for other modules are implemented on a cloud computing platform (i.e., collaborative work between online and offline modes). This invention only needs to find a reasonable functional module segmentation scheme. This functional module segmentation method maximizes the smoothness and response speed of smart terminal apps by finding a functional segmentation scheme that achieves the optimal balance between smartphone data traffic consumption (i.e., bandwidth resources) and CPU and memory usage (i.e., computing resources), thereby enhancing the flexibility of indoor visible light positioning systems. Attached Figure Description

[0032] Figure 1 The system architecture diagram provided for the preferred embodiment of the present invention for verifying the performance of a positioning method that improves the flexibility of visible light positioning systems is shown below.

[0033] Figure 2 The schematic diagram shows the lightweight image processing algorithm and image decoding algorithm based on equidistant sampling designed for this invention.

[0034] Figure 3 This is a schematic diagram illustrating the principle of positioning achieved by the present invention using computer vision and geometric principles;

[0035] Figure 4 The positioning algorithm designed for this invention to improve the flexibility of the visible light positioning system (i.e., the principle diagram for positioning a smart terminal in a three-dimensional tilted state);

[0036] Figure 5 The diagram illustrates the functional module segmentation method for improving the flexibility of the visible light positioning system designed for this invention (different segmentation schemes);

[0037] Figure 6 This is a statistical chart of the image decoding accuracy measured in the system built according to the present invention;

[0038] Figure 7This is a statistical graph showing the relationship between the tilt angle of the smartphone and the positioning error, as measured in the system built in this invention.

[0039] Figure 8 This is a statistical chart showing the relationship between data traffic, memory consumption, and CPU utilization under different partitioning schemes, as measured in the system built by this invention.

[0040] Figure 9 This is a statistical chart showing the changes in rc and ro under different segmentation schemes, as measured in the system built by this invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings. The described embodiments are merely some embodiments of the present invention.

[0042] 1. A positioning algorithm to improve the flexibility of visible light positioning systems, comprising the following steps:

[0043] Step 1-1: After capturing image frames, a lightweight image processing algorithm based on equidistant sampling is used to extract the regions of interest (ROIs) of all LEDs and obtain the centroid pixel coordinates of all LEDs in the current coordinate system (the current coordinate system is called the tilted image plane coordinate system). The lightweight image processing algorithm based on equidistant sampling includes: blurring algorithm, grayscale thresholding algorithm, edge detection algorithm, contour extraction algorithm, and centroid pixel coordinate extraction algorithm. It is worth noting that due to the user's hand-holding habits, the tilted image plane is highly likely to have an angle of inclination with the world coordinate system (in this invention, the world coordinate system is established using the ceiling as a plane) and the parallel image plane (when the smartphone is completely parallel to the ceiling, this invention uses the image plane captured by the smartphone to establish a coordinate system, which is called the parallel image plane coordinate system).

[0044] Step 1-2: Decode the regions of interest (ROIs) of all extracted LEDs to obtain several LED beacon sequences (the number of beacon sequences is determined by the number of extracted ROIs of LEDs).

[0045] Steps 1-3: Match the obtained LED beacon sequences with all the pre-set LED beacon sequences in the database to identify specific LEDs and obtain their corresponding world coordinates.

[0046] Steps 1-4: Since the smartphone captures images while in a three-dimensional tilted state, the current image plane (i.e., the tilted image plane) must have an angle of inclination with the world coordinate system plane and the parallel image plane. However, existing imaging-based indoor visible light positioning algorithms only address the case where the smartphone is completely parallel to the world coordinate system plane (i.e., the smart terminal plane is completely parallel to the ceiling plane). To solve the problem of poor positioning performance of smartphones in a three-dimensional tilted state, fill the research gap in this area, and improve the system's flexibility, firstly, it is necessary to convert the centroid pixel coordinates of several LEDs obtained in the tilted image plane coordinate system into pixel coordinates in the parallel image plane coordinate system; then, the position of the smart terminal is calculated by combining computer vision and geometric principles to achieve high-precision positioning.

[0047] This positioning algorithm allows users to maintain the habit of holding a handheld smart device without requiring the smart device to be perfectly parallel to the ceiling. It can still achieve centimeter-level high-precision positioning even when the smart device is at an angle of ±60° to the ceiling.

[0048] 2. The functional module segmentation method to improve the flexibility of the visible light positioning system includes the following: the indoor visible light positioning system uses LEDs as light sources, an Android smartphone as a receiver, and a cloud computing platform is deployed in a Linux environment (the smartphone and the cloud computing platform communicate bidirectionally through a Socket communication module), providing both online and offline positioning services.

[0049] For the offline mode: This invention developed an application on the Android system, which implemented a complete process from capturing the complete LED to location calculation to obtain the location result of the client user; For the online mode: The image is compressed and sent to the cloud computing platform, and some functions in the offline mode are divided and processed by the cloud computing platform to form an online visible light positioning system for the cloud computing platform.

[0050] For the online visible light positioning system, the system first divides its three main functional modules: image processing, image decoding, and LED recognition and positioning calculation. The image processing module's execution flow is further subdivided, including sub-modules such as blur processing, grayscale thresholding, edge detection, contour extraction, and centroid pixel coordinate extraction. Then, at the code level, the algorithms for some modules are implemented on a smartphone, while the algorithms for the remaining modules are implemented on a cloud computing platform.

[0051] This functional module segmentation method maximizes the smoothness and response speed of smart terminal apps by finding a functional segmentation scheme that achieves the optimal balance between smartphone data traffic consumption (i.e., bandwidth resources) and CPU and memory usage (i.e., computing resources), thereby enhancing the flexibility of indoor visible light positioning systems.

[0052] 3. The image decoding algorithm described above: First, a lightweight image processing algorithm based on equidistant sampling is used to obtain the regions of interest (ROIs) of all complete LEDs in the image frame; then, the center line of the ROI of the LED is taken, and the gray values ​​of all points on the center line are calculated. The gray value of each point is calculated by averaging the gray values ​​of the 10 pixels above and below the center line perpendicular to the point; based on the calculated gray values, it is determined whether the point belongs to a bright stripe or a dark stripe, and the x-coordinates of the starting points of all bright and dark stripes are sorted from smallest to largest, thus obtaining the x-coordinate value of the starting point of each stripe; the width of each stripe is traversed to obtain the bright and dark stripe sequences of the ROI; based on the known data packet format, the useful bright and dark stripe sequences, i.e., the LED beacon sequences, are obtained.

[0053] This image decoding algorithm ensures the accurate extraction of the beacon sequence of all LEDs in each image frame, providing a strong guarantee for the smooth execution of the positioning algorithm and improving the overall stability of the system.

[0054] Figure 1 This diagram illustrates the system architecture deployed to verify the performance of the positioning algorithm and functional module segmentation method for improving the flexibility of the visible light positioning system, as provided by a preferred embodiment of the present invention. The indoor visible light positioning system built by this invention uses LEDs as the light source, an Android smartphone as the receiver, and a cloud computing platform deployed in a Linux environment (the smartphone and the cloud computing platform communicate bidirectionally via a Socket communication module). It provides both online and offline positioning services, allowing users to choose the preferred method.

[0055] For offline mode: This invention develops an application on the Android system, and the specific location process is as follows:

[0056] Step 1-1: Open the Android app and capture an image containing the LEDs;

[0057] Steps 1-2: Perform image processing on the captured LED images to obtain several regions of interest for the LEDs;

[0058] Steps 1-3: Decode the obtained LED regions of interest to obtain beacon sequences for several LEDs;

[0059] Steps 1-4: Place the obtained beacon sequences of several LEDs into the numerical database and match them with the beacon sequences of all LEDs to identify specific LEDs and obtain their world coordinates;

[0060] Steps 1-5: Run the indoor visible light positioning algorithm designed in this invention to calculate the user's positioning result.

[0061] For the online mode: images are compressed and sent to a cloud computing platform. Some functions from the offline mode are processed by the cloud computing platform to form an online visible light positioning system for the cloud computing platform. For the online visible light positioning system, firstly, the three main functional modules—image processing, image decoding, and LED recognition and positioning calculation—are functionally segmented. The image processing module's execution flow is further segmented, including sub-modules such as blur processing, grayscale thresholding, edge detection, contour extraction, and centroid pixel coordinate extraction. Then, the code-level segmentation of each module is performed, with the algorithm implementation for some modules completed on the smartphone, and the algorithm implementation for the remaining modules completed on the cloud computing platform. Finally, the processing results from the smart terminal and the cloud computing platform are merged to calculate the user's positioning result, which is then returned to the smart terminal for display.

[0062] Figure 2 The schematic diagram shows the lightweight image processing algorithm and image decoding algorithm based on equidistant sampling designed for this invention.

[0063] Image Processing: First, the present invention converts the captured image into a grayscale image and performs binarization on the grayscale image. Then, the present invention performs equidistant pixel intensity sampling on the binarized image according to the rows and columns of pixels to obtain the approximate region of interest (ROI) of the LED. Finally, the present invention performs pixel intensity sampling on all pixels near this region according to the rows and columns to obtain the accurate boundary of the LED ROI.

[0064] Image Decoding: For a specific region of interest (ROI) of an LED, firstly, the centerline of the ROI is selected, and the grayscale values ​​of all points along this centerline are calculated. The grayscale value of each point is averaged over the 10 pixels perpendicular to the centerline. Based on the calculated grayscale value, it is determined whether the point belongs to a bright or dark stripe. The starting coordinates of all bright and dark stripes are then sorted from smallest to largest, yielding the starting coordinate value of each stripe. Further, the width of each stripe is traversed to obtain the bright and dark stripe sequences within the ROI. Based on the known data packet format, the useful bright and dark stripe sequences, i.e., the LED beacon sequences, are obtained.

[0065] Figure 3 This is a schematic diagram illustrating the principle of positioning achieved by the present invention using computer vision and geometric principles.

[0066] Figure 3 (a) Indoor visible light positioning scenario where the smartphone plane is completely parallel to the ceiling plane (the parallel image plane coordinate system is parallel to the world coordinate system plane, and the axes of the two coordinate systems point in the same direction). Figure 3 As shown in (a), the system used to verify the performance of the present invention uses several LEDs arranged on the ceiling as transmitters and a smartphone as receivers. In this case, a world coordinate system is established with the ceiling as the plane, and the origin of the coordinate system is the northwest corner of the ceiling; a parallel image plane coordinate system is established with the LED image captured by the smartphone as the plane, and the origin of the coordinate system is the upper left corner of the image. Figure 3 As shown in (a), point P is the location point to be determined, and its coordinates in the world coordinate system are (X,Y). A and B are two LEDs on the ceiling, and their coordinates in the world coordinate system are (X,Y). A ,Y A ) and (X B ,Y B The distance between the two is S1. According to computer vision and projection principles, the points P, A, and B to be determined correspond to points M, C, and D in the image parallel to the image plane, respectively, and their pixel coordinates are (X, B, C, D) and (X, D). mid ,Y mid ), (X a ,Y a ) and (X b ,Y b ), where the pixel distance between points C and D is S2. The present invention will be as follows: Figure 3 The positioning algorithm shown in (a) is called the positioning algorithm under parallel image plane. Based on geometric principles, the following geometric relationships can be obtained:

[0067]

[0068] The coordinates of P can be calculated using the above geometric relationships, thus enabling positioning under parallel image plane.

[0069] Figure 3 (b) Indoor visible light positioning scenario where the smartphone plane is completely parallel to the ceiling plane (compared to) Figure 3 In case (a), although the plane of the parallel image plane is parallel to the plane of the world plane, there is a certain angle between the two coordinate planes. Figure 3 As shown in (b), when the phone rotates only on the horizontal plane, the preview image captured by the phone's camera also rotates with the phone. In this case, a coordinate system is also established with the captured LED image as the plane (this coordinate system is called the rotated image coordinate system, and the coordinate plane is x).r oy r (Plane), this coordinate system is equivalent to a coordinate system parallel to the image plane rotated by an angle θ around the image center point. The pixel coordinates of the LED obtained from the image are in the rotated image coordinate system. Figure 3 Given that the scenario in (a) has been demonstrated, it is only necessary to equivalently transform the pixel coordinates of the LED in the rotated image coordinate system into the pixel coordinates in the parallel image plane coordinate system before it can be used. Figure 3 The derivation formula in (a) is used for calculation. According to the principle of coordinate system rotation, the rotation formula is:

[0070]

[0071] Among them, (x r ′,y r Let (x', y') be the pixel coordinates of the two LEDs in the rotated image coordinate system, which can be directly measured from the image. Let (x', y') be the pixel coordinates of the two LEDs in the parallel image plane coordinate system, which is the target coordinate. Only the rotation angle θ needs to be determined. Regardless of the rotation, the angle α between the extension of the line connecting the two LEDs in the image and the x-axis of the world coordinate system plane is uniquely determined. Therefore, the angle α between the extension of the line connecting the two LEDs in the image and the rotated image coordinate system x' is uniquely determined. r The angle β between the axes and angles α and θ have the following relationship:

[0072]

[0073] Since both β and α can be determined based on known information, the final result is obtained through (X). A -X) / (X a -X mid ) and (X B -X) / (X b -X mid The discovery of the consistency of the values ​​of θ determines which rotation method is used to ultimately determine θ.

[0074] Figure 4 This is a schematic diagram of the positioning algorithm designed to improve the flexibility of the visible light positioning system for this invention (i.e., for the scenario where the smart terminal is in a three-dimensional tilted state for positioning). For example... Figure 4 As shown, in this case, the objective of the present invention is to convert the pixel coordinates of the two LEDs and the pixel coordinates of the image center point obtained under the tilted image plane coordinate system into pixel coordinates under the rotated image coordinate system (parallel image plane coordinate system), so that it can be used... Figure 3 The formulas for cases (a) and (b) are used to calculate the positioning results.

[0075] like Figure 4 As shown, point M is the center point of the convex lens of the smartphone camera, and P is the parallel image plane.t Let P be the image plane formed when the smartphone is tilted at an angle γ (called the tilted image plane). In plane P, point a represents the LED. t Mapped to point a. f is the focal length, the distance between the convex lens of the smartphone camera and the image plane, which is a known quantity. Point O is the center point parallel to the image plane. t Let C be the center point of the tilted image plane. Point C is a point of equal angle, and hc is a line of equal ratio. tc and r c It is from point C to point a t The diameter formed by connecting point a and point a. and It's a horizontal angle. According to photography theory, we can deduce:

[0076]

[0077] Furthermore, this invention establishes a polar coordinate system with the isoangular point C as the pole and the isotropic line hc as the polar axis. Because radius r tc and r c If they are collinear, then the difference r tc -r c This is the image point displacement value, δ. a The approximate calculation formula is:

[0078]

[0079] At this point, by simply using the smartphone's gravity-based accelerometer to obtain the unique unknown γ in the above displacement formula, the pixel coordinates of the two LEDs and the pixel coordinates of the image center point obtained under the tilted image plane coordinate system can be converted into pixel coordinates under the rotated image coordinate system (parallel image plane coordinate system). This allows for the utilization of... Figure 3 The formulas for cases (a) and (b) are used to calculate the positioning results.

[0080] Figure 5 The diagram illustrates the functional module segmentation method for improving the flexibility of the visible light positioning system designed for this invention (different segmentation schemes). For example... Figure 5As shown, this invention is functionally divided into three main aspects: image processing module, beacon decoding module (6), and positioning calculation module (7). The image processing module is further divided according to the specific steps of image processing. In the indoor visible light positioning system used to verify the performance of this invention, the main image processing sub-modules are: blurring (1), grayscale thresholding (2), edge detection (3), contour extraction (4), and centroid pixel coordinate calculation (5). This paper divides the aforementioned modules at the code level, so that the algorithm implementations of some of the seven functional modules are completed on the smart terminal, while the algorithm implementations of the remaining modules are completed on the cloud computing platform.

[0081] Figure 6 This is a statistical chart showing the image decoding accuracy measured in the system built according to the present invention. In this test, the brightness of the LED was kept constant, while the distance between the mobile phone and the ceiling was varied from 0.5m to 2.5m. Data was collected at different locations indoors to test the image decoding accuracy measured in the system built according to the present invention. Figure 6 As shown, the decoding accuracy decreases when the distance reaches 2.5m. This is because the proportion of LEDs in the captured image decreases as the distance between the phone and the ceiling increases, leading to a reduction in image decoding accuracy. Typically, the distance between the phone and the ceiling is between 1.3m and 1.7m. Figure 6 As can be seen, the image decoding accuracy is nearly 100% in this case, and the reliability is good.

[0082] Figure 7 This is a statistical graph showing the relationship between the tilt angle of the smartphone and the positioning error, measured in the system built in this invention. For smartphones in a three-dimensional tilt state, this invention tested the positioning accuracy of the smartphone at different tilt angles. During the test, the smartphone was kept at a random three-dimensional tilt angle (ranging from ±60°), and it was ensured that two or more LEDs were still present in the captured image when the smartphone was tilted. The statistical results of testing the impact of the tilt angle on positioning accuracy are as follows: Figure 7 As shown in the image, it can be observed that the positioning error increases with the increase of the tilt angle. This is because when the tilt angle increases, the bright and dark stripes in the captured image become blurred, further causing image point displacement and ultimately leading to an increase in positioning error. In addition, the thickness of the lampshade and the height of the phone's position also cause some image point displacement, resulting in increased positioning error. Even so, the indoor visible light positioning algorithm designed in this invention for smartphones in a three-dimensional tilted state can still achieve centimeter-level positioning accuracy, filling a research gap in this area.

[0083] Figure 8 This is a statistical chart showing the relationship between data traffic, memory consumption, and CPU utilization under different partitioning schemes, measured in the system built by this invention. The partitioning schemes are shown below. Figure 5 Introduction. For example... Figure 8 As shown in (a) and (b), for Scheme 1, the smart terminal is in a completely offline positioning state, and all computational processing is completed on the smart terminal itself. In this case, its data traffic consumption is the lowest, while its memory usage and CPU usage are the highest; that is, the smart terminal has the lowest bandwidth resource consumption and the highest computational resource consumption. For Scheme 8, the smart terminal is in a completely online positioning state, and the smart terminal only performs image capture; all computational processing is completed on the cloud computing platform. In this case, the smart terminal has the highest data traffic consumption, while its memory usage and CPU usage are the lowest; that is, the smart terminal has the highest bandwidth resource consumption and the lowest computational resource consumption.

[0084] Figure 9 This is a statistical chart showing the changes in rc and ro under different segmentation schemes, measured in the system built according to this invention. This invention establishes a mathematical model for the segmentation schemes, specifically as follows: In the baseline scheme, the memory consumption of the smart terminal is taken as m1, the CPU utilization rate as c1, and the data traffic consumption as f1. In scheme k, the memory consumption of the smart terminal is m... k CPU utilization is c k Data traffic consumption is f k Then we have:

[0085]

[0086] Where rc represents relative memory consumption and ro represents relative CPU usage. In the system built in this invention, the larger the values ​​of rc and ro, the better the overall system performance, indicating that the smart terminal can obtain more computing resources with less bandwidth compared to Scheme 1. This invention aims to find an optimal partitioning scheme other than Scheme 1 and Scheme 8, allowing the smart terminal to save the most memory consumption and the most CPU usage with the least bandwidth. In other words, it aims to obtain the most computing resources with the least bandwidth. This invention uses Scheme 1 as the benchmark and compares Schemes 2 to 8 with it. In this test, the values ​​of parameters rc and ro were statistically analyzed for each scheme, such as... Figure 9 As shown in (a)(b). From Figure 9As can be seen in (a) and (b), the curve reaches its maximum value at Scheme 3, which becomes the golden functional segmentation point in the system functional module segmentation method for improving system flexibility designed in this invention. That is, compared to Scheme 1, under Scheme 3, the smartphone can obtain the most computing resources with the least bandwidth. This demonstrates that the system functional module segmentation method for improving system flexibility designed in this invention has extraordinary significance and value.

[0087] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0088] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0089] The above embodiments should be understood as illustrative only and not as limiting the scope of protection of the present invention. After reading the description of the present invention, those skilled in the art can make various alterations or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A positioning method for improving the flexibility of a visible light positioning system, characterized in that, Includes the following steps: Step 1-1: After capturing the image frame, a lightweight image processing algorithm based on equidistant sampling is used to extract all the regions of interest of the LEDs and obtain the centroid pixel coordinates of all LEDs in the tilted image plane coordinate system; Steps 1-2: Decode the images of all extracted LED regions of interest to obtain several LED beacon sequences; Steps 1-3: Match the obtained LED beacon sequences with all the pre-set LED beacon sequences in the database to identify specific LEDs and obtain their corresponding world coordinates; Steps 1-4: Convert the pixel coordinates of several LEDs obtained in the tilted image plane coordinate system into pixel coordinates in the parallel image plane coordinate system, and combine computer vision and geometric principles to obtain the positioning results of the smart terminal. A cloud computing platform is deployed, and the image processing, image decoding, LED recognition, and positioning calculation functions involved in the above steps are divided into two parts. One part of the functions are deployed on the cloud computing platform, and the other part is deployed on the smart terminal. This division employs a function partitioning scheme that optimizes the data traffic consumption, CPU usage, and memory consumption of the smart terminal. The mathematical model for this partitioning scheme is as follows: Taking the memory consumption of the smart terminal in the baseline scheme as... CPU utilization rate Data traffic consumption is ,plan The memory consumption of smart terminals is CPU utilization rate Data traffic consumption is Then we have: in, For relative memory consumption, In terms of relative CPU usage overhead, the system built and The larger the value, the more computing resources the smart terminal can obtain with less bandwidth compared to the benchmark scheme, and the better the overall performance of the system. The benchmark scheme is that all computing processing is completed on the smart terminal.

2. The positioning method for improving the flexibility of a visible light positioning system according to claim 1, characterized in that: The lightweight image processing algorithm based on equidistant sampling described in step 1-1 includes: blurring, grayscale thresholding, edge detection, contour extraction, and centroid pixel coordinate extraction.

3. The positioning method for improving the flexibility of a visible light positioning system according to claim 1, characterized in that: The image decoding specifically includes: taking the center line of the LED region of interest; calculating the gray values ​​of all points on the center line; averaging the gray values ​​of each point with the 10 pixels above and below the center line perpendicular to that point; determining whether the point belongs to a bright or dark stripe based on the calculated gray values; sorting the x-coordinates of the starting points of all bright and dark stripes from smallest to largest to obtain the x-coordinate value of the starting point of each stripe; traversing the width of each stripe to obtain the bright and dark stripe sequence of the region of interest; and obtaining the bright and dark stripe sequence, i.e., the beacon sequence of the LED, according to the known data packet format.

4. The positioning method for improving the flexibility of a visible light positioning system according to claim 1, characterized in that: The coordinate transformation in steps 1-4 includes: point Let be the location point to be determined, and its coordinates in the world coordinate system be... , and For the two LEDs on the ceiling, their coordinates in the world coordinate system are respectively and The distance between the two is Points to be sought ,lamp and lights Corresponding to the midpoints of the image in the parallel image plane ,point and points The pixel coordinates of the three points are respectively , and The pixel distance between points C and D is Then there is The solution can be obtained using the above geometric relationships. With the coordinates, positioning can be completed under parallel image plane.

5. The positioning method for improving the flexibility of a visible light positioning system according to claim 4, characterized in that: When the smart terminal rotates only on the horizontal plane, a coordinate system is established with the captured LED image as the plane, called the rotated image coordinate system, and the coordinate plane is... The plane, this coordinate system is equivalent to a coordinate system parallel to the image plane rotating around the image center point. The angle is used to equivalently transform the pixel coordinates of the LED in the rotated image coordinate system into the pixel coordinates in the parallel image plane coordinate system. The rotation formula is: in, Let these be the pixel coordinates of the two LEDs in the rotated image coordinate system. The pixel coordinates of the two LEDs in the parallel image plane coordinate system; the extension of the line connecting the two LEDs in the image and the coordinates of the rotated image. Angle between axes The angle between the extension of the line connecting the two LEDs and the x-axis of the world coordinate system plane. , The following relationships exist between angles: because and All can be determined based on known information, and finally through... and The consistency of values ​​determines which rotation method will ultimately result in... It's settled.

6. The positioning method for improving the flexibility of a visible light positioning system according to claim 4, characterized in that: When the smart terminal is in a three-dimensional tilt state, the point It is the center point of the convex lens of a smartphone camera. Parallel image plane, For when smartphone is tilted The image plane formed at an angle is called the tilted image plane. In the middle, the dots represent LEDs. Mapping to point , For focal length, point Point is the center point parallel to the image plane. The center point of the tilted image plane, point It is an isoangular point. It is a geometric line. and It is a point Time and points The diameter formed by connecting the two sides, and It is a horizontal angle. According to photography theory, we can obtain: With isogonal points For poles and isolines To establish a polar coordinate system for the polar axis, since ,radius and If they are collinear, then the difference is... This is the image point displacement value. The approximate calculation formula is: The only unknown in the above displacement formula is obtained through the gravity acceleration sensor of the smart terminal. .

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that, when executed by a processor, implements the positioning method for improving the flexibility of a visible light positioning system as described in any one of claims 1-6.

8. A positioning system for improving the flexibility of a visible light positioning system constructed using the method described in any one of claims 1-6, characterized in that: Including smart terminals and cloud computing platforms, the lightweight image processing algorithm based on equidistant sampling, image decoding, LED recognition and positioning calculation functions are divided, so that some functions are deployed on the cloud computing platform and other functions are deployed on the smart terminal, and the capture of continuous image frames is only performed by the smart terminal. The functional module segmentation adopts a functional segmentation scheme that achieves the best performance balance between data traffic consumption, CPU usage, and memory consumption of the smart terminal; the lightweight image processing algorithm extracts all LED regions of interest and obtains the centroid pixel coordinates of all LEDs in the tilted image plane coordinate system; the image decoding obtains several LED beacon sequences.

Images