A visible light indoor positioning system and positioning method based on mobile phone camera

Abstract

A visible light indoor positioning system and positioning method based on a mobile phone camera, the system includes a transmitting subsystem and a receiving subsystem, the transmitting subsystem includes a microcontroller, a serial port module, a cache circuit, an LED drive circuit and an LED light source connected in sequence, and the receiving subsystem includes a mobile phone with a camera and an indoor positioning module. First, the position coordinate information to be transmitted is encoded by the microcontroller, and the encoded data stream is input into the input port of the LED drive circuit after being cached, thereby controlling the high-speed flashing of the LED light source. Then, the light emitted by the LED light source is photographed by the mobile phone to obtain a stripe image with light and dark changes. After the image is clustered, curve fitting is used to generate a dynamic threshold to determine whether the information represented by the current brightness is "0" or "1", and the coordinate information is obtained by decoding. The present invention has high reliability and strong anti-interference ability.

Description

Technical Field

[0001] The present invention belongs to the field of optical communications and relates to a visible light indoor positioning system and a positioning method based on a mobile phone camera. Background Art

[0002] Currently, the Global Positioning System (GPS) is the most widely used positioning system. However, GPS signals can be blocked by building roofs and walls, and due to severe indoor multipath, they can be very weak or even disappear indoors. This significantly reduces GPS positioning accuracy, often reaching errors of tens of meters, making it difficult to meet users' indoor positioning needs. In modern society, people spend most of their time indoors, in large indoor spaces such as shopping malls, underground parking lots, libraries, and museums. Therefore, a more effective and accurate indoor positioning system is increasingly needed.

[0003] In recent years, new semiconductor lighting technologies, exemplified by light-emitting diodes (LEDs), often referred to as "green lighting," have experienced rapid development. The advantages of white LEDs include low power consumption, long lifespan, compact size, and environmental friendliness. Compared to traditional lighting sources, LEDs offer superior modulation performance and increased responsiveness. Visible light communication (VLC) leverages these characteristics to meet lighting requirements while also modulating signals onto the LEDs for data transmission. VLC has also been developed to transmit identification information for positioning purposes, utilizing variations in visible light (color, intensity, or position) to transmit identification information. This communication technology for transmitting identification information relies on high-frequency flickering light signals. Specifically, the identification information to be transmitted is first encoded into a digital signal; this digital signal is then used to modulate the duration or frequency of the light source's driving current or voltage, causing the light source to flicker at a high frequency. This high-frequency flicker signal can be detected by a photosensitive device, such as an image sensor. Image sensors can use a "rolling shutter" mechanism to expose different portions of the sensor, or each row or column, at different times. This characteristic allows image sensors with a rolling shutter to capture images containing bright or dark fringes when capturing images of light sources with rapidly varying brightness. By measuring the width of the stripes, the frequency of the light source's driving current and voltage can be calculated, thereby recovering the transmitted identification information. Traditionally, visible light communication (VLC) reception utilizes a photodetector (PD) at the receiving end to receive optical signals, convert them into electrical signals, and then process the information. Therefore, VLC receivers must possess both optical signal reception and photoelectric conversion capabilities. Research indicates that mobile phones with cameras currently on the market are a suitable alternative to receivers with PDs. Smartphones have firmly established themselves as the mainstream mobile phone market, with 4G becoming ubiquitous and the 5G era imminent. Smartphones are poised to completely dominate the mobile phone market. Furthermore, smartphones, with their open operating systems, fast speeds, comprehensive functionality, and user-friendly design, facilitate the proper operation of cameras.

[0004] Smartphones on the electronics market today are generally equipped with CMOS cameras, which use a rolling shutter mode for exposure. Unlike the global shutter mode (which exposes the entire frame at once), this mode exposes each row of pixels sequentially. Therefore, when receiving the high-frequency white light signal emitted by an LED lamp (the frequency is much higher than the frame rate of the CMOS image sensor), the CMOS camera will capture a rolling shutter effect pattern consisting of alternating light and dark stripes. In the rolling shutter effect pattern, the grayscale value of the middle column of pixels is selected for data reading, and the row pixels of the light stripes are recorded as the logical byte "1", and the row pixels of the dark stripes are recorded as the logical byte "0". Multiple logical bytes can be transmitted within a frame, enabling short-distance visible light communication. Summary of the Invention

[0005] The purpose of the present invention is to address the problem of inconvenience in implementing visible light communication in the above-mentioned prior art, and to provide a visible light indoor positioning system and positioning method based on a mobile phone camera. The system utilizes the rolling shutter working mode of the mobile phone camera to receive visible light information sent by the LED, thereby realizing indoor positioning. The system has high reliability and strong anti-interference ability.

[0006] In order to achieve the above object, the present invention has the following technical solutions:

[0007] A visible light indoor positioning system based on a mobile phone camera comprises a transmitting subsystem and a receiving subsystem. The transmitting subsystem comprises a microcontroller, a serial port module, a buffer circuit, an LED driving circuit and an LED light source connected in sequence, and the receiving subsystem comprises a mobile phone with a camera and an indoor positioning module. The transmitting subsystem encodes the position information of the LED light source and modulates the light intensity of the light source to send a positioning signal to the indoor space. The receiving subsystem obtains an image of an object under illumination through the mobile phone camera, decodes the image to obtain coordinate values, and obtains accurate position information in combination with map data. The LED light source emits a light positioning image containing the light positioning signal. The camera is a CMOS camera, which utilizes the rolling shutter exposure characteristics of the CMOS camera to image the flickering light signal to generate a stripe image with varying brightness. The stripe image is then processed to detect the flickering frequency of the light source to achieve information acquisition.

[0008] Preferably, in one embodiment of the visible light indoor positioning system based on a mobile phone camera of the present invention, the LED light source adopts a white light LED, the white light LED modulates the signal into light irradiation energy, and the LED driving circuit is a current driving circuit.

[0009] Preferably, in an embodiment of the visible light indoor positioning system based on a mobile phone camera of the present invention, the indoor positioning module in the mobile phone includes a map database import module, an indoor object photo processing module and a positioning result output module; the indoor object photo processing module collects light positioning image information containing a light positioning signal emitted by an LED light source, processes the collected image information, and determines the distance information and position information by searching for identification information in a predefined map database; the specific position information is obtained through the positioning result output module and its ID number is marked on a pre-imported map.

[0010] Preferably, in an embodiment of the visible light indoor positioning system based on a mobile phone camera of the present invention, the map database import module determines the area of the location through an image processing algorithm before searching for identification information, and uses the data of the area to search for identification information in a predefined map database to determine distance information and location information.

[0011] Preferably, in an embodiment of the visible light indoor positioning system based on a mobile phone camera of the present invention, the LED driving circuit is constructed using an NPN transistor, and the LED driving circuit can make the forward voltage drop of the LED light source higher than the input DC voltage drop, and the LED driving circuit can be controlled, and the current and luminous flux of the LED light source maintain a linear relationship.

[0012] Preferably, in an embodiment of the visible light indoor positioning system based on a mobile phone camera of the present invention, the image format captured by the CMOS camera is a YUV format, and the shooting mode is a preview mode;

[0013] At a certain position, one or more images are acquired from a visible light signal emitted by an LED light source at one or more time points, wherein the level of the visible light signal varies between high and low levels at a varying frequency at different time points.

[0014] Preferably, in one embodiment of the visible light indoor positioning system based on a mobile phone camera of the present invention, the identification information of the number of changes in bright fringes or dark fringes captured by the CMOS camera includes data bits, the data bits include data corresponding to frequency changes, and each data bit corresponds to one of the numbers of changes in the number of bright fringes or dark fringes;

[0015] The identification information corresponding to the number of changes in bright stripes or dark stripes also includes one or more start bits and one or more error correction bits. The start bit is used to identify the starting position of a group of data bits, and the error correction bit is used to check errors in the data bits.

[0016] The present invention also provides a visible light indoor positioning method based on a mobile phone camera, comprising the following steps:

[0017] First, the position coordinate information to be transmitted is encoded by the microcontroller and output to the input end of the buffer circuit through the serial port module. The encoded data stream is cached and then input into the input port of the LED driver circuit to control the high-speed flashing of the LED light source. Then, a mobile phone equipped with a camera and an indoor positioning module is used to take a picture of the light emitted by the LED light source to obtain a stripe image with varying brightness. The image is clustered using the k-means algorithm, and curve fitting is then used to generate a dynamic threshold to determine whether the information represented by the current brightness is "0" or "1". Finally, the coordinate information is decoded and obtained.

[0018] Preferably, in an embodiment of the visible light indoor positioning method based on a mobile phone camera of the present invention, the microcontroller converts the position coordinate information to be transmitted into binary information, and then converts it into a code through Manchester encoding. The coding information output by the microcontroller through the serial port module is TTL level, 0V voltage represents binary "0", and 3.3V voltage represents binary "1". The LED light source is used as a carrier to encode this position coordinate information and transmit it to the receiving mobile phone.

[0019] Preferably, in an embodiment of the visible light indoor positioning method based on a mobile phone camera of the present invention, an LED flashing picture is obtained through the camera of the mobile phone, the bright stripes in the LED flashing picture represent binary "1", and the dark stripes represent binary "0", the coding information is identified through image processing, and the coordinate information carried by the LED flashing picture is obtained after decoding; the decoded coordinate information is sent to a database, and the position map of the LED light source at this time is obtained through comparative query.

[0020] Compared to existing technologies, the present invention has the following beneficial effects: a circuit drives an LED light source to carry certain location information, and a mobile phone's CMOS camera serves as a terminal for receiving the location information. The LED light source's location information is converted into binary information, which is then converted into a code using Manchester encoding. The LED light source is controlled to flicker at high speed, and the LED flickering image is used as a carrier to transmit the encoded location information to a receiving subsystem. The receiving subsystem primarily uses a mobile phone equipped with a camera and an indoor positioning module to capture an image of the LED light source, generating a light and dark stripe image. The receiving subsystem then uses image processing technology to parse the location information carried in the LED flicker image. The human eye is generally insensitive to flicker above 60 Hz, so to ensure the quality of LED light, a flicker frequency of 1000 Hz is used to ensure that the LED flicker does not cause discomfort to the human eye. When the portable receiving subsystem approaches the LED lighting area, the mobile phone can be used to parse the LED's coded information. The obtained coded information is then used to determine the corresponding location information in a map database, achieving positioning. The LED driving circuit enhances the clarity of the LED light source's light and dark stripe image, resulting in more accurate image processing results. The present invention can obtain coordinate information by identifying the light reflected by the LED light source on the object, without the need to aim the camera at the LED light source, and is convenient and has strong anti-interference performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of the overall structure of the positioning system of the present invention;

[0022] Figure 2 A structural block diagram of the transmitter terminal system of the present invention;

[0023] Figure 3 A schematic diagram of the circuit structure of the LED driving circuit of the present invention;

[0024] Figure 4 The light and dark stripe image obtained by the receiving terminal system of the present invention;

[0025] Figure 5 Structural block diagram of the receiving terminal system of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0027] The present invention is a visible light indoor positioning system based on a mobile phone camera, comprising a transmitting subsystem and a receiving subsystem, wherein the transmitting subsystem comprises a microcontroller, a serial port module, a cache circuit, an LED driving circuit and an LED light source connected in sequence, and the receiving subsystem comprises a mobile phone with a camera and an indoor positioning APP. The transmitting subsystem encodes the position information of the LED light source and modulates it on the light intensity of the light source to send a positioning signal to the indoor space; the receiving subsystem obtains an image of the object under illumination through the camera of the mobile phone, decodes the image to obtain the coordinate value, and obtains accurate position information in combination with map data. The above-mentioned LED light source emits a light positioning image containing a light positioning signal, and the camera is a CMOS camera of the mobile phone. The rolling shutter exposure characteristics of the CMOS camera are used to image the flickering light signal to generate a stripe image with light and dark changes. The stripe image is then processed to detect the flickering frequency of the light source to achieve information acquisition.

[0028] like Figure 1 As shown in the figure, the microcontroller encodes the position coordinate information to be transmitted and outputs it to the input end of the cache circuit through the serial port module. The encoded data stream is cached and then input into the input port of the LED driver circuit, thereby controlling the high-speed flashing of the LED light source. Then, a mobile phone with a camera and an indoor positioning APP is used to take a picture of the light emitted by the LED light source to obtain a picture of light and dark stripes. The light and dark stripes picture is clustered by the k-means algorithm, and then curve fitting is used to obtain the information carried by the LED light source. After decoding, the positioning result can be obtained and displayed.

[0029] The key to this invention is to use a circuit to drive an LED light source to carry certain location information, and use a mobile phone camera as a terminal to receive the location information carried by the LED light source. The main principle is to convert the location information into binary information, and then convert it into a code through Manchester encoding, control the high-speed flashing of the LED light source, and use the LED light source as a carrier to transmit this location information code to the receiving end. Figure 5The receiving subsystem primarily relies on a mobile phone equipped with a camera and an indoor positioning app to capture the light emitted by the LED light source and generate an image of light and dark stripes. The receiving end then uses image processing techniques to parse the location information carried by the LED light source. The human eye is generally insensitive to flicker above 60Hz, so to ensure the quality of LED lighting, the flicker frequency is set at 1000Hz to ensure that the flicker of the LED light source does not cause discomfort to the human eye. When a person carrying the receiving subsystem approaches the LED lighting area, they can use their mobile phone to parse the LED light source's coded information and use this information to determine the corresponding location information in the map database, achieving positioning.

[0030] The specific steps of the visible light indoor positioning method based on the mobile phone camera of the present invention are as follows:

[0031] The system works in indoor spaces, such as supermarkets and museums;

[0032] LED light sources in indoor spaces carry specific location coordinate information;

[0033] Mobile phones with cameras can normally take pictures of indoor objects and decode the location information carried by LED light sources;

[0034] Open the indoor positioning app of the mobile phone of this applicant, and check whether there is map data of the scene where the shooting is located in the existing map data in the map database import module of the app. If not, you need to import the map information; after completing the establishment of the map data, use the camera function of the indoor object photo processing module of the app, point the mobile phone camera at the object to be photographed or the LED light source, wait for a few seconds, and the mobile phone positioning result output module will display its specific location coordinate information and mark it on the corresponding map. At the same time, you can also export and save the positioning results.

[0035] First, we need to drive the LED light source to illuminate and record its coordinates in the map, so that the LED light source can send the coordinate information at a frequency of 1000Hz. In order to ensure the integrity of the information sent, the coordinate information must be Manchester encoded before sending, and then the encoded information is transmitted to the input section of the LED driver port through the serial port, such as Figure 2 shown.

[0036] The microcontroller of the present invention outputs coded information through the serial port in the form of TTL level (voltage 0V represents binary "0", voltage 3.3V represents binary "1"). In order to ensure that the LED can effectively carry the coded information, the present application uses an NMOS tube to convert the voltage signal of the microcontroller into a current signal to control the high-speed flashing of the LED. At the same time, it can also amplify the driving current of the LED so that the LED brightness meets the requirements. Since the flashing frequency of the LED is to reach 1000Hz, in order to ensure the effectiveness of information transmission, a buffer circuit must be added between the LED driver and the serial port output of the microcontroller. The buffer circuit is a common-collector amplifier circuit with the characteristics of an amplification factor of 1, a large input impedance, and a very small output impedance, which plays an isolation role.

[0037] Specific circuits such as Figure 3 As shown in the figure, the voltage signal encoded by the microcontroller is input to the buffer circuit input terminal Ui through the serial port. After isolation, the buffer circuit output Uo is connected to the input terminal of the LED driver circuit, controlling the high-speed flashing of the LED. Due to the extremely large input impedance of the common-collector amplifier circuit, when the microcontroller serial port input is connected to Ui1, almost all of the voltage signal is applied to Ui1. The common-collector amplifier circuit has an extremely small output impedance. When Uo is input to the LED driver circuit input terminal Ui2, almost all of the voltage signal is applied to the driver circuit input terminal Ui2, thus effectively performing a buffering and isolation function. Thus, the encoded signal output by the microcontroller is buffered and applied to the LED driver circuit. The NMOS transistor in the LED driver circuit converts the voltage signal into a current signal, thereby driving the LED. Due to the high output frequency of the microcontroller, a fixed bias of approximately 0.9V is added to the NMOS gate through resistors R4 and R6 in the circuit to increase the switching speed of the NMOS transistor and improve the brightness of the LED. Capacitor C1 filters the DC signal from the microcontroller, while capacitor C2 filters the DC bias of the buffer circuit, making the LED light source flicker in bright and dark conditions rather than in bright conditions. This allows for more accurate image processing results. Secondly, the mobile phone camera identifies the coordinate information carried by the LED light source. By adjusting the shutter time of the mobile phone camera, an image of the LED flickering can be obtained, such as Figure 3 As shown in the picture, the bright stripes in the LED flashing picture represent binary "1", and the dark stripes represent binary "0". Figure 3 Image processing is performed to identify the coded information carried in the image, and after decoding, the coordinate information carried by the LED can be obtained. The receiving terminal system sends the decoded coordinate information to the database, and through comparison and query, a map of the current LED light source position can be obtained.

[0038] The technical solution of the present invention has outstanding features in the following aspects:

[0039] 1) Encoding method of LED coordinate information.

[0040] 2) By inserting the coded coordinate information into the LED through an LED driver circuit, the human eye cannot perceive the flickering of the LED. This invention uses an NMOS transistor to convert the voltage signal output from the microprocessor serial port into current information to drive the LED flickering. A common-collector circuit acts as a buffer, allowing the light emitted by the LED light source to effectively carry the coded coordinate information, resulting in high reliability. Furthermore, the light and dark stripe images obtained at the final receiving end of the LED light source driven by the NMOS transistor are more distinct than those obtained by driving the LED receiving end using a triode, and the image processing results are more accurate.

[0041] 3) Technology for identifying the coordinate information carried by LEDs using a mobile phone camera. This invention utilizes a CMOS image sensor to receive LED light signals. By adjusting the rolling shutter time, an image of light and dark stripes is obtained. The image is clustered using a k-means algorithm. Curve fitting is then used to generate a dynamic threshold to determine whether the current brightness represents a "0" or "1" signal. Ultimately, the coordinate information is decoded. This method can obtain coordinate information by identifying light reflected from an LED source, eliminating the need to align the camera with the LED source. This method offers excellent convenience and interference resistance.

[0042] The above description is merely a preferred embodiment of the present invention and is not intended to impose any limitation on the technical solution of the present invention. Those skilled in the art should understand that, without departing from the spirit and principles of the present invention, the technical solution can also be subjected to several simple modifications and replacements, and these modifications and replacements also fall within the scope of protection of the claims of the present invention.

Claims

1. A visible light indoor positioning system based on a mobile phone camera, characterized by: The invention comprises a transmitting subsystem and a receiving subsystem, wherein the transmitting subsystem comprises a microcontroller, a serial port module, a buffer circuit, an LED driving circuit and an LED light source connected in sequence, and the receiving subsystem comprises a mobile phone with a camera and an indoor positioning module; the transmitting subsystem encodes the position information of the LED light source and modulates it on the light intensity of the light source to send a positioning signal to the indoor space; the receiving subsystem obtains an image of the object under illumination through the camera of the mobile phone, then decodes the image to obtain the coordinate value, and obtains accurate position information in combination with map data; the LED light source emits a light positioning image containing a light positioning signal, and the camera is a CMOS camera, which utilizes the rolling shutter exposure characteristics of the CMOS camera to image the flickering light signal, and obtains a stripe image with light and dark changes by adjusting the rolling shutter time, wherein the bright stripe represents a binary "1" and the dark stripe represents a binary "0", and then the stripe image is processed to identify the coded information carried in the image, and the coordinate information carried is obtained after decoding; The indoor positioning module in the mobile phone includes a map database import module, an indoor object photo processing module, and a positioning result output module. The indoor object photo processing module collects light positioning image information containing light positioning signals emitted by the LED light source, processes the collected image information, and determines distance information and position information by searching for identification information in a pre-defined map database. The positioning result output module obtains specific position information and marks its ID number on a pre-imported map. The voltage signal encoded by the microcontroller is input to the input terminal Ui of the buffer circuit through the serial port. After isolation, the output Uo of the buffer circuit is connected to the input terminal of the LED drive circuit. The LED driving circuit is constructed using an NPN transistor. The LED driving circuit can make the forward voltage drop of the LED light source higher than the input DC voltage drop. The LED driving circuit can be controlled and the current and luminous flux of the LED light source maintain a linear relationship. The NMOS tube in the LED driver circuit converts the voltage signal into a current signal to drive the LED; The voltage signal output from the microprocessor serial port is converted into current information by using NMOS tube to drive the LED to flash, and the common collector circuit is used as a buffer to allow the light emitted by the LED light source to carry the encoded coordinate information. First, the microcontroller encodes the location coordinate information to be transmitted and outputs it to the input of the buffer circuit through the serial port module. The encoded data stream is then buffered and input into the input port of the LED driver circuit, thereby controlling the high-speed flashing of the LED light source. A mobile phone equipped with a camera and an indoor positioning module then takes a picture of the light emitted by the LED light source, generating an image of stripes with varying brightness. This image is then clustered using the k-means algorithm, and curve fitting is used to generate a dynamic threshold to determine whether the current brightness represents "0" or "1". Finally, the coordinate information is decoded. The microcontroller converts the location coordinate information to be transmitted into binary information, and then converts it into a code using Manchester encoding. The code information output by the microcontroller through the serial port module is TTL level, with 0V voltage representing binary "0" and 3.3V voltage representing binary "1". The LED light source acts as a carrier to transmit this location coordinate information code to the receiving mobile phone. The mobile phone camera captures an LED flashing image. The bright stripes in the image represent binary "1" and the dark stripes represent binary "0." Image processing identifies the coded information, and after decoding, the coordinate information carried by the LED flashing image is obtained. The decoded coordinate information is sent to a database, and a comparative query is performed to obtain a map of the LED light source's location at that moment.

2. The visible light indoor positioning system based on a mobile phone camera according to claim 1, characterized in that: The LED light source adopts white light LED, which modulates the signal into light irradiation energy, and the LED driving circuit is a current driving circuit.

3. The visible light indoor positioning system based on a mobile phone camera according to claim 1, characterized in that: The map database import module determines the area of the location through an image processing algorithm before searching for identification information, and uses the area data to search for identification information in a predefined map database to determine distance information and location information.

4. The visible light indoor positioning system based on a mobile phone camera according to claim 1, characterized in that: The image format of the CMOS camera is YUV format, and the shooting mode is preview mode; At a certain position, one or more images are acquired from a visible light signal emitted by an LED light source at one or more time points, wherein the level of the visible light signal varies between high and low levels at a varying frequency at different time points.

5. The visible light indoor positioning system based on a mobile phone camera according to claim 1, characterized in that: The identification information of the number of changes in bright fringes or dark fringes captured by the CMOS camera includes data bits, the data bits include data corresponding to the frequency change, and each data bit corresponds to one of the numbers of changes in the number of bright fringes or dark fringes; The identification information corresponding to the number of changes in bright stripes or dark stripes also includes one or more start bits and one or more error correction bits. The start bit is used to identify the starting position of a group of data bits, and the error correction bit is used to check errors in the data bits.

Images