A visible light positioning system based on binocular vision algorithm

Abstract

The present invention relates to the fields related to optical camera communication technology and visible light positioning technology, and discloses a visible light positioning system based on a binocular vision algorithm, including a modulation transmitting end and a positioning receiving end; the system overcomes the disadvantage that a traditional direct line-of-sight visible light positioning system cannot locate under occlusion, and 3D positioning can still be achieved through an ambient reflection light system, and only one LED is required to achieve high-precision positioning, which greatly improves the environmental adaptability of the system; at the same time, compared with the traditional satellite positioning method, the system effectively improves the problem of poor indoor environmental signals and ineffective navigation.

Description

Technical Field

[0001] The present invention relates to the fields related to optical camera communication technology and visible light positioning technology, and in particular to a visible light positioning system based on a binocular vision algorithm. Background Art

[0002] With the rapid development of Internet of Things technology, the demand for location-based services has grown rapidly, especially for location services in many large public indoor places such as shopping malls and underground garages. However, the global satellite positioning system's signals are easily attenuated or even directly blocked by walls in indoor environments, resulting in low positioning accuracy or even inability to perform positioning.

[0003] For high-precision positioning systems, the visible light positioning system can achieve high-precision positioning while having a low cost, making it suitable for large-scale promotion and coverage in a short period of time. However, due to the linear propagation characteristic of visible light, when the propagation link between the transmitter and the receiver is blocked, the traditional visible light positioning system cannot work normally. In addition, the traditional visible light positioning algorithm requires that the number of LEDs captured in the FOV of the receiver be more than 3, which will limit the environmental adaptability of the system. Summary of the invention

[0004] The purpose of the present invention is to provide a visible light positioning system based on a binocular vision algorithm to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A visible light positioning system based on binocular vision algorithm, comprising:

[0007] A modulating transmitting end, used for measuring and obtaining LED coordinate information of an LED transmitter relative to the world coordinate system after selecting a world coordinate system, and encoding and modulating the LED coordinate information to obtain coordinate modulated light, so as to output the coordinate modulated light through the LED transmitter;

[0008] The positioning receiving end is used to obtain a modulated image through long and short exposure of the binocular camera, adjust the modulated image model to obtain the coordinate information of the LED transmitter corresponding to the coordinate modulated light it receives, and obtain the posture information of the binocular camera, and calculate the camera coordinates based on the posture information and the LED coordinate information.

[0009] As a further solution of the present invention: the modulation transmitting end specifically includes:

[0010] A coordinate encoding unit, used for measuring and obtaining the LED coordinate information of the LED emitter in the currently selected world coordinate system, and performing Manchester encoding on the LED coordinate information through a micro control unit to obtain the LED coordinate code, wherein the Manchester encoding is a synchronous clock encoding technology that represents information through level jumps;

[0011] The modulation output unit is used to perform OKK modulation on the LED coordinate code, limit it to a preset output frequency, and control the LED transmitter through an output voltage signal so that the LED transmitter emits coordinate modulated light.

[0012] As a further solution of the present invention: the positioning receiving end specifically includes:

[0013] An optical communication unit, used to capture reflected light from the LED transmitter through binocular camera exposure, generate a light and dark stripe image corresponding to the coordinate modulated light, grayscale the light and dark stripe image, and demodulate and decode based on the grayscale value to obtain LED coordinate information, wherein the minimum stripe width in the light and dark stripe image is related to the exposure rolling shutter effect of the camera and the output frequency, and corresponds to dark stripes when the LED is turned off, and corresponds to bright stripes when the LED is turned on;

[0014] A posture judgment unit, used to obtain the current camera posture through inertial measurement structure measurement;

[0015] The coordinate analysis unit is used to calculate the coordinate information of the highlight point in the left camera coordinate system based on the highlight point pixel coordinates of the left and right cameras of the binocular vision algorithm, and based on the projection geometric relationship between the highlight point and the world coordinates of the LED emitter symmetrical about the reflection plane, calculate and obtain the world coordinates of the highlight point corresponding to the LED coordinate information, and based on the camera model of the binocular camera, calculate and obtain the focal coordinates of the camera focus in the world coordinate system, that is, the camera coordinates.

[0016] As a further solution of the present invention: the optical communication unit comprises:

[0017] The image decoding subunit is used to grayscale the light and dark stripe image, filter the background noise through a low-pass filter, extract the edge of the highlight point from the light and dark stripe image after the background noise is filtered using an edge recognition algorithm, and fit the edge of the highlight point through an ellipse fitting algorithm, and the center coordinates of the ellipse are set as the pixel coordinates of the highlight point.

[0018] As a further solution of the present invention: the optical communication unit comprises:

[0019] A short exposure control subunit, used to control the binocular camera to expose in a short exposure mode, and to obtain light and dark stripe images for demodulation to obtain LED coordinate information;

[0020] The long exposure control subunit is used to control the binocular camera to expose in a long exposure mode to obtain the pixel coordinates of the highlight point.

[0021] Compared with the prior art, the beneficial effects of the present invention are: it overcomes the shortcoming of the traditional direct line-of-sight visible light positioning system that cannot locate under obstruction, and can still achieve 3D positioning through the ambient reflection light system, and only requires one LED to achieve high-precision positioning, which greatly improves the environmental adaptability of the system; at the same time, compared with the traditional satellite positioning method, it effectively improves the problem of poor indoor environmental signals and ineffective navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a block diagram of a visible light positioning system based on binocular vision algorithm.

[0023] Figure 2 This is a block diagram of the modulation transmitter in a visible light positioning system based on binocular vision algorithm.

[0024] Figure 3 The figure is a block diagram of the positioning receiving end in a visible light positioning system based on a binocular vision algorithm.

[0025] Figure 4 This is a model diagram of a visible light positioning system based on binocular vision algorithm.

[0026] Figure 5 Schematic diagram of a binocular vision algorithm for a visible light positioning system based on a binocular vision algorithm. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0028] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0029] like Figure 1 The present invention provides a visible light positioning system based on a binocular vision algorithm according to an embodiment of the present invention, comprising:

[0030] The modulation transmitting end 100 is used to measure and obtain the LED coordinate information of the LED transmitter relative to the world coordinate system after selecting the world coordinate system, and encode and modulate the LED coordinate information to obtain coordinate modulated light, so as to output the coordinate modulated light through the LED transmitter.

[0031] The positioning receiving end 200 is used to obtain a modulated image through long and short exposure of the binocular camera, adjust the modulated image model to obtain the coordinate information of the LED transmitter corresponding to the coordinate modulated light it receives, and obtain the posture information of the binocular camera, and calculate the camera coordinates based on the posture information and the LED coordinate information.

[0032] Further, such as Figure 2 As shown, the modulation transmitting end specifically includes:

[0033] The coordinate encoding unit 101 is used to measure and obtain the LED coordinate information of the LED transmitter in the currently selected world coordinate system, and Manchester encodes the LED coordinate information through the micro control unit to obtain the LED coordinate code. The Manchester encoding is a synchronous clock encoding technology that represents information through level jumps.

[0034] The modulation output unit 102 is used to perform OKK modulation on the LED coordinate code, limit it to a preset output frequency, and control the LED transmitter through an output voltage signal so that the LED transmitter emits coordinate modulated light.

[0035] Further, such as Figure 3 As shown, the positioning receiving end 200 specifically includes:

[0036] The optical communication unit 201 is used to capture the reflected light from the LED transmitter through binocular camera exposure, generate a light and dark stripe image corresponding to the coordinate modulated light, grayscale the light and dark stripe image, and demodulate and decode based on the grayscale value to obtain LED coordinate information. The minimum stripe width in the light and dark stripe image is related to the exposure rolling shutter effect of the camera and the output frequency. When the LED is turned off, it corresponds to dark stripes, and when the LED is turned on, it corresponds to bright stripes.

[0037] The posture judgment unit 202 is used to obtain the current camera posture through inertial measurement structure measurement.

[0038] The coordinate analysis unit 203 is used to calculate the coordinate information of the highlight point in the left camera coordinate system based on the highlight point pixel coordinates of the left and right cameras of the binocular camera through a binocular vision algorithm, and to obtain the world coordinates of the highlight point corresponding to the LED coordinate information based on the projection geometric relationship between the highlight point and the world coordinates of the LED emitter symmetrical about the reflection plane, and to calculate the focal coordinates of the camera's focus in the world coordinate system, that is, the camera coordinates, based on the camera model of the binocular camera.

[0039] Furthermore, the optical communication unit 201 includes:

[0040] The image decoding subunit is used to grayscale the light and dark stripe image, filter the background noise through a low-pass filter, extract the edge of the highlight point from the light and dark stripe image after the background noise is filtered using an edge recognition algorithm, and fit the edge of the highlight point through an ellipse fitting algorithm, and the center coordinates of the ellipse are set as the pixel coordinates of the highlight point.

[0041] Furthermore, the optical communication unit 201 includes:

[0042] The short exposure control subunit is used to control the binocular camera to expose in a short exposure mode, and to obtain light and dark stripe images to demodulate and obtain LED coordinate information.

[0043] The long exposure control subunit is used to control the binocular camera to expose in a long exposure mode to obtain the pixel coordinates of the highlight point.

[0044] In this embodiment, the binocular vision algorithm is applied to visible light positioning. The whole system model is as follows: Figure 4 As shown in the figure, the world coordinates of the LED transmitter are Manchester encoded and OOK modulated, flashing at a high speed of 3.3kHz. The binocular camera at the receiving end captures the LED light signal reflected by the ground through long and short exposure modes respectively. On the one hand, the LED coordinates are demodulated using the threshold algorithm according to the light and dark stripes on the short exposure photo. On the other hand, the pixel coordinates of the highlight points on the long exposure photo are extracted using edge recognition and ellipse fitting algorithms, and then the coordinates of the highlight points in the left camera coordinate system are calculated using the binocular vision algorithm. At the same time, the IMU fixed on the binocular camera will measure the camera's posture. Finally, the camera's position is estimated by the reprojection error minimization algorithm based on the LED world coordinate system, the pixel coordinates of the highlight points and the camera posture. For the hardware test platform selection of the system, at the transmitting end, the STM32 MCU encodes the LED coordinates, and then controls the drive controller in OOK modulation mode, thereby controlling the on and off of the voltage output by the DC power supply to the LED fixed on the ceiling. At the receiving end, the IMU and the binocular camera ZED2 are fixed together to capture the reflected light from the ground.

[0045] More specifically, first, the world coordinate system is selected and the coordinates of the LED in the world coordinate system are measured; the coordinates of the LED are encoded, and then the LED is switched on and off using the OOK modulation method; in the initial frame, the binocular camera is set to short exposure mode to capture reflected light, and a high-contrast photo with light and dark stripes is captured by adjusting the camera's ISO and exposure time. The symbols "0" and "1" are demodulated based on the light and dark stripes on the photo and the OOK modulation method, and based on the grayscale value; then, the bit data streams of "0" and "1" are decoded into the world coordinates of the initial data LED using Manchester encoding; in the case of direct line of sight, there will be highlight overflow, and only a small area (ROI) where the LED is located will have light and dark stripes, In non-line-of-sight conditions, due to the existence of diffuse reflection, all pixels in the entire photo will collect light signals, and due to the decrease in light intensity, highlight overflow will not occur, so we can perform cumulative sampling on the entire photo to reduce the bit error rate; however, due to the relatively weak intensity of the reflected light, the signal-to-noise ratio of the reflected light signal is relatively low, the contrast of the light and dark stripes on the photo is not as high as in direct line-of-sight, and the edges are more blurred. In order to improve the accuracy of decoding, we adopted a two-level threshold solution. In addition to considering the pixel width difference near the peak of the grayscale curve, it is also necessary to analyze the pixel width difference near the trough, so as to more accurately determine whether the current waveform is "11" and "1" or "00" and "0"; therefore, based on the traditional first-order threshold demodulation algorithm, we designed a two-level threshold demodulation algorithm based on full-pixel cumulative sampling.

[0046] Specifically, for extracting the pixel coordinates of the highlight points, first, perform the following operations on both the left and right cameras: adjust the ISO and exposure time of the binocular cameras, then take a long-exposure photo containing the reflection highlight points of the LED on the ground, first convert the photo into a grayscale image, and then use a low-pass filter to filter the background noise. After that, use the edge recognition algorithm to extract the edges of the highlight points, and then use the ellipse fitting algorithm to fit the edges of the highlight points, and the center coordinates of the fitted ellipse are regarded as the pixel coordinates of the highlight points.

[0047] For the calculation estimate of the camera coordinates, such as Figure 5 As shown in the figure, before the experiment, the binocular camera calibration algorithm is used to calibrate the camera's internal parameters to reduce the distortion error. After obtaining the internal parameters, in addition to knowing the internal parameters of the two cameras, the relative position between the two cameras can also be known. The coordinates of the highlight point on the left and right camera coordinate systems can be calculated through the pinhole camera model based on the pixel coordinates of the highlight point on the left and right camera and the relative position between the two cameras.

[0048] Regarding the method of measuring camera attitude based on IMU, first, the IMU is fixed on the binocular camera, and the internal coordinate system of the IMU is adjusted to be consistent with the camera coordinate system. The IMU has an accelerometer, a gyroscope and a magnetometer. The accelerometer can be used to know the three components of gravity acceleration in the camera coordinate system under the current attitude. The relationship between the processed acceleration vector in the world coordinate system and the gravity acceleration vector in the camera coordinate system can be used to calculate the rotation angle of the camera coordinate system relative to the world coordinate system on the X and Y axes. Through these two angles and the magnetometer, the rotation angle of the other Z axis can be calculated; in addition, the gyroscope can calculate the change in angle over a period of time. Through the Kalman filter algorithm, the data of the three instruments can be fused to calculate a more accurate and stable camera attitude angle.

[0049] For estimating the camera position through the reprojection error minimization algorithm, the position of the highlight point in the world coordinate system is equal to the coordinates of the symmetric point of the LED's world coordinate system with respect to the ground, which is converted to the left camera coordinate system, and the second norm of the coordinates of the highlight point in the left camera coordinate system is calculated. Since the camera position vector is included in the formula, the value of the corresponding position vector when the second norm takes the minimum value is the target value.

[0050] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0051] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the disclosure in the specification and examples. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0052] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A visible light positioning system based on binocular vision algorithm, characterized in that: include: A modulating transmitting end, used for measuring and obtaining LED coordinate information of an LED transmitter relative to the world coordinate system after selecting a world coordinate system, and encoding and modulating the LED coordinate information to obtain coordinate modulated light, so as to output the coordinate modulated light through the LED transmitter; A positioning receiving end, used to obtain a modulated image through long and short exposure of a binocular camera, adjust the modulated image model to obtain the coordinate information of the LED transmitter corresponding to the coordinate modulated light received by the positioning receiving end, and obtain the posture information of the binocular camera, and calculate and obtain the camera coordinates based on the posture information and the LED coordinate information; The modulation transmitting end specifically includes: A coordinate encoding unit, used for measuring and obtaining the LED coordinate information of the LED emitter in the currently selected world coordinate system, and performing Manchester encoding on the LED coordinate information through a micro control unit to obtain the LED coordinate code, wherein the Manchester encoding is a synchronous clock encoding technology that represents information through level jumps; A modulation output unit, used for performing OKK modulation on the LED coordinate code, limiting it to a preset output frequency, and controlling the LED transmitter through an output voltage signal so that the LED transmitter emits coordinate modulated light; The positioning receiving end specifically includes: An optical communication unit, used to capture reflected light from the LED transmitter through binocular camera exposure, generate a light and dark stripe image corresponding to the coordinate modulated light, grayscale the light and dark stripe image, and demodulate and decode based on the grayscale value to obtain LED coordinate information, wherein the minimum stripe width in the light and dark stripe image is related to the exposure rolling shutter effect of the camera and the output frequency, and corresponds to dark stripes when the LED is turned off, and corresponds to bright stripes when the LED is turned on; A posture judgment unit, used to obtain the current camera posture through inertial measurement structure measurement; The coordinate analysis unit is used to calculate the coordinate information of the highlight point in the left camera coordinate system based on the highlight point pixel coordinates of the left and right cameras of the binocular vision algorithm, and based on the projection geometric relationship between the highlight point and the world coordinates of the LED emitter symmetrical about the reflection plane, calculate and obtain the world coordinates of the highlight point corresponding to the LED coordinate information, and based on the camera model of the binocular camera, calculate and obtain the focal coordinates of the camera focus in the world coordinate system, that is, the camera coordinates.

2. A visible light positioning system based on binocular vision algorithm according to claim 1, characterized in that: The optical communication unit comprises: The image decoding subunit is used to grayscale the light and dark stripe image, filter the background noise through a low-pass filter, extract the edge of the highlight point from the light and dark stripe image after the background noise is filtered using an edge recognition algorithm, and fit the edge of the highlight point through an ellipse fitting algorithm, and the center coordinates of the ellipse are set as the pixel coordinates of the highlight point.

3. A visible light positioning system based on binocular vision algorithm according to claim 2, characterized in that: The optical communication unit comprises: A short exposure control subunit, used to control the binocular camera to expose in a short exposure mode, and to obtain light and dark stripe images for demodulation to obtain LED coordinate information; The long exposure control subunit is used to control the binocular camera to expose in a long exposure mode to obtain the pixel coordinates of the highlight point.

Images