Flat panel light source visual recognition matching positioning method and system based on visible light communication

By using visible light communication technology and CMOS image sensor on flat panel LED light sources for visual imaging, combined with IMU attitude sensor, the characteristic information of rectangular light sources is extracted, and the problems of insufficient accuracy and high cost in the prior art are solved, and the high level of centimeters is achieved Accurate indoor positioning to meet the robot navigation needs.

CN113607158BActive Publication Date: 2024-11-19CHINA RAILWAY ENGINEERING EQUIPMENT GROUP CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202110895779.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-05
Publication Date
2024-11-19
Estimated Expiration
2041-08-05

AI Technical Summary

Technical Problem

The existing indoor positioning technology has insufficient accuracy, high cost and poor real-time performance, which is difficult to meet the centimeter-level positioning requirements of indoor robots. Especially in the visible light communication visual positioning method using circular downlights, the image bit error rate is high and the positioning delay is large. Moreover, the size of a single positioning mark is limited, making it difficult to provide high-precision visual positioning.

Method used

Using a flat panel LED light source visual recognition matching positioning method based on visible light communication, the rectangular LED light source is visually imaged through a CMOS image sensor, and the light and dark stripes and contours of the rectangular light source are extracted using digital image processing technology, and combined with the IMU attitude sensor to record the inclined attitude angle and heading The signal autocorrelation method is used to identify the light source ID information and calculate the four corner points to achieve high-precision visual positioning.

Benefits of technology

It realizes high-precision indoor mobile positioning at centimeter level, meets the navigation needs of indoor robots, simplifies the visual positioning process, and requires only a rectangular flat light source to position, improving the accuracy and stability of positioning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113607158B_ABST
    Figure CN113607158B_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of indoor positioning of visible light communication, and proposes a visual recognition matching positioning method and system of a flat-panel light source based on visible light communication, so as to solve the technical problems of insufficient precision, high cost and poor real-time performance of existing indoor positioning methods. The present invention images a rectangular flat-panel LED light source of visible light communication through the rolling shutter effect of a CMOS image sensor, obtains the outline of the rectangular area of ​​the light source and the light and dark stripe information of the image using a digital image processing method, and realizes light source matching and recognition by defining the autocorrelation sequence and proximity of the light source image signal, thereby obtaining the ID of the flat-panel light source and the coordinate information of the rectangular corner points, and realizes the matching of the image coordinates of the four corner points of the rectangular light source with the world coordinates on the basis of the auxiliary imaging of the IMU attitude sensor, realizes high-precision visual positioning based on the visible light communication flat-panel light source, realizes centimeter-level mobile positioning, and can meet the application requirements of indoor robot mobile positioning and navigation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of indoor visible light communication visual positioning, and in particular to a method and system for visual recognition, matching and positioning of a flat-panel light source based on visible light communication, which realizes visual recognition, matching and positioning of a flat-panel LED light source based on visible light communication technology. Background Art

[0002] It is difficult for indoor and other underground space environments to receive outdoor satellite navigation signals. Some indoor carriers need to use indoor positioning technology when they move autonomously indoors. At present, indoor positioning technology is developing rapidly. Among them, indoor positioning technology based on visible light communication can realize the unification of location information services and lighting, and has become one of the main development directions of indoor positioning technology in the future. With the upgrade of indoor lighting systems, LED lighting systems are widely used in large buildings such as underground parking lots, shopping malls, and automated factories. These areas are also key places for future applications of autonomous driving and intelligent robots. For example, the flat-panel LED light source in an underground parking lot has a length of more than 0.5m and a width of more than a quarter of the length. It has a significant rectangular visual feature, so it can be used as a logo for building location information services while meeting lighting requirements.

[0003] At present, the main problems of existing indoor positioning solutions on the market are:

[0004] (1) The positioning method based on radio signals is not accurate enough. The positioning accuracy of indoor positioning methods using WIFI / Bluetooth / ZigBee is generally 1-10m, which is difficult to meet the centimeter-level positioning accuracy requirements of indoor robots.

[0005] (2) Excessive cost and complex system. The positioning solution based on UWB (ultra-wideband) can achieve a test accuracy of 10cm, but it requires the deployment of dense positioning base stations. The system cost is high, the deployment is complex, and it is susceptible to electromagnetic interference. The system positioning accuracy is not stable enough. Too many base station deployments may interfere with the indoor electromagnetic environment.

[0006] (3) The positioning delay is large and the real-time performance is insufficient. The solution based on visual image monitoring can monitor and locate moving targets. After the system completes the positioning, it will feedback the position information to the moving carrier with a certain delay, which is not suitable for autonomous positioning of mobile targets.

[0007] (4) Currently, most visible light communication visual positioning uses circular downlights as positioning markers. The size of circular downlights is limited and can only provide one positioning reference point and scale information. Therefore, more than two circular downlights are required to provide an effective visual positioning reference.

[0008] (5) Visible light communication-based visual imaging decoding technology has high requirements on imaging quality and a high image bit error rate, making it difficult to decode in real time. Summary of the invention

[0009] In view of the technical problems that existing indoor positioning methods are insufficiently accurate, high in cost, poor in real-time performance, and fail to meet the positioning and navigation requirements of indoor mobile robots, the present invention proposes a flat-panel light source visual recognition and matching positioning method and system based on visible light communication. High-precision indoor positioning is achieved based on visible light communication, i.e., indoor flat-panel LED and monocular vision measurement, which meets the indoor high-precision mobile positioning scenario and is suitable for high-precision mobile positioning and navigation in indoor and other underground spaces.

[0010] In order to achieve the above-mentioned purpose, the technical solution of the present invention is implemented as follows: a flat-panel light source visual recognition, matching and positioning method based on visible light communication, which uses visible light communication technology to load an ID signal on a rectangular LED light source, uses a CMOS image sensor to perform visual imaging of the rectangular LED light source, and uses an IMU attitude sensor to record the tilt attitude angle and heading angle during imaging; uses digital image processing technology to extract the rectangular light source image, obtains the light and dark stripes grayscale image and rectangular outline of the rectangular light source area, and then uses the signal autocorrelation method to identify the ID information of the matching light source and calculate the four corner points of the rectangular outline of the light source, and realizes high-precision visual positioning according to the principle of photogrammetry.

[0011] Furthermore, the CMOS image sensor adjusts the ISO value and the exposure to perform visual imaging on the rectangular LED light source to obtain a flat light source image, wherein the flat light source image contains continuous stripe information of the light source.

[0012] Furthermore, the method for obtaining the rectangular outline of the rectangular light source area is as follows: perform Gaussian operation on the flat light source image, binarize it through the OPENCV library and find the outline of the rectangular area; use the edge extraction method to extract the outline sequence points of the rectangular area, fit four outline lines according to the outline sequence points, and the connection points of adjacent outline lines are the corner points of the rectangular light source area.

[0013] Furthermore, the method of using the signal autocorrelation method to identify the ID information of matching light sources is: converting the ID information matching problem of the two light sources into the autocorrelation sequence matching problem of the stripe signals of the stripe grayscale images of the two light sources: calculating the period of the grayscale stripes in the light and dark stripe grayscale image based on the autosignal autocorrelation method and calculating the autocorrelation sequence of one period, using the autocorrelation sequence as a matching feature to associate with the signal modulated ID information to establish a light source matching database, and calculating the minimum proximity K to perform light source matching in the light source matching database.

[0014] Furthermore, the table in the light source matching database includes the light source number, the light source matching template, and the coordinates of the four corner points of the light source; the light source matching obtains the light source number and the world coordinates of the four corner points of the light source corresponding to the light source number.

[0015] Furthermore, the method for establishing the light source matching database is: performing mean calculation on the acquired light and dark stripe grayscale image of the rectangular light source area row by row to generate a light and dark stripe sequence V = {v1, v2, v3, ···, v i},v1,v2,v3,···,v i ∈[0,255], i is the row number of the gray light and dark stripe image; the light and dark stripe sequence V is subjected to signal autocorrelation calculation to obtain the signal autocorrelation sequence N={n1,n2,n3,···,n T ,···}, and calculate the period T of the autocorrelation sequence N; extract the first T sequence values ​​of the signal autocorrelation sequence N for normalization processing, and obtain the standardized sequence X={x1,x2,x3,···,x T},x1,x2,x3,···,x T ∈[0,1]; associate the autocorrelation sequence N as a matching feature with the ID information of the signal modulation to establish a light source matching database.

[0016] Furthermore, the calculation method of the proximity K is as follows: define the matching degree sequence D of two light sources T =X1 T -X2 T , X1 T , X2 T are the normalized signal autocorrelation sequences of two light sources in one cycle, and D T ={d1,d2,d3,···,d T},d1,d2,d3,···,d T ∈(-1,1);

[0017] Then the matching degree of the ID information of the two light sources, i.e. the proximity, is K = a×H+b×M+c×S+d×W;

[0018] Among them, a, b, c, d are coefficients, H is the sequence D T The difference between the maximum and minimum values ​​of T The average value of the sum of the absolute values ​​of T The standard deviation of the sequence D T Exceeding sequence D T The average of the sum of the absolute values ​​of the sequence with the mean value being twice the standard deviation; the smaller the value of proximity K is, the higher the matching degree is.

[0019] Furthermore, when the CMOS type is the same, the period T is the same and the proximity K is the smallest, it is the best matching condition, and the ID information of the light source that best matches the rectangular flat light source to be positioned in the light source matching database is obtained, and the world coordinates of the four corner points of the light source corresponding to the ID information are used as the coordinates of the light source corner points {P1, P2, P3, P4}.

[0020] Furthermore, when determining the correspondence between the world coordinates of the corner points of the rectangular LED light source and the coordinates of its image point, it is assumed that the position and posture of the CMOS image sensor is calculated based on the homography direct method using a correspondence relationship, and the difference between the calculated heading angle and the geomagnetic direction angle of the IMU attitude sensor is compared. If the difference is less than a preset threshold, it is a correct relationship, otherwise the calculation is continued; there are 4 types of sequence correspondences for the 4 corner points, and the correspondence can be determined by calculating 4 times at most; according to the correct correspondence, the IMU attitude sensor-assisted iterative method is used to perform imaging positioning calculations.

[0021] Furthermore, the preset threshold is a deviation angle C between the heading angle and the geomagnetic azimuth angle < 45°.

[0022] Furthermore, the photogrammetry principle uses a visual plane four-point positioning method to calculate the position information of the camera of the CMOS image sensor relative to the rectangular flat light source to achieve visual positioning.

[0023] A flat-panel light source visual recognition matching positioning system based on visible light communication comprises a rectangular flat-panel light source and a positioning device, wherein the rectangular flat-panel light source is a visible light communication flat-panel LED light source; the positioning device comprises a COMS imaging sensor and an IMU attitude sensor, and both the COMS imaging sensor and the IMU attitude sensor are connected to a microprocessor.

[0024] Furthermore, the microprocessor is connected to a battery, and both the microprocessor and the battery are arranged in a shell, a COMS imaging sensor is fixedly arranged on the shell, and an IMU attitude sensor is arranged in the shell; the microprocessor processes data collected by the IMU attitude sensor and the COMS imaging sensor to realize visual recognition matching positioning.

[0025] The beneficial effects of the present invention are as follows: a rectangular flat light source is used as a visual positioning beacon, visible light communication technology is used to modulate the positioning ID information on the rectangular flat light source, the flat light source is imaged by a CMOS image sensor, the ISO value is adjusted to obtain continuous stripe information, the stripe signal period is calculated by calculating the autocorrelation sequence of the stripe image, and a signal autocorrelation sequence of a period is used as a matching feature to associate with the ID information to establish a light source matching database, the light source matching database contains the signal autocorrelation sequence of the light source matching and the spatial three-dimensional coordinate information of the four corner points of the corresponding light source, and a proximity K calculation method is proposed to realize the visual recognition and matching of the light source; while the light source is recognized and matched, the rectangular corner points of the rectangular flat light source are extracted by digital image processing technology, and visual positioning can be realized using only one rectangular flat light source, making the visual positioning simpler and more robust. The positioning method of the present invention based on the fusion of visible light communication and visual measurement can realize centimeter-level mobile positioning using indoor rectangular flat LED light sources, and can meet the application requirements of indoor robot mobile positioning and navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 It is a schematic diagram of the process of the present invention.

[0028] Figure 2 It is a structural schematic diagram of the present invention.

[0029] Figure 3 This is a visual extraction effect diagram of the rectangular flat light source of the present invention.

[0030] Figure 4 This is a flow chart of the light source visual matching of the present invention.

[0031] Figure 5 This is an effect diagram of visual matching of multiple light sources of the present invention.

[0032] Figure 6 It is a flow chart of the visual positioning of the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] Embodiment 1, as Figure 1 As shown, a flat panel light source visual recognition matching positioning method based on visible light communication, the steps are as follows:

[0035] Step 1: Use visible light communication technology to perform signal modulation on the rectangular flat light source, use the CMOS image sensor to perform visual imaging of the rectangular flat light source, and use the IMU attitude sensor to record the tilt attitude angle and heading angle during visual imaging.

[0036] A flat panel light source matching algorithm and positioning reference point extraction method are proposed for visible light communication flat panel LED light sources, and an indoor visual positioning system based on flat panel light sources is realized. Visible light communication technology is used to realize visible light communication signal modulation of rectangular flat panel light sources and modulate ID information. In visible light communication technology, signal modulation is performed based on LED-ID technology, the modulation frequency is 16kHz, and Manchester encoding is used to form 3 hexadecimal digits.

[0037] The present invention uses a common CMOS image sensor, i.e., a CMOS rolling shutter camera, to adjust the ISO value and exposure to perform visual imaging of the light source, such as Figure 3 The obtained flat light source image shown includes two light source images, a rectangular light source and a square light source; the flat light source image contains continuous stripe information of the rectangular flat light source. The ISO value is a parameter for adjusting the photosensitivity of the CMOS image sensor, which is used to cooperate with the exposure to make the light source image easier to distinguish and form a light and dark stripe image. Visual imaging is used for subsequent visual positioning, and the visual imaging positioning accuracy is high, and the light source imaging can be recognized at the same time. The positioning device includes an IMU attitude sensor, which is integrated with the CMOS image sensor to form an imaging positioning module device. The IMU attitude sensor is used to assist imaging, and the heading angle measured by the IMU is used to determine the correspondence between the four corner points of the light source image and its coordinates in the world coordinate system. The light and dark stripe sequence obtained by controlling the photosensitivity ISO value of the CMOS image sensor has continuity, and the rectangular contour line of the light source is extracted by the contour edge extraction method of the OPENCV library of digital image processing. The present invention uses a rectangular LED light source to extract stripe information and rectangular contour features.

[0038] Step 2: Use computer vision extraction algorithm to extract the rectangular area of ​​the rectangular flat light source in the visual imaging image. Fit four straight lines to the four contour edges of the rectangular area respectively. The four straight lines intersect to obtain the coordinates of the four corner points of the light source. Sort the image coordinates of the four corner points counterclockwise.

[0039] The implementation method of the computer vision extraction algorithm is as follows: obtain the image of the rectangular light source collected by the visual imaging of the CMOS image sensor, first perform Gaussian operation on the image, then binarize it through the OPENCV library and find the rectangular contour. Use the digital image edge extraction method to extract the contour image points of the light source area, fit four contour lines according to the light source contour sequence points, and the connecting points of the contour lines are the rectangular corner points of the flat light source area. The coordinates of the four corner points of the rectangular light source are obtained by fitting the light source contour points. The rectangular contour points are all sequences, and four straight lines can be fitted first, and then the adjacent straight lines are intersected to obtain four corner points for subsequent visual positioning reference.

[0040] Step 3: Calculate the period of the grayscale stripes of the light source image based on the signal autocorrelation method and calculate the autocorrelation sequence of one period, and associate the autocorrelation sequence with the ID information of the signal modulation in step 1 as a matching feature to establish a light source matching database.

[0041] The difference between this application and the invention patent with application number 201710150722.9 is that the light and dark stripe information obtained by the mobile phone has a large degree of distinction and can be directly decoded. The present invention does not decode directly, but solves the period to perform periodic signal matching. The advantage lies in recognition robustness. Decoding has high requirements on images and is prone to decoding errors or non-recognition. Light source matching and recognition can be completed using an ordinary camera.

[0042] Signal autocorrelation is a commonly used digital signal processing method. The light source matching obtains the light source number, which corresponds to the world coordinates of the four corner points of the light source. This is measured in advance and entered in advance in the database information.

[0043] The extracted light source grayscale fringe image is averaged row by row to generate a light and dark fringe sequence V = {v1, v2, v3, ···, v i}, v∈[0,255], i is the row number of the grayscale stripe image, corresponding to the row of the image. The light and dark stripe sequence V is subjected to signal autocorrelation calculation to obtain the signal autocorrelation sequence N={n1,n2,n3,···,n T ,···}, and calculate the period T of the autocorrelation sequence N. The periodic function will have a maximum value at the multiple of period T.

[0044] Extract the first T sequence values ​​of the signal autocorrelation sequence N and perform normalization processing to obtain the standardized sequence X = {x1, x2, x3, ···, x T}, x∈[0,1], the normalized range. The autocorrelation sequence is used as a matching feature and associated with the ID information of the signal modulation in step 1 to establish a light source matching database. The table in the light source matching database contains the light source number, light source matching template, and the coordinates of the four corner points of the light source.

[0045] Step 4: Perform light source matching database matching based on the minimum proximity K (nearest neighbor calculation method) calculation method, obtain the reference coordinates of the four corner points of the rectangular flat light source, and realize light source visual recognition matching.

[0046] The light source stripes can be normalized to a periodic numerical sequence X, which has T dimensions. This sequence is the matching template, and the calculation method of the proximity K is the calculation method of the matching degree of the two sequences. The matching degree problem of the ID information of the two light sources of a periodic standardized sequence X is converted into the matching degree problem of the autocorrelation sequence of the stripe signal of the grayscale stripe image of the two light sources. Define the matching degree sequence D of the two light sources T =X1 T -X2 T , X1 T , X2 T are the normalized signal autocorrelation sequences of two light sources in one cycle, and D T ={d1,d2,d3,···,d T},d∈(-1,1).

[0047] The ID information matching degree of two light sources is defined as: K = a×H+b×M+c×S+d×W;

[0048] Among them, a, b, c, d are coefficients, H is the sequence D T The difference between the maximum and minimum values ​​of T The average value of the sum of the absolute values ​​of T The standard deviation of the sequence D T Exceeding sequence D T The average value of the sum of the absolute values ​​of the sequence with the mean value being twice the standard deviation. As a special case, in the specific implementation, the coefficients a=0.25; b=0.25, c=4, d=0.02; the smaller the value of the proximity K, the higher the matching degree.

[0049] The specific light source matching process is as follows: Figure 4As shown, for the grayscale stripe image of the rectangular flat LED light source, the type of CMOS image sensor is obtained, the light and dark stripe sequence V of the grayscale stripe image is extracted, a periodic autocorrelation sequence is calculated and normalized to a standardized sequence X, and the proximity K is calculated to match the light source information (matching) database. When the CMOS type is the same, the period T is the same, and the proximity K is the minimum, it is the best matching condition. The ID information of the light source in the light source information database that best matches the rectangular flat light source to be located (that is, the imaging to be matched) is obtained and used as the coordinates of the light source corner points {P1, P2, P3, P4}. According to the light source matching process, the visual matching effect of multiple light sources is as follows Figure 5 As shown by Figure 5 It can be seen that the above light source matching method can solve the problem of multi-light source recognition and can also extract the four corner points of the light source.

[0050] Step 5. Finally, the visual plane four-point positioning method is used to calculate the position information of the camera of the CMOS image sensor relative to the rectangular flat light source to achieve visual positioning. The visual plane four-point positioning method is used to convert the reference coordinates of the four corner points of the rectangular flat light source obtained in step 4 into world coordinates to achieve the positioning of the rectangular flat light source.

[0051] The coordinates of the four corner points of the rectangular flat light source corresponding to the ID information are obtained according to the light source visual matching. When determining the corresponding relationship between the world coordinates of the corner points of the rectangular light source and its image point coordinates, it is first assumed that a corresponding relationship is used to calculate the position and posture of the CMOS image sensor based on the homography direct method, and then the calculated heading angle is compared with the difference in the geomagnetic direction of the IMU attitude sensor. If the difference is less than a certain threshold, it is a correct relationship, otherwise it continues to be assumed to be calculated. The process is as follows Figure 6 As shown in the figure, there are 4 kinds of sequence correspondences of the 4 corner points, so the correspondence can be determined by calculating at most 4 times, and then the coordinates of the positioning device are calculated using the common visual positioning algorithm according to the correct correspondence, and the IMU attitude sensor is used to assist the iterative method for imaging positioning calculation.

[0052] The digital image processing method of the opencv library obtains the image counterclockwise sequence coordinates {c1, c2, c3, c4}. The homography direct method assumes a correspondence between the world coordinates of the rectangular corner point sequence and the image coordinate sequence. The heading angle H of the CMOS image sensor is calculated according to the visual positioning algorithm. The imaging positioning obtains the three-dimensional position and three-dimensional posture, one of which is the heading angle H. At the same time, the geomagnetic azimuth M of the IMU sensor is obtained, and the deviation angle C between the heading angle H and the geomagnetic azimuth M is calculated. If C < 45°, the correct coordinate correspondence is output.

[0053] The present invention images a rectangular flat-panel light source for visible light communication through the rolling shutter effect of a CMOS imaging sensor, obtains the outline of the rectangular area of ​​the light source and the light and dark stripe information of the image using a digital image processing method, realizes light source matching and identification by defining the autocorrelation sequence and matching degree K of the light source image signal, thereby obtaining the ID of the light source and the coordinate information of the rectangular corner points of the flat-panel light source, realizes the matching of the image coordinates of the four corner points of the rectangular light source with the world coordinates on the basis of the auxiliary imaging of the IMU attitude sensor, and further realizes high-precision visual positioning based on the visible light communication flat-panel light source.

[0054] Embodiment 2, a flat panel light source visual recognition matching positioning system, such as Figure 2 As shown, it includes a rectangular flat light source 1 and a positioning device. The rectangular flat light source 1 is a visible light communication flat light source and serves as a positioning light source. The positioning device positions the rectangular flat light source 1. The positioning device includes a COMS imaging sensor 2 and an IMU attitude sensor 3. The COMS imaging sensor 2 and the IMU attitude sensor 3 are both connected to a microprocessor 4. The microprocessor 4 is connected to a battery 5. The microprocessor 4 and the battery 5 are both arranged in a housing 6. The COMS imaging sensor 2 is fixedly arranged on the housing 6 for collecting images of the rectangular flat light source 1; the IMU attitude sensor 3 is arranged in the housing 6 for collecting the attitude of the positioning device. The microprocessor 4 is used to process the data collected by the IMU attitude sensor 3 and the COMS imaging sensor 2. The microprocessor 4 is provided with a visual computing positioning module, and the visual recognition matching positioning is realized by using a visual positioning algorithm. The battery 5 is used to power the entire device.

[0055] The other structures and principles are the same as those in Example 1.

[0056] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A flat panel light source visual recognition matching positioning method based on visible light communication, characterized in that: Use visible light communication technology to load ID signals on rectangular LED light sources, use CMOS image sensors to perform visual imaging of rectangular LED light sources, and use IMU attitude sensors to record the tilt attitude angle and heading angle during imaging; use digital image processing technology to extract the rectangular light source image, obtain the light and dark stripe grayscale image and rectangular outline of the rectangular light source area, and then use the signal autocorrelation method to identify the ID information of the matching light source and calculate the four corner points of the rectangular outline of the light source, and achieve high-precision visual positioning based on the principle of photogrammetry; The method for identifying the ID information of the matching light source using the signal autocorrelation method is as follows: converting the ID information matching problem of two light sources into the matching problem of the autocorrelation sequence of the stripe signals of the stripe grayscale images of the two light sources: calculating the period of the grayscale stripes in the light and dark stripe grayscale image based on the autocorrelation method of the self-signal, and calculating the autocorrelation sequence of one period, associating the autocorrelation sequence with the ID information modulated by the signal as a matching feature to establish a light source matching database, and calculating the minimum proximity K to perform light source matching in the light source matching database; The calculation method of the proximity K is as follows: define the matching degree sequence D of two light sources T =X1 T -X2 T , X1 T , X2 T are the normalized signal autocorrelation sequences of two light sources in one cycle, and D T ={d1,d2,d3,···,d T },d1,d2,d3,···,d T ∈(-1,1); Then the matching degree of the ID information of the two light sources, i.e. the proximity, is K = a×H+b×M+c×S+d×W; Among them, a, b, c, d are coefficients, H is the sequence D T The difference between the maximum and minimum values ​​of T The average value of the sum of the absolute values ​​of T The standard deviation of the sequence D T Exceeding sequence D T The average of the sum of the absolute values ​​of the sequence with the mean value being twice the standard deviation; the smaller the value of proximity K is, the higher the matching degree is.

2. The method for visual recognition, matching and positioning of a flat panel light source based on visible light communication according to claim 1, characterized in that: The CMOS image sensor adjusts the ISO value and the exposure to perform visual imaging on the rectangular LED light source to obtain a flat light source image, wherein the flat light source image contains continuous stripe information of the light source.

3. The method for visual recognition, matching and positioning of a flat panel light source based on visible light communication according to claim 2, characterized in that: The method for obtaining the rectangular outline of the rectangular light source area is as follows: performing Gaussian operation on the flat light source image, binarizing it through the OPENCV library and finding the outline of the rectangular area; using the edge extraction method to extract the outline sequence points of the rectangular area, fitting four outline lines according to the outline sequence points, and the connection points of adjacent outline lines are the corner points of the rectangular light source area.

4. The method for visual recognition, matching and positioning of a flat panel light source based on visible light communication according to claim 1, characterized in that: The table in the light source matching database includes the light source number, the light source matching template, and the coordinates of the four corner points of the light source; the light source matching obtains the light source number and the world coordinates of the four corner points of the light source corresponding to the light source number.

5. The method for visual recognition, matching and positioning of a flat panel light source based on visible light communication according to claim 1, characterized in that: The method for establishing a light source matching database is: performing mean calculation on the acquired light and dark stripe grayscale image of the rectangular light source area row by row to generate a light and dark stripe sequence V = {v1, v2, v3, ···, v i },v1,v2,v3,···,v i ∈[0,255], i is the row number of the gray light and dark stripe image; the light and dark stripe sequence V is subjected to signal autocorrelation calculation to obtain the signal autocorrelation sequence N={n1,n2,n3,···,n T ,···}, and calculate the period T of the autocorrelation sequence N; extract the first T sequence values ​​of the signal autocorrelation sequence N for normalization processing, and obtain the standardized sequence X={x1,x2,x3,···,x T },x1,x2,x3,···,x T ∈[0,1]; associate the autocorrelation sequence N as a matching feature with the ID information of the signal modulation to establish a light source matching database.

6. The method for visual recognition, matching and positioning of a flat panel light source based on visible light communication according to claim 1, characterized in that: When the CMOS type is the same, the period T is the same and the proximity K is the smallest, it is the best matching condition. The ID information of the light source that best matches the rectangular flat light source to be located in the light source matching database is obtained, and the world coordinates of the four corner points of the light source corresponding to the ID information are used as the coordinates of the light source corner points {P1, P2, P3, P4}.

7. The method for visual recognition, matching and positioning of a flat panel light source based on visible light communication according to claim 6, characterized in that: When determining the correspondence between the world coordinates of the corner points of the rectangular LED light source and the coordinates of its image point, it is assumed that a correspondence is used to calculate the position and posture of the CMOS image sensor based on the homography direct method, and the difference between the calculated heading angle and the geomagnetic direction angle of the IMU attitude sensor is compared. If the difference is less than the preset threshold, it is a correct relationship, otherwise the calculation is continued; there are 4 kinds of sequence correspondences of the 4 corner points, and the correspondence can be determined by calculating at most 4 times; according to the correct correspondence, the IMU attitude sensor-assisted iterative method is used to perform imaging positioning calculations.

8. The method for visual recognition, matching and positioning of a flat panel light source based on visible light communication according to claim 7, characterized in that: The preset threshold is a deviation angle C between the heading angle and the geomagnetic azimuth angle < 45°.

9. The method for visual recognition, matching and positioning of a flat panel light source based on visible light communication according to claim 7, characterized in that: The photogrammetry principle uses a visual plane four-point positioning method to calculate the position information of the camera of the CMOS image sensor relative to the rectangular flat light source to achieve visual positioning.

10. A positioning system according to any one of claims 1 to 9, characterized in that: The invention comprises a rectangular flat light source (1) and a positioning device, wherein the rectangular flat light source (1) is a visible light communication flat LED light source; the positioning device comprises a COMS imaging sensor (2) and an IMU attitude sensor (3), and both the COMS imaging sensor (2) and the IMU attitude sensor (3) are connected to a microprocessor (4).

11. The positioning system according to claim 10, characterized in that: The microprocessor (4) is connected to a battery (5), the microprocessor (4) and the battery (5) are both arranged in a housing (6), the COMS imaging sensor (2) is fixedly arranged on the housing (6), and the IMU attitude sensor (3) is arranged in the housing (6); the microprocessor (4) processes data collected by the IMU attitude sensor (3) and the COMS imaging sensor (2) to achieve visual recognition matching positioning.

Citation Information

Patent Citations

  • Mobile terminal camera visible light communication method and system compatible with multiple models

    CN106877929A

  • System and method for interactive image annotation

    CN103918008A

  • Indoor navigation method, device and system

    CN103940419A

  • System and method for guiding machine capable of autonomously moving

    CN110471402A

  • Full gloss register for easy reference plant light filling lamp of photovoltaic power supply

    CN205896711U