A visual benchmark method for visible light positioning

Abstract

The present invention proposes a visual reference method for visible light positioning, including: constructing a Manchester one-dimensional visual reference mechanism and generating a grayscale image with light and dark stripes; processing the grayscale image based on a visual reference extraction algorithm for image processing to obtain a barcode front view; analyzing and calculating the grayscale image and the barcode front view, detecting the barcode information load, and obtaining the 3D coordinates of the four corner points of the barcode in the world coordinate system; using a 3D-2D point pair matching method to perform point pair matching on the 3D coordinates, and then solving them through a PnP algorithm to achieve single-lamp positioning. In the implementation process of the visual reference method proposed in the present invention, there is no need to add an additional modulation module to control the flickering of the lamp, which reduces the complexity of the equipment, simplifies the image processing process for restoring the LED ID, improves the reliability of decoding and increases the refresh rate of positioning; at the same time, through the 3D-2D point pairs provided by the lamp, the mature PnP algorithm can be used to achieve 3D positioning of the camera in the case of a single lamp.

Description

Technical Field

[0001] The present invention is directed to the technical field of indoor visible light positioning, and in particular relates to a visual reference method for visible light positioning. Background Art

[0002] The imaging visible light positioning (iVLP) technology mainly includes two levels: ID detection and precise positioning. ID detection is used to distinguish different lamps, and then precise positioning is performed based on photogrammetry methods to obtain the position information of user equipment (UE).Usually, in iVLP systems, optical camera communication (OCC) is widely used to transmit and detect the LED-ID information of the light source. [1] Z. Li, M. Jiang, X. Zhang, X. Chen, and W. Hou, “Space-time-multiplexedmulti-image visible light positioning system exploiting pseudo-Miller-codingfor smart phones,” IEEE Trans. Wireless Commun., vol. 16, no. 12, pp. 8261–8274, Dec. 2017. [2] J. Fang, Z. Yang, S. Long, Z. Wu, X. Zhao, F. Liang, Z. Jiang, and Z. Chen, “High-speed indoor navigation system based on visible light and mobile phone,” IEEE Photon. J., vol. 9, no. 2, pp. 1–11, Apr. 2017. [3] H. Li, H. Huang, Y. Xu, Z. Wei, S. Yuan, P. Lin, H. Wu, W. Lei, J.Fang, and Z. Chen, “A fast and high-accuracy real-time visible light positioning system based on single LED lamp with a beacon,” IEEE Photon. J., vol. 12, no. 6, pp. 1–12, Dec. 2020.[4]W. Guan, L. Huang, B. Hussain, and C.Yue, “Robust robotic localization using visible light positioning andinertial fusion,” IEEE Sensors J., vol. 22, no. 6, pp. 4882–4892, Mar. 2022. However, ordinary lamps need to add a control module to have the OCC function, which means an increase in installation and maintenance costs.Specifically, the light source controlled by the On-Off Keying (OOK) modulation signal will flash rapidly at a frequency that is imperceptible to the human eye. In the image captured by the rolling shutter camera, the light-emitting surface of the light source will produce black and white stripes. In addition, it can be proved that the pixel width of the black and white stripes has no direct relationship with the distance and relative posture between the camera and the light source, but is only related to the flashing frequency of the light source and the camera parameters [1]. Therefore, when the camera parameters are known, the flashing frequency of the corresponding light source can be derived from the pixel width of the black and white stripes, and the LED-ID information can be restored. Reference [1] proposed an asynchronous mode LED-ID transmission scheme, which maps the LED-ID information bits into modulation symbols through pseudo-Mueller coding, and proved that there is a mapping uniqueness between the number of consecutive identical symbols and the number of consecutive image frames. References‎[5]R. Qv, L. Feng, A. Yang,P. Guo, B. Lin, and H. Huang, “A high efficient code for visible light positioning system based on image sensor,” IEEE Access, vol. 7, pp. 77762–77770, 2019. A LED-ID encoding method based on mask pattern is proposed to avoid the appearance of too long continuous ‘1’ and continuous ‘0’ in the encoded LED-ID. References‎[6]H. Zhang and C. Yang, “Efficient coding and detection of ultra-long ids for visible light positioning systems,” Opt. Express, vol. 26, no. 10, pp. 13397–13407, 2018. A LED-ID encoding scheme based on Reed-Solomon coding is proposed to compromise between encoding and decoding efficiency and complexity. When the number of LED-ID bits is long, multiple frames of images will be required to transmit an ID information, which will reduce the transmission rate of LED-ID and affect the positioning refresh rate of the iVLP system. In addition, in the literature [7], a scheme for transmitting LED-ID through LED square flat panel lights is proposed, but a modulation module is still needed to control the lamp to flash quickly to produce black and white stripes.

[0003] At the level of precise positioning algorithms, the shape characteristics of the light source and visual measurement methods are usually used to locate the UE. There are two main types of iVLP solutions based on the shape of the lamps used, namely, iVLP solutions based on round lamps and iVLP solutions based on square lamps. On the other hand, traditional monocular vision positioning algorithms can be migrated to iVLP systems for camera pose estimation, such as the classic perspective n point (PnP) algorithm. [8] V. Lepetit, F. Moreno-Noguer, and P. Fua, “Epnp: An accurate o(n)solution to the pnpproblem,” Int. J. Comput. Vis., vol. 81, no. 2, pp. 155–166, 2009. [9] Y. Li, S. Fan, Y. Sun, W. Qiang, and S. Sun, “Bundle adjustment method using sparseBFGS solution,” Remote Sens. Lett., vol. 9, no. 3, pp. 789–798, 2018.

[10] C. Xu, L. Zhang, L. Cheng, and R. Koch, “Pose estimation from line correspondences: a complete analysis and aseries of solutions,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no.6, pp. 1209–1222, Jun. 2017, and it is usually assumed that point pairs and line pairs have been matched. References ‎

[11] R. Yuanand W. Hui, “3D information acquisition from rectangles and circles in single images,” in Proc. WRI World Congr. Comput. Sci. Inf. Eng., Los Angeles, USA, Mar. 2009, pp. 43–47. proposed to use the four corner points of a square reference object for pose estimation, which requires the 3D-2D point pairs to be matched as a prerequisite.Reference

[12] Bai, Lin. "Novel Visible Light Communication Assisted Perspective-Four-Line Algorithm for Indoor Localization." IEEE (2020). A perspective four-line positioning algorithm using square flat panel lights is proposed. Since the line pair matching relationship cannot be known, it is necessary to calculate the corresponding solutions of all matching situations and obtain the estimated solution through clustering. Reference [3] proposes a single-lamp positioning algorithm using circular lamps, but it is still necessary to add additional point light sources at the edge of the lamp, and then perform 2D positioning through 2 pairs of paired points. Therefore, how to select appropriate shape features or visual references for pose estimation in iVLP systems has not been fully studied. Summary of the invention

[0004] In order to overcome the shortcomings of the existing technology that transmits LED-ID through OCC, which brings high equipment complexity and requires the use of multiple lamps for precise positioning, a visual benchmark method for visible light positioning is provided, a one-dimensional visual benchmark construction method based on Manchester coding is designed, and a visual benchmark method based on image processing is designed, which uses the PnP algorithm to realize the solution of camera pose in the single-lamp case.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] A visual reference method for visible light positioning comprises the following steps:

[0007] S1: Constructing the Manchester one-dimensional visual benchmark mechanism;

[0008] S2: Generates grayscale images with light and dark stripes based on Manchester one-dimensional visual reference mechanism;

[0009] S3: Process the grayscale image based on the visual reference extraction algorithm of image processing to obtain the front view of the barcode;

[0010] S4: Analyze and calculate the grayscale image and the barcode front view, detect the barcode information load, and obtain the 3D coordinates of the four corner points of the barcode in the world coordinate system;

[0011] S5: After point-pair matching of 3D coordinates using the 3D-2D point pair matching method, the PnP algorithm is used to solve the problem and achieve single lamp positioning.

[0012] The Manchester one-dimensional visual reference mechanism proposed in step S1 specifically generates a one-dimensional barcode based on Manchester encoding and parity check mechanism; the Manchester one-dimensional visual reference mechanism is specifically:

[0013] Assume that the original LED-ID information contains bit, after adding parity check and Manchester encoding bit, Indicates the bit sequence after adding parity check; Pre-continuous bit'1' and 1bit'0', at the same time The last bit is set to '1', which forms a complete LED-ID information frame. The continuous '1' bit is used as the LED-ID frame header FH, indicating the starting position of a frame; the post-positioned 1 bit '1' indicates the frame tail FT, indicating the end position of a frame; to avoid FH conflict, it is necessary to meet ,by Represents the Manchester-encoded bit sequence; with width The light and dark stripes represent the LED-ID frame binary sequence, where bit '1' is represented by a dark stripe and bit '0' is represented by a light stripe.

[0014] Wherein, in step S2, the pixel width of the light and dark stripes of the grayscale image depends on the physical width of the stripes on the light-emitting surface of the lamp itself and the distance and posture of the camera relative to the lamp.

[0015] Wherein, the step S3 specifically includes the following steps:

[0016] S31: Binarizing the grayscale image, and extracting a quadrilateral connected domain in the binary image by a closed contour search algorithm;

[0017] S32: Setting a filtering threshold to filter out the subtle contours in the quadrilateral connected domain that are affected by noise in the non-barcode area detection, and obtaining the remaining quadrilateral contours;

[0018] S33: Calculate the coordinates of four corner points of the quadrilateral outline, and reorder the obtained four corner point coordinates to obtain four rough corner point coordinates;

[0019] S34: Optimize the four rough corner point coordinates through the classic sub-pixel corner point detection algorithm to obtain four sub-pixel corner points;

[0020] S35: adjusting the quadrilateral contour image in the grayscale image to a rectangular region image, and obtaining a projection matrix expression corresponding to the image;

[0021] S36: Solve the projection matrix through direct linear transformation, and then obtain the barcode front view through projection transformation.

[0022] Wherein, in the step S32, the remaining quadrilateral outline is recorded as , quadrilateral outline Existence Continuous points, represented by ,in ;

[0023] In step S33, the quadrilateral outline The four corner points of , ; Quadrilateral outline The first coordinate of , while Of the remaining points, The point with the largest distance must be one of the corner points. Without loss of generality, let the corner point be , The solution formula is:

[0024] (1)

[0025] Get the first corner point After that, by searching with The contour point with the largest distance is taken as the second corner point , so we have:

[0026] (2)

[0027] at this time and A quadrilateral outline Two corner points at a certain diagonal position; further, by traversing the remaining contour points, search for , The contour point with the largest sum of distances is taken as the third corner point , at this time:

[0028] (3)

[0029] Similarly, the fourth corner point Obtained by the following formula:

[0030] (4)

[0031] At this point, the calculation of the coordinates of the four corner points of the quadrilateral outline is completed.

[0032] Among them, in the step S33, the calculated These are unordered corner points, which need to be reordered for subsequent processing, specifically:

[0033] Will Arrange in counterclockwise order, and use the corner point of the short side of the barcode as the starting point of the arrangement, and record the sorted corner point as ; Due to the uneven brightness of the light emitting surface of the lamp, the camera imaging quality and the binary processing, These are just rough corner points. So far, we have obtained four rough corner point coordinates.

[0034] In step S34, in order to ensure the subsequent projection transformation and accurate positioning effect, the four rough corner point coordinates are optimized by the classic sub-pixel corner point detection algorithm. Iterate the search in a certain neighborhood to get the sub-pixel corner point ;

[0035] In step S35, the quadrilateral contour image in the grayscale image is adjusted to a width of pixels and a height of A rectangular region image of pixels, where is the scale factor, which is selected reasonably according to the number of stripes and image size Make each stripe in the front view occupy a certain pixel width; is the pixel width of the grayscale image; is the pixel height of the grayscale image, we have:

[0036]

[0037] The projection matrix corresponding to the image is specifically expressed as:

[0038] (5)

[0039] in Respectively represent the projected corner points The image coordinates of Represents the original corner point The image coordinates of Represents the scale factor for the homogeneous equation.

[0040] Among them, in the step S36, the element is limited , then H is transformed by direct linear transformation The unique solution is obtained, and then the front view of the barcode is obtained through projection transformation.

[0041] Wherein, the step S4 specifically includes the following steps:

[0042] S41: Barcode front view Continuously perform Sobel edge extraction and binarization operations to obtain a binary image , ; At this time, the binary image Non-zero elements are mainly distributed in The corresponding light and dark stripes are alternate areas, so the binary image is counted by column The number of non-zero elements in is denoted by ,in ,but It is expressed as:

[0043] (6)

[0044] Corresponding to the binary image At the column index of the distribution of non-zero elements in the statistics array will produce a maximum value; There will be glitch noise, making the statistical array Pseudo peaks appear, so it is necessary to set the threshold reasonably. Smaller outliers are filtered out;

[0045] S42: Further, due to At the peak of The two consecutive peaks in the figure are the left and right edges of the same stripe, so the index difference of the two consecutive peaks is the index of the corresponding stripe at The pixel width in; introduce the non-maximum suppression algorithm to extract the array The index value of the maximum value in the , the obtained index array is recorded as ,in ,and Indicates the number of maximum values; for index array Perform a difference operation, and the difference array obtained is the pixel width of each stripe. The difference array is recorded as , so we have:

[0046] (7)

[0047] According to the barcode construction rules, FH contains the widest dark stripe, so the difference array The largest value among them is the pixel width of the dark stripe in FH, recorded as ; The widest dark stripe is composed of continuous The width of a single stripe is By comparing The values ​​of each element in the barcode are used to obtain the distribution of stripes. Since the LED-ID information is represented by the stripe distribution law, the corresponding LED-ID is deduced according to the stripe distribution. Finally, the original LED-ID information is obtained through parity check and Manchester decoding, and then the 3D coordinates of the four corner points of the barcode in the world coordinate system are obtained, that is, .

[0048] Wherein, the step S5 is specifically as follows:

[0049] Since the position of FH in the quadrilateral area is not determined, the ordered corner points There is no guarantee that Sequential matching, so further correction of corner points is required The order of arrangement is Indicates the correctly matched corner point; assuming that FH is located in the image On the left side, there is , , , ; Since the position of FH is unknown before the projection transformation, FH may be located in the image , there should be , , , ; Since the stripe pixel width sequence has been obtained , and the black stripes in FH occupy the maximum pixel width, so according to The index of the largest element in FH determines If the maximum element index value Greater than It is believed that FH and vice versa, so we have

[0050] (8)

[0051] The final matched 3D-2D point pair is ; According to the camera imaging model, there is the following relationship:

[0052] (9)

[0053] in for Camera intrinsic parameter matrix, express The rotation matrix, express After completing LED-ID decoding and 3D-2D point pair matching, we know and , and the internal parameter matrix Can be obtained through camera calibration and then solved by PnP algorithm , , to achieve single lamp positioning.

[0054] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0055] The present invention proposes a visual reference method for visible light positioning, which represents LED-ID information by black and white stripes distributed on the light-emitting surface of the lamp, without adding an additional modulation module to control the flickering of the lamp, thereby reducing the complexity of the equipment; the streamlined coding design is conducive to simplifying the image processing process for recovering the LED-ID, which can improve the reliability of decoding and increase the refresh rate of positioning; a complete square lamp can provide 4 pairs of 3D-2D point pairs, and with the help of a mature PnP algorithm, 3D positioning of the camera can be achieved in the case of a single lamp. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram of the flow of the visual reference method for visible light positioning according to the present invention;

[0057] Figure 2 A schematic diagram of a visual reference mechanism design rule in one embodiment of the present invention;

[0058] Figure 3 This is an example diagram of visual reference positioning in one embodiment of the present invention;

[0059] Figure 4 A diagram of a barcode information payload extraction process in one embodiment of the present invention;

[0060] Figure 5 This is an example diagram of a visual reference structure in one embodiment of the present invention;

[0061] Figure 6 A PE distribution diagram using a PnP algorithm in one embodiment of the present invention;

[0062] Figure 7 This is a PE distribution diagram using the BA algorithm in one embodiment of the present invention;

[0063] Figure 8 This is a comparison diagram of PE distribution CDF under the PnP algorithm and the BA algorithm in one embodiment of the present invention. DETAILED DESCRIPTION

[0064] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0065] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0066] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0067] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0068] Example 1

[0069] like Figure 1 As shown, this embodiment provides a visual reference method for visible light positioning, comprising the following steps:

[0070] S1: Constructing the Manchester one-dimensional visual benchmark mechanism;

[0071] S2: Generates grayscale images with light and dark stripes based on Manchester one-dimensional visual reference mechanism;

[0072] S3: Process the grayscale image based on the visual reference extraction algorithm of image processing to obtain the front view of the barcode;

[0073] S4: Analyze and calculate the grayscale image and the barcode front view, detect the barcode information load, and obtain the 3D coordinates of the four corner points of the barcode in the world coordinate system;

[0074] S5: After point-pair matching of 3D coordinates using the 3D-2D point pair matching method, the PnP algorithm is used to solve the problem and achieve single lamp positioning.

[0075] In the specific implementation process of this embodiment, the LED-ID information is represented by black and white stripes distributed on the light-emitting surface of the lamp, without adding an additional modulation module to control the flashing of the lamp, thereby reducing the complexity of the equipment; the streamlined encoding design is conducive to simplifying the image processing process for recovering the LED-ID, which can improve the reliability of decoding and increase the refresh rate of positioning; a complete square lamp can provide 4 pairs of 3D-2D point pairs, and with the help of the mature PnP algorithm, the 3D positioning of the camera can be achieved in the case of a single lamp.

[0076] Example 2

[0077] More specifically, based on Example 1, a single-lamp visible light imaging positioning method based on Manchester-encoded one-dimensional visual reference is specifically proposed. A one-dimensional visual reference construction method based on Manchester encoding is designed, and a visual reference extraction algorithm based on image processing, an information load detection algorithm, and a 3D-2D point pairing method are designed, so that the PnP algorithm can be used to solve the camera pose in the single-lamp case. This embodiment specifically includes the following processes:

[0078] 1. Manchester one-dimensional visual benchmark construction mechanism

[0079] One-dimensional barcodes are widely used in product numbering, which use black and white stripes arranged in a certain pattern to represent numbering information. Based on the design concept of the widely used cross 2-5 code, this embodiment proposes a one-dimensional barcode based on Manchester encoding and parity check mechanism, which is further streamlined compared to the cross 2-5 code, thereby simplifying the process of recovering the barcode information payload and having a basic error detection function.

[0080] More specifically, assuming the original LED-ID information bits contain bit, after adding parity check and Manchester encoding bit, Indicates the bit sequence after adding parity check. bit '1' and then set the next bit '1' to form a complete LED-ID information frame. The continuous '1' bit is used as the LED-ID frame header (Frame Header, FH), indicating the starting position of a frame; the post-positioned 1 bit '1' indicates the frame tail (Frame Tail, FT), indicating the end position of a frame. To avoid FH conflict, the following conditions must be met: ,by Represents the Manchester-encoded bit sequence. The light and dark stripes represent the LED-ID frame binary sequence, where bit '1' is represented by a dark stripe and bit '0' is represented by a light stripe. The design rules of the stripe code are as follows: Figure 2 shown.

[0081] In the specific implementation process, dark stripes can be represented by pasting semi-transparent materials on the luminous surface, while the area corresponding to the bright stripes can be pasted with translucent materials or remain unchanged, so it will not have too much impact on the lighting function of the lamp. While the receiving end recovers the LED-ID information through image processing, it can obtain the 3D coordinates of the four corner points of the flat panel light in the world coordinate system (World Coordinate System, WCS) and their 2D coordinates in the image coordinate system (Image Coordinate System, PCS). After further establishing the 3D-2D point pair matching relationship, the coordinates of the camera in the WCS can be solved by the PnP algorithm, such as Figure 3 shown.

[0082] 2. Quadrilateral region extraction algorithm

[0083] More specifically, compared to the traditional iVLP system that uses OCC and rolling shutter cameras to produce black and white stripes in the image, since the light and dark stripes in this embodiment do not require the lamp to flash quickly, the shutter mode of the camera can be a rolling shutter or a global shutter. Before extracting the LED-ID information payload, it is necessary to extract the quadrilateral area occupied by the barcode from the image. In order to obtain a high-contrast stripe image, the exposure time of the camera needs to be adjusted to a lower value. At this time, the grayscale value of the corresponding luminous surface area in the image will be larger than the grayscale value of the background environment, which is conducive to threshold segmentation. The pixel width and height of the image are respectively recorded as , ,by Represents two-dimensional data of a grayscale image, where . In addition, in the related schemes based on the OCC mechanism, the pixel width of the stripes is independent of the distance and posture of the camera relative to the light source; while the stripes in this embodiment are not generated by the rolling shutter effect, and the pixel width of the stripes depends on the physical width of the stripes on the light-emitting surface of the lamp itself and the distance and posture of the camera relative to the lamp. Therefore, before detecting the barcode information load, it is necessary to extract the irregular quadrilateral barcode area in the image and adjust it to a front view. In other words, the pixel width of the stripes in the barcode that has not been adjusted is uneven, while the pixel width of the barcode in the adjusted front view will be uniform. The specific steps are:

[0084] Step 1: First, the grayscale image is binarized and the quadrilateral connected domain in the binary image is extracted by the closed contour search algorithm

[13] S. Suzuki, “Topological structural analysis of digitized binary images by borderfollowing,” Comput. Vis., Graph., Image Process., vol. 30, pp. 32–46, 1985. Under the influence of noise, subtle contours will also be detected in the non-barcode area. The threshold can be reasonably set to filter out contours with too few points. The remaining contours after filtering are recorded as . Assume that there is Continuous points, represented by ,in ;

[0085] Step 2: Outline the Quad The four corner points of ( ).contour The first coordinate of , while Of the remaining points, The point with the largest distance must be one of the corner points. Without loss of generality, the corner point can be , which is equivalent to It can be solved by the following formula

[0086] (1)

[0087] Get the first corner point After that, you can search and The contour point with the largest distance is taken as the second corner point , so there is

[0088] (2)

[0089] at this time and A quadrilateral outline Two corner points at a certain diagonal position. Furthermore, the corner points By traversing the remaining contour points, we can search for , The contour point with the largest sum of distances is taken as the third corner point , at this time there is

[0090] (3)

[0091] Similarly, the fourth corner point It can be obtained by the following formula

[0092] (4)

[0093] It should be pointed out that are unordered corner points. To facilitate subsequent processing, Arrange in counterclockwise order, and use the corner point of the short side of the barcode as the starting point of the arrangement, and record the sorted corner point as In addition, due to the uneven brightness of the lamp surface, the camera imaging quality and the binary processing, It is only a rough corner point. In order to ensure the subsequent projection transformation and accurate positioning effect, the image coordinates of the extracted corner points are further optimized through the classic sub-pixel corner point detection algorithm

[14] A. Kaehler and G. Bradski, Learning openCV3: computer vision in C++ with the openCV library. O'Reilly Media, Inc, 2016. Iterate the search in a certain neighborhood to get the sub-pixel corner point .

[0094] Step 3: To adjust the barcode to a front view, the corresponding homography matrix needs to be calculated. In this paper, the irregular quadrilateral area in the original grayscale image is adjusted to a width of pixels and a height of A rectangular region of pixels, where is the scale factor, which is selected reasonably according to the number of stripes and image size So that each stripe in the front view occupies a certain pixel width. The corresponding projection matrix can be expressed as

[15] R. Hartley and A. Zisserman, Multiple view geometry in computer vision, 2nd ed. Cambridge University Press, 2004:

[0095] (5)

[0096] in Respectively represent the projected corner points The image coordinates of Represents the original corner point The image coordinates of Represents the scale factor of the homogeneous equation. , at this time H can be transformed by direct linear transformation (Direct Linear Transform, DLT) The only solution is obtained. Then the barcode front view can be obtained through projection transformation. Indicates that, among them, Figure 4 shown.

[0097] 3. Barcode information load detection

[0098] Step 1: Grayscale image Continuously perform Sobel edge extraction and binarization operations to obtain a binary image ( ).from Figure 4 It can be seen that the barcode front view There are light and dark stripes distributed horizontally in the image, so the Sobel operation can extract the edge of the alternating light and dark stripes, and the horizontal gradient value at the edge will be larger. Further binary operation can extract the edge of the stripes, so the binary image Non-zero elements are mainly distributed in Based on this analysis, the binary image can be counted by column The number of non-zero elements in is denoted by ,in ,but It can be expressed as

[0099] (6)

[0100] Corresponding to the binary image At the column index of the distribution of non-zero elements in the statistics array will produce a maximum value. Glitch noise may occur, making the statistical array Pseudo peaks appear, so it is necessary to set the threshold reasonably. Smaller outliers are filtered out.

[0101] Step 2: Further analysis, due to At the peak of The two consecutive peaks in the figure are the left and right edges of the same stripe, so the index difference of the two consecutive peaks is the index of the corresponding stripe at The pixel width in. Introducing the non-maximum suppression (NMS) algorithm

[16] Neubeck, Alexander, and L. JV Gool. "Efficient Non-Maximum Suppression." International Conference on Pattern Recognition IEEE Computer Society, 2006. Extract array The index value of the maximum value in the , the obtained index array is recorded as ,in ,and Indicates the number of maximum values. Perform a difference operation, and the difference array obtained is the pixel width of each stripe. The difference array is recorded as , so there is

[0102] (7)

[0103] According to the barcode construction rules, FH contains the widest dark stripe, so the difference array The largest value among them is the pixel width of the dark stripe in FH, recorded as The widest dark stripe is composed of continuous The width of a single stripe is By comparing The values ​​of each element in can be used to obtain the distribution of stripes. Since the LED-ID information is represented by the stripe distribution law, the corresponding LED-ID can be deduced based on the stripe distribution. Finally, after parity check and Manchester decoding, the original LED-ID information can be obtained, and then the 3D coordinates of the four corner points of the barcode (visual reference) in the world coordinate system can be obtained, that is, .

[0104] 4.3D-2D Point Pair Matching

[0105] Since the position of FH in the quadrilateral area is not determined, the ordered corner points There is no guarantee that Sequential matching, so further correction of corner points is required The order of arrangement is Indicates the correctly matched corner points. Figure 3 In the image, FH is located On the left side, there is , , , It is worth noting that since the position of FH is unknown before the projection transformation, FH may be located in the image , there should be , , , Since the stripe pixel width sequence has been obtained , and the black stripes in FH occupy the maximum pixel width, so it can be based on The index of the largest element in FH determines If the maximum element index value Greater than It is believed that FH and vice versa, so we have

[0106] (8)

[0107] The final matched 3D-2D point pair is According to the camera imaging model

[15] , the following relationship holds

[0108] (9)

[0109] in for Camera intrinsic parameter matrix, express The rotation matrix, express After completing LED-ID decoding and 3D-2D point pair matching, we can get and , and the internal parameter matrix It can be obtained through camera calibration and then solved by the PnP algorithm [8] [9] , , to achieve single lamp positioning.

[0110] In the specific implementation process, the solution provided in this embodiment represents the LED-ID information through black and white stripes distributed on the light-emitting surface of the lamp, without adding an additional modulation module to control the flashing of the lamp, thereby reducing the complexity of the equipment; its streamlined encoding design is conducive to simplifying the image processing process for recovering the LED-ID, which can improve the reliability of decoding and increase the refresh rate of positioning; and in this embodiment, a complete square lamp can provide 4 pairs of 3D-2D point pairs, and with the help of the mature PnP algorithm, the 3D positioning of the camera can be achieved in the case of a single lamp.

[0111] Example 3

[0112] In order to more fully illustrate the beneficial effects of the present invention, the effectiveness and advancement of the present solution are further illustrated below in combination with analysis of specific embodiments and simulation results of related parties.

[0113] A. Example of Manchester 1D Vision Benchmark Construction

[0114] Taking the 12-bit LED-ID source information bits as an example, if the information sequence is "{000011010001}", the sequence after odd parity check is "{0000110100011}". According to the visual reference construction rule, the sequence "{0000110100011}" is represented by black and white stripes, and then FH and FT are added to obtain the following: Figure 5 The barcode shown, where .

[0115] B. Positioning Performance Evaluation

[0116] This example will evaluate the performance of the proposed visual benchmark solution based on experimental results. The experimental environment is set as The three-dimensional space, in A test point was taken every 15 cm in the plane area, with a total of 169 test points. Unless otherwise specified, the experimental parameters are shown in Table 1.

[0117] Table 1 Positioning accuracy test system parameters

[0118]

[0119] After the images collected at 169 test points are decoded and matched with point pairs, Figure 6 and Figure 7 The 3D positioning result distributions of the two algorithms are given respectively. Figure 8The distribution diagram of the cumulative probability density function of positioning error (PE) corresponding to the two algorithms is given in Figure 2. The 3D WCS coordinates required for solving the projection equation are obtained after barcode information payload detection, while the 2D ICS coordinates are given by the corner point extraction algorithm. Finally, 3D-2D matching point pairs are established according to the position of FH. The algorithms used to solve the camera pose through matching point pairs are the EPnP algorithm [8] and the Bundle Adjustment (BA) algorithm [9]. Among them, the average PE of the BA algorithm in 2D and 3D are 2.25 cm and 2.44 cm respectively, and the corresponding values ​​of the EPnP algorithm are 3.85 cm and 4.10 cm respectively.

[0120] It can be observed that the BA algorithm provides better positioning accuracy than the EPnP algorithm. This is because the EPnP algorithm solves the camera pose in a closed form, and its computational complexity is low, but it may not be possible to obtain a better solution through a closed expression in one calculation; while the pose estimation of the BA algorithm is an iterative optimization process, which first needs to calculate the reprojection error and its Jacobian matrix relative to the pose change, and then optimize the camera pose through iterative algorithms such as Gauss-Newton and LM to minimize the reprojection error. Its computational complexity is high, but the probability of obtaining a better solution is also greater, so it shows better positioning performance.

[0121] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A visual reference method for visible light positioning, characterized in that: The following steps are involved: S1: Constructing the Manchester one-dimensional visual benchmark mechanism; The Manchester one-dimensional visual reference mechanism is used to generate a one-dimensional barcode based on Manchester encoding and parity check mechanism, and the steps include: Assume that the original LED-ID information contains bit, after adding parity check and Manchester encoding bit, Indicates the bit sequence after adding parity check; Pre-continuous bit'1' and 1bit'0', at the same time The last bit is set to '1', which forms a complete LED-ID information frame. Continuous '1' bits and 1 bit '0' together serve as the LED-ID frame header FH, indicating the start position of a frame; the post-positioned 1 bit '1' represents the frame tail FT, indicating the end position of a frame; to avoid FH conflict, the following conditions must be met ,by Represents the Manchester-encoded bit sequence; with width The light and dark stripes represent the LED-ID frame binary sequence, where bit '1' is represented by a dark stripe and bit '0' is represented by a light stripe; S2: Generates grayscale images with light and dark stripes based on Manchester one-dimensional visual reference mechanism; S3: Process the grayscale image based on the visual reference extraction algorithm of image processing to obtain the front view of the barcode; S4: Analyze and calculate the grayscale image and the barcode front view, detect the barcode information load, and obtain the 3D coordinates of the four corner points of the barcode in the world coordinate system; S5: After point-pair matching of 3D coordinates is performed using a 3D-2D point-pair matching method, a solution is obtained using a PnP algorithm to achieve single-lamp positioning. The steps include: Correct the four corners of the barcode The order of arrangement is Indicates the correctly matched corner point; if FH is located in the front view of the barcode On the left side, there is , , , ; If FH is located in the image On the right side, there is , , , ; According to the stripe pixel width sequence The index of the largest element in determines the FH in the image If the maximum element index value Greater than It is believed that FH Otherwise, it is considered that FH is on The left side, that is: in M For the front view of the barcode Binarized image obtained after continuous Sobel edge extraction and binarization operations The number of non-zero elements in the array The number of maximum values ​​in ; The final matched 3D-2D point pair is ; According to the camera imaging model, there is the following relationship: in for Camera intrinsic parameter matrix, express The rotation matrix, express After completing LED-ID decoding and 3D-2D point pair matching, we know and , and the internal parameter matrix Can be obtained through camera calibration and then solved by PnP algorithm , , to achieve single lamp positioning.

2. A visual reference method for visible light positioning according to claim 1, characterized in that: In step S2, the pixel width of the light and dark stripes in the grayscale image depends on the physical width of the stripes on the light-emitting surface of the lamp itself and the distance and posture of the camera relative to the lamp.

3. A visual reference method for visible light positioning according to claim 2, characterized in that: The step S3 specifically comprises the following steps: S31: Binarizing the grayscale image, and extracting a quadrilateral connected domain in the binary image by a closed contour search algorithm; S32: Setting a filtering threshold to filter out the subtle contours in the quadrilateral connected domain that are affected by noise in the non-barcode area detection, and obtaining the remaining quadrilateral contours; S33: Calculate the coordinates of four corner points of the quadrilateral outline, and reorder the obtained four corner point coordinates to obtain four rough corner point coordinates; S34: Optimize the four rough corner point coordinates through the classic sub-pixel corner point detection algorithm to obtain four sub-pixel corner points; S35: adjusting the quadrilateral contour image in the grayscale image to a rectangular region image, and obtaining a projection matrix expression corresponding to the image; S36: Solve the projection matrix through direct linear transformation, and then obtain the barcode front view through projection transformation.

4. A visual reference method for visible light positioning according to claim 3, characterized in that: In step S32, the remaining quadrilateral outline is recorded as , quadrilateral outline Existence Continuous points, represented by ,in ; In step S33, the quadrilateral outline The four corner points of , ; Quadrilateral outline The first coordinate of , while Of the remaining points, The point with the largest distance must be one of the corner points. Let the corner point be , The solution formula is: Get the first corner point After that, by searching with The contour point with the largest distance is taken as the second corner point , so we have: at this time and A quadrilateral outline Two corner points at a certain diagonal position; further, by traversing the remaining contour points, search for , The contour point with the largest sum of distances is taken as the third corner point , at this time: The 4th corner point Obtained by the following formula: At this point, the calculation of the coordinates of the four corner points of the quadrilateral outline is completed.

5. A visual reference method for visible light positioning according to claim 4, characterized in that: In step S33, the calculated These are unordered corner points, which need to be reordered for subsequent processing, specifically: Will Arrange in counterclockwise order, and use the corner point of the short side of the barcode as the starting point of the arrangement, and record the sorted corner point as ; Due to the uneven brightness of the light emitting surface of the lamp, the camera imaging quality and the binary processing, These are just rough corner points. So far, we have obtained four rough corner point coordinates.

6. A visual reference method for visible light positioning according to claim 5, characterized in that: In step S34, in order to ensure the subsequent projection transformation and accurate positioning effect, the four rough corner point coordinates are optimized by the classic sub-pixel corner point detection algorithm. Iterate the search in a certain neighborhood to get the sub-pixel corner point ; In step S35, the quadrilateral contour image in the grayscale image is adjusted to a width of pixels and a height of A rectangular region image of pixels, where is the scale factor, which is selected reasonably according to the number of stripes and image size Make each stripe in the front view occupy a certain pixel width; is the pixel width of the grayscale image; is the pixel height of the grayscale image, we have: The projection matrix corresponding to the image is specifically expressed as: in Respectively represent the projected corner points The image coordinates of Represents the original corner point The image coordinates of Represents the scale factor for the homogeneous equation.

7. A visual reference method for visible light positioning according to claim 6, characterized in that: In step S36, the element , then H is transformed by direct linear transformation The unique solution is obtained, and then the front view of the barcode is obtained through projection transformation.

8. A visual reference method for visible light positioning according to claim 7, characterized in that: The step S4 specifically comprises the following steps: S41: Barcode front view Continuously perform Sobel edge extraction and binarization operations to obtain a binary image , ; At this time, the binary image The non-zero elements are distributed in The corresponding light and dark stripes are alternate areas, so the binary image is counted by column The number of non-zero elements in is denoted by ,in ,but It is expressed as: Corresponding to the binary image At the column index of the distribution of non-zero elements in the statistics array will produce a maximum value; There will be burr noise, making the statistical array Pseudo peaks appear, so it is necessary to set the threshold reasonably. Smaller outliers are filtered out; S42: Further, due to There are alternating light and dark fringes at the peak of The two consecutive peaks in the figure are the left and right edges of the same stripe, so the index difference of the two consecutive peaks is the index of the corresponding stripe at The pixel width in; introduce the non-maximum suppression algorithm to extract the array The index value of the maximum value in the , the obtained index array is recorded as ,in ,and Indicates the number of maximum values; for index array Perform a difference operation, and the difference array obtained is the pixel width of each stripe. The difference array is recorded as , so we have: According to the barcode construction rules, FH contains the widest dark stripe, so the difference array The largest value among them is the pixel width of the dark stripe in FH, recorded as ; The widest dark stripe is composed of continuous The width of a single stripe is By comparing The values ​​of each element in the barcode are used to obtain the distribution of stripes. Since the LED-ID information is represented by the stripe distribution law, the corresponding LED-ID is deduced according to the stripe distribution. Finally, the original LED-ID information is obtained through parity check and Manchester decoding, and then the 3D coordinates of the four corner points of the barcode in the world coordinate system are obtained, that is, .

Images