Visible light positioning method, device, electronic device and storage medium

Abstract

The present application discloses a visible light positioning method, device, electronic device, and storage medium. The method includes: obtaining an original image captured by a mobile terminal; obtaining angle information of the mobile terminal; demodulating the ID information of an LED light using a stripe decoding algorithm based on the original image; obtaining the position and posture information of the LED light based on the ID information; performing coordinate transformation on the original image based on the position and posture information and the angle information to obtain a parallel reconstructed image; obtaining the center coordinates and imaging radius of the LED light in the parallel reconstructed image; based on the center coordinates and imaging radius, obtaining the target distance from the center of the LED light to the center of the camera lens of the mobile terminal according to the imaging principle and the proportional properties of the side lengths of similar triangles; obtaining the position coordinates of the mobile terminal through coordinate transformation based on the position and posture information, the center coordinates, and the target distance. The embodiments of the present application are efficient and convenient in achieving visible light positioning and can be widely used in the field of data processing technology.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a visible light positioning method, device, electronic device and storage medium. Background Art

[0002] The Global Positioning System (GPS) is a mature and widely used outdoor positioning solution. However, due to severe GPS signal attenuation caused by obstructions from buildings, GPS positioning is not ideal in indoor environments. To provide reliable positioning services indoors, researchers have developed a variety of indoor positioning technologies, such as WiFi, Bluetooth, ultra-wideband, and ZigBee. However, these traditional indoor positioning technologies suffer from high deployment costs and low positioning accuracy, making them unsuitable for widespread indoor positioning applications.

[0003] With the growing demand for indoor positioning, visible light positioning technology, with its numerous advantages such as being free of radio frequency interference, environmentally friendly and safe, and having low deployment costs, has gradually attracted attention. It can achieve high-precision positioning in indoor scenarios. Currently, visible light positioning systems use light-emitting diodes (LEDs) as transmitters and photodetectors as receivers to achieve visible light positioning. However, due to the specialized equipment required, they still have many limitations. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to propose a visible light positioning method, device, electronic device and storage medium, in order to solve at least one problem of the prior art. The present application can efficiently perform visible light positioning.

[0005] To achieve the above objectives, one aspect of an embodiment of the present application provides a visible light positioning method, the method comprising:

[0006] Acquire an original image captured by a mobile terminal; the original image includes an imaging pattern of at least one LED lamp with a circular lampshade installed; the LED lamp is at an angle to a horizontal plane;

[0007] Obtaining angle information of the mobile terminal based on a tilt sensor of the mobile terminal;

[0008] According to the original image, the ID information of the LED light is demodulated through the stripe decoding algorithm; based on the ID information, the position and posture information of the LED light is obtained from the database;

[0009] Based on the position and attitude information and the angle information, the coordinate transformation of the original image is performed to obtain a parallel reconstructed image in which the plane of the mobile terminal is relatively parallel to the plane of the LED light;

[0010] Obtain the central coordinates of the imaging center of the LED light in the image plane coordinate system and the imaging radius of the LED light in the parallel reconstructed image;

[0011] Based on the center coordinates and imaging radius, the target distance from the center of the LED light to the center of the camera lens of the mobile terminal is obtained according to the imaging principle and the proportional property of the side lengths of similar triangles;

[0012] The position coordinates of the mobile terminal are obtained through coordinate transformation based on the position and posture information, center coordinates and target distance.

[0013] In some embodiments, obtaining an original image captured by a mobile terminal includes:

[0014] The mobile terminal captures an image containing LED lights to obtain a color image;

[0015] The color image is gray-scale transformed using a gray-scale transformation algorithm to obtain the original image.

[0016] In some embodiments, different LED lights are set with different light signal frequencies; based on the original image, the ID information of the LED lights is demodulated using a fringe decoding algorithm, including:

[0017] According to the original image, the light signal frequency of the LED light is demodulated through the fringe decoding algorithm;

[0018] The ID information of the LED lamp is determined based on the optical signal frequency matching.

[0019] In some embodiments, the position and attitude information includes a set of attitude characteristic parameters of the LED light; the angle information includes a roll angle and a pitch angle; based on the position and attitude information and the angle information, coordinate transformation is performed on the original image to obtain a parallel reconstructed image in which the plane of the mobile terminal is relatively parallel to the plane of the LED light, including:

[0020] Based on the roll angle and pitch angle, the first position relationship between any point in the original image and the corresponding point of the LED light is obtained through the coordinate transformation principle;

[0021] Based on the posture feature parameter set, a second position relationship between any point in the parallel reconstructed image to be reconstructed and the corresponding point of the LED light is obtained through coordinate transformation principle;

[0022] According to the first position relationship and the second position relationship, a first transformation relationship is obtained for converting any point in the LED light into a parallel reconstructed image;

[0023] Based on the first transformation relationship, the coordinate transformation of the original image is performed in combination with a preset azimuth angle to obtain an alternative reconstructed image; the preset azimuth angle is traversed based on a preset step size;

[0024] When the major axis and the minor axis of the imaging of the LED light in the candidate reconstructed image are equal, determining that the candidate reconstructed image is a parallel reconstructed image in which the plane of the mobile terminal is relatively parallel to the plane of the LED light;

[0025] When the plane of the mobile terminal is not parallel to the plane of the LED light, the image of the LED light in the alternative reconstructed image is an ellipse.

[0026] In some embodiments, obtaining the central coordinates of the imaging center of the LED light in the image plane coordinate system and the imaging radius of the LED light in the parallel reconstructed image includes:

[0027] Perform edge extraction on the imaging of the LED light in the parallel reconstructed image to obtain the imaging edge;

[0028] According to the imaging edge, the least squares fitting circle method is used to obtain the central coordinates of the imaging center of the LED lamp in the image plane coordinate system and the imaging radius of the LED lamp in the parallel reconstructed image.

[0029] In some embodiments, the position and posture information includes the physical radius of the circular lampshade of the LED lamp; based on the center coordinates and the imaging radius, the target distance from the center of the LED lamp to the center of the camera lens of the mobile terminal is obtained according to the imaging principle and the proportional property of the side lengths of similar triangles, including:

[0030] Based on the center coordinates and imaging radius, combined with the entity radius and the focal length of the camera lens, the target distance from the center of the LED light to the center of the mobile terminal's camera lens is obtained according to the imaging principle and the proportional properties of the side lengths of similar triangles;

[0031] The expression of target distance is:

[0032]

[0033] Where, Indicates the target distance, represents the center coordinates, represents the entity radius, represents the imaging radius, Indicates the focal length of the camera lens.

[0034] In some embodiments, the position and posture information includes a set of posture characteristic parameters of the LED light and the WCS coordinates of the center of the LED light; and obtaining the position coordinates of the mobile terminal through coordinate transformation based on the position and posture information, the center coordinates, and the target distance includes:

[0035] According to the WCS coordinates, attitude feature parameter set and center coordinates, the second transformation relationship between the center of the LED light and the imaging center is obtained through the principle of coordinate transformation, and then the general solution is obtained by using the method of solving the underdetermined equation group;

[0036] Based on the general solution, the WCS coordinates and target distance are introduced to obtain the position coordinates of the mobile terminal in the world coordinate system.

[0037] To achieve the above objectives, another aspect of the present application provides a visible light positioning device, comprising:

[0038] The first module is configured to obtain an original image captured by a mobile terminal; the original image includes an imaging pattern of at least one LED lamp with a circular lampshade installed; the LED lamp is at an angle to a horizontal plane;

[0039] The second module is used to obtain angle information of the mobile terminal based on the tilt sensor of the mobile terminal;

[0040] The third module is used to demodulate the ID information of the LED light based on the original image through the fringe decoding algorithm; based on the ID information, the position and posture information of the LED light is obtained from the database;

[0041] The fourth module is used to perform coordinate transformation on the original image based on the position and attitude information and the angle information to obtain a parallel reconstructed image in which the plane of the mobile terminal is relatively parallel to the plane of the LED light;

[0042] The fifth module is used to obtain the central coordinates of the imaging center of the LED lamp in the image plane coordinate system and the imaging radius of the LED lamp in the parallel reconstructed image;

[0043] The sixth module is used to obtain a target distance from the center of the LED light to the center of the camera lens of the mobile terminal based on the center coordinates and the imaging radius, according to the imaging principle and the proportional property of the side lengths of similar triangles;

[0044] The seventh module is used to obtain the position coordinates of the mobile terminal through coordinate transformation processing based on the position posture information, center coordinates and target distance.

[0045] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the above-mentioned method when executing the computer program.

[0046] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned method is implemented.

[0047] The embodiments of the present application include at least the following beneficial effects: The present application provides a visible light positioning method, device, electronic device, and storage medium, which obtains an original image captured by a mobile terminal; the original image includes an imaging pattern of at least one LED lamp with a circular lampshade; the LED lamp is at an angle to the horizontal plane; based on the tilt sensor of the mobile terminal, angle information of the mobile terminal is obtained; based on the original image, the ID information of the LED lamp is demodulated using a fringe decoding algorithm; based on the ID information, the position and posture information of the LED lamp is obtained from a database; based on the position and posture information and the angle information, the original image is coordinate-transformed to obtain a parallel reconstructed image in which the plane of the mobile terminal and the plane of the LED lamp are relatively parallel; the center coordinates of the imaging center of the LED lamp in the parallel reconstructed image in the image plane coordinate system and the imaging radius of the LED lamp are obtained; based on the center coordinates and the imaging radius, a target distance from the center of the LED lamp to the center of the camera lens of the mobile terminal is obtained according to the imaging principle and the proportional property of the side lengths of similar triangles; based on the position and posture information, the position coordinates of the mobile terminal are obtained through coordinate transformation. The embodiments of the present application provide a visible light positioning method for positioning using a tilt sensor and a camera of a mobile terminal. This method addresses the problem that existing solutions do not consider the tilt of LED lamps and ignore the impact of LED lamp tilt on positioning accuracy. It is the first to use tilted circular LED lamps for visible light positioning and proposes a corresponding positioning algorithm, expanding the application scenarios of visible light positioning. The embodiments of this application can achieve visible light positioning efficiently and conveniently. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of the visible light positioning method provided in an embodiment of the present application;

[0049] Figure 2 is a schematic diagram of the relationship between the pinhole imaging model and three coordinate systems provided in an embodiment of the present application;

[0050] Figure 3 is a schematic diagram of the parallel reconstruction image and similar triangle principle provided by an embodiment of the present application;

[0051] Figure 4 is a schematic diagram of an example of a visible light positioning system model provided in an embodiment of the present application;

[0052] Figure 5 Schematic diagram of the overall process of the visible light positioning method provided in the embodiment of the present application;

[0053] Figure 6 Schematic diagram comparing the estimated position and the actual position in the test results provided in the embodiment of the present application;

[0054] Figure 7is a CDF diagram of the positioning error in the test results provided in the embodiment of the present application;

[0055] Figure 8 Schematic diagram of the structure of the visible light positioning device provided in an embodiment of the present application;

[0056] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0058] It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0059] The terms "at least one", "plurality", "each", "any", etc. used in this application include "at least one", "two" or more, "plurality" or "each", "any" or "any one", "each" or "any one" as used herein.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0061] The visible light positioning method provided in the embodiment of the present application relates to the field of data processing technology. The visible light positioning method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server, or can be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the visible light positioning method, etc., but is not limited to the above forms.

[0062] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0063] Figure 1 This is an optional flowchart of the visible light positioning method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S100 to S700.

[0064] S100, obtaining an original image captured by a mobile terminal;

[0065] It should be noted that the original image includes an imaging pattern of at least one LED lamp installed with a circular lampshade; the LED lamp is at an angle to the horizontal plane; in some embodiments, step S100 may include: photographing an image containing the LED lamp based on a mobile terminal to obtain a color image; performing grayscale transformation processing on the color image through a grayscale transformation algorithm to obtain the original image.

[0066] For example, in some specific embodiments, a mobile terminal photographs an LED light in any posture capable of being photographed by its built-in camera, obtains a color image, and then converts the color image into a grayscale image (hereinafter referred to as the "original image") using any grayscale conversion algorithm. The original image must include an image pattern of at least one LED light.

[0067] S200, obtaining angle information of the mobile terminal based on a tilt sensor of the mobile terminal;

[0068] For example, in some specific embodiments, the roll angle of the mobile terminal is measured using a tilt sensor built into the mobile terminal. and pitch angle .

[0069] S300: Demodulate the ID information of the LED light using a fringe decoding algorithm based on the original image; and obtain the position and posture information of the LED light from a database based on the ID information.

[0070] It should be noted that different LED lights are set with different light signal frequencies; in some embodiments, demodulating the ID information of the LED light through a stripe decoding algorithm based on the original image may include: demodulating the light signal frequency of the LED light through a stripe decoding algorithm based on the original image; and determining the ID information of the LED light based on light signal frequency matching.

[0071] For example, in some specific embodiments, a universal fringe decoding algorithm is used to demodulate the ID information of each LED light from the original image. This ID information is then matched against a locally stored LED-ID database to obtain a priori information corresponding to the ID information, including the WCS (world coordinate system) coordinates of the LED light center, the radius of the circular lampshade, and a set of characteristic parameters for the LED light's posture. Specifically, each LED light is loaded with a light signal of a different frequency. The receiver detects the flashing frequency of each light to determine the light's LED-ID. Furthermore, the location and posture information of the light corresponding to the LED-ID can be retrieved from the database. Specifically, the demodulation of ID information can also be achieved using the "Space-Time-Multiplexed Multi-ImageVisible Light Positioning System Exploiting Pseudo-Miller-Coding for SmartPhones" method.

[0072] S400, performing coordinate transformation on the original image based on the position and attitude information and the angle information to obtain a parallel reconstructed image in which the plane of the mobile terminal is relatively parallel to the plane of the LED light;

[0073] It should be noted that the position and posture information includes the posture feature parameter set of the LED lamp; the angle information includes the roll angle and the pitch angle; step S400 may include: based on the roll angle and the pitch angle, obtaining the first position relationship between any point in the original image and the corresponding point of the LED lamp through the coordinate transformation principle; based on the posture feature parameter set, obtaining the second position relationship between any point in the parallel reconstructed image to be reconstructed and the corresponding point of the LED lamp through the coordinate transformation principle; according to the first position relationship and the second position relationship, obtaining the first transformation relationship of converting any point in the LED lamp to the parallel reconstructed image; based on the first transformation relationship, combining the preset azimuth angle to perform coordinate transformation on the original image to obtain an alternative reconstructed image; the preset azimuth angle is traversed based on a preset step size; when the major axis and minor axis of the imaging of the LED lamp in the alternative reconstructed image are equal, determining that the alternative reconstructed image is a parallel reconstructed image in which the plane of the mobile terminal is relatively parallel to the plane of the LED lamp; wherein, when the plane of the mobile terminal is not parallel to the plane of the LED lamp, the imaging of the LED lamp in the alternative reconstructed image is elliptical.

[0074] Specifically, if Figure 2 As shown in the figure, the imaging positioning system in this application can be regarded as a pinhole imaging model, which involves coordinate transformations between three coordinate systems, namely the three-dimensional world coordinate system (WCS), the three-dimensional camera coordinate system (CCS), and the two-dimensional imaging plane coordinate system (IPCS). For ease of description, the letters w, c, and i are used below to represent the relevant parameters of the WCS, CCS, and IPCS coordinate systems, respectively. The origins of the WCS, CCS, and IPCS are 、 and , CCS Axis and IPCS The directions of the axes are the same, and the CCS Axis and IPCS The axes are in the same direction. In addition, It is the center of the camera lens. Completing the positioning of the mobile terminal is equivalent to solving the center point of the camera lens. Coordinates in WCS The center of the camera lens With IPCS origin The distance between them is the focal length of the camera lens .

[0075] According to the principle of coordinate transformation, any point on the lamp WCS coordinates With CCS coordinates The conversion relationship between them is given by formula (1) and formula (2):

[0076] (1)

[0077] (2)

[0078] in: Yes The WCS coordinates, Respectively represent the WCS Axis rotation , around Axis rotation And around Axis rotation The rotation matrix of 、 、 They are given by formula (3), formula (4) and formula (5) respectively:

[0079] (3)

[0080] (4)

[0081] (5)

[0082] And the rotation matrices in equations (3), (4), and (5) satisfy the following properties: , , .

[0083] In general, if the imaging plane is parallel to the plane where the screen of the mobile terminal is located (hereinafter referred to as the “terminal plane”), then the value in equation (2) is They can be equivalent to the roll angle, pitch angle and azimuth angle of the mobile terminal respectively. The combination of can be used to characterize the spatial posture of the mobile terminal, recorded as the feature parameter set Among them, the roll angle and pitch angle Characterizes the tilt state of the mobile terminal. When both values ​​are 0, the terminal plane is in a horizontal state.

[0084] Assume that any point on the lamp , passing through the center of the camera lens The image mapped onto the imaging plane is the image point , then according to the collinearity property of pinhole imaging, any point on the LED light The CCS coordinates of the corresponding image point The relationship between the IPCS coordinates can be given by formula (6):

[0085] (6)

[0086] in Yes The CCS coordinates, Yes IPCS coordinates.

[0087] In the positioning system model of this application, the LED light plane is not horizontal, that is, there is a relative angle between the LED light plane and the horizontal plane. , and the tilt posture of the LED light can be determined by the characteristic parameter set Characterization. Specifically, The LED light plane is formed by the horizontal LED light plane around the WCS first. Axis rotation Then go around Axis rotation And finally around Axis rotation formed later.

[0088] For example, in some specific embodiments, step S400 may be implemented through the following process:

[0089] Reconstruct the original image into an image when the terminal plane is relatively parallel to the LED light plane (hereinafter referred to as the "parallel reconstructed image"), and solve the azimuth angle of the mobile terminal This step is further described below.

[0090] Generally, the terminal plane is not parallel to the lamp plane, so the image of the LED lamp in the original image is elliptical. Since it is very difficult to obtain the projection point corresponding to the center of the LED lamp in the elliptical LED lamp image, this application proposes to first reconstruct the original image into a parallel reconstructed image. Since the image of the LED lamp in the parallel reconstructed image is circular, and the projection point corresponding to the center of the LED lamp is the center of the LED lamp image, the least squares circle fitting method or other circle fitting methods can be used to obtain the center of the circular image.

[0091] Based on this idea, the core problem is how to obtain a parallel reconstructed image of the original image. This application proposes a technical approach to solve this problem by traversing several reconstructed image data sets corresponding to different azimuth angles of the terminal, calculating the parallel reconstructed image corresponding to the inclination angle of the LED light and simultaneously obtaining the azimuth angle of the terminal. The specific technical approach is as follows:

[0092] First, take any point on the edge of the LED light , assuming The imaging point on the imaging plane is the image point . Assume that the original image is in the mobile terminal The image is taken in a tilted state, based on the roll angle of the mobile terminal and pitch angle , using the coordinate transformation principle in equations (1), (2) and (6), the image point in the original image is The IPCS (image coordinate system, image plane coordinate system or imaging plane coordinate system) coordinates with dot The relationship between the WCS coordinates (i.e., the first position relationship) can be expressed by equations (7) and (8):

[0093] (7)

[0094] (8)

[0095] in, Yes On mobile terminals CCS coordinates in the tilted state.

[0096] Secondly, the assumption point The corresponding points in the parallel reconstructed image are points Since the parallel reconstructed image is an image obtained when the terminal plane is relatively parallel to the LED light plane, the tilt postures of the mobile terminal and the LED light should be the same. Assume that the LED light is in The parallel reconstructed image can be regarded as the tilt state of the mobile terminal. Based on the obtained LED lamp posture feature parameter set , using the coordinate transformation principle in equations (1), (2) and (6), we can get IPCS coordinates and The relationship between the WCS coordinates (i.e., the second position relationship) can be expressed by equations (9) and (10):

[0097] (9)

[0098] (10)

[0099] in, Yes On mobile terminals CCS coordinates in the tilted state.

[0100] Furthermore, according to equations (7), (8), (9) and (10), when the azimuth angle of the mobile terminal is When the corresponding image point coordinates in the reconstructed image The coordinates of the image points in the original image The corresponding relationship between them (i.e. the first transformation relationship) is given by equations (11) and (12):

[0101] (11)

[0102] (12)

[0103] Due to the azimuth of the mobile terminal The value range is 0 to , so the azimuth The value range is Step length traversal .for For each value of , use Equation (11) and Equation (12) to reconstruct the edge pixels of the LED light imaging in the original image, and after obtaining the corresponding reconstructed image, any image edge extraction technology can be used to extract the edge pixels of the LED light imaging in the reconstructed image to calculate the major axis length and minor axis length of the elliptical LED light imaging. When a certain value of makes the long axis length and short axis length of the LED light imaging equal, it means that the LED light imaging in the reconstructed image is circular. traverse and record the current The value of is the azimuth angle of the mobile terminal, and the corresponding reconstructed image is equivalent to the image taken when the terminal plane is parallel to the LED light plane, that is, the parallel reconstructed image.

[0104] The parallel reconstructed image obtained through the above operation can help obtain the terminal coordinate information in the subsequent steps; and the azimuth angle obtained , which can be used to assist in detecting the terminal’s direction, posture, etc., and provide assistance for applications such as wearable terminal control.

[0105] S500, obtaining the central coordinates of the imaging center of the LED lamp in the image plane coordinate system and the imaging radius of the LED lamp in the parallel reconstructed image;

[0106] It should be noted that, in some embodiments, step S500 may include: performing edge extraction on the imaging of the LED lamp in the parallel reconstructed image to obtain the imaging edge; based on the imaging edge, obtaining the central coordinates of the imaging center of the LED lamp in the parallel reconstructed image in the image plane coordinate system and the imaging radius of the LED lamp by using the least squares fitting circle method.

[0107] For example, in some specific embodiments, the coordinates of the center of the LED light imaging in the parallel reconstructed image at the IPCS and the radius of the LED light imaging are obtained. The parallel reconstructed image obtained in the previous step can be used to obtain the circular imaging of the circular LED light in the parallel reconstructed image. Therefore, any image edge extraction technology can be used to obtain the edge pixel points of the LED light imaging in the parallel reconstructed image, and then the least squares circle fitting method or other circle fitting methods can be used to obtain the coordinates of the center of the LED light imaging in the parallel reconstructed image at the IPCS. And the radius of LED light imaging .

[0108] S600: Based on the center coordinates and the imaging radius, a target distance from the center of the LED light to the center of the camera lens of the mobile terminal is obtained according to the imaging principle and the proportional property of the side lengths of similar triangles.

[0109] It should be noted that the position and posture information includes the physical radius of the circular lampshade of the LED lamp; in some embodiments, step S600 may include: based on the center coordinates and the imaging radius, combined with the physical radius and the focal length of the camera lens, according to the imaging principle and the proportional property of the side lengths of similar triangles, obtaining the target distance from the center of the LED lamp to the center of the camera lens of the mobile terminal; wherein the expression of the target distance is:

[0110]

[0111] Where, Indicates the target distance, represents the center coordinates, represents the entity radius, represents the imaging radius, Indicates the focal length of the camera lens.

[0112] For example, in some specific embodiments, the distance between the center of the camera lens and the center of the LED light plane is obtained. .like Figure 3 As shown, the image captured when the terminal plane is parallel to the LED light plane is a parallel reconstructed image, and the terminal plane and the imaging plane are parallel to each other, so the LED light plane and the imaging plane of the parallel reconstructed image are parallel to each other. Figure 3 , take any point on the edge of the LED light , set up a point The imaging point on the imaging plane is the image point , LED light center point The imaging point on the imaging plane is the image point .Depend on Figure 3 It can be seen that since the LED light plane and the imaging plane of the parallel reconstructed image are parallel to each other, the triangle With triangle Similar. According to the proportional properties of the side lengths of similar triangles, the distance between the center of the lens and the center of the LED light plane is It can be given by formula (13):

[0113] (13)

[0114] S700 , obtaining the position coordinates of the mobile terminal through coordinate transformation according to the position and posture information, the center coordinates and the target distance.

[0115] It should be noted that the position and posture information includes the posture characteristic parameter set of the LED lamp and the WCS coordinates of the center of the LED lamp; in some embodiments, step S700 may include: according to the WCS coordinates, the posture characteristic parameter set and the center coordinates, the second transformation relationship between the center of the LED lamp and the imaging center is obtained by processing through the coordinate transformation principle, and then the general solution is obtained by using the method of solving the underdetermined equation group; based on the general solution, the WCS coordinates and the target distance are introduced to solve the position coordinates of the mobile terminal in the world coordinate system.

[0116] For example, in some specific embodiments, the geometric properties of imaging can be used to estimate the position of the mobile terminal. By obtaining the WCS coordinates of the center of the LED light , LED light posture feature parameter set , and the coordinates of the LED light imaging center in the IPCS in the obtained parallel reconstructed image , using the coordinate transformation principle in equations (1), (2) and (6), the imaging equation between the center of the LED light and the imaging center of the LED light in the parallel reconstructed image can be obtained as follows:

[0117] (14)

[0118] (15)

[0119] Among them: Formula (14) contains two independent equations, and Formula (15) contains three unknowns Therefore, Equations (14) and (15) constitute an underdetermined system of equations, and the general solution can be obtained by using the method of solving underdetermined systems of equations as follows:

[0120] (16)

[0121] in: and is a three-dimensional vector with a specific value, and is a parameter to be solved.

[0122] On the other hand, using the WCS coordinates of the LED light center And the distance between the center of the lens and the center of the LED light plane , substitute The definition of , we can get:

[0123] (17)

[0124] Substituting equation (16) into equation (17), we can solve There are two solutions and ,therefore There are also two corresponding solutions, given by equations (18) and (19):

[0125] (18)

[0126] (19)

[0127] Only one of the solutions is correct. and Only one of them can meet the reasonable value range of mobile terminal height , then you can take its corresponding As the position coordinates of the mobile terminal in the WCS, the positioning of the mobile terminal is completed.

[0128] In order to explain the principles of the technical solution of this application in detail, the overall process of this application is described below in combination with some specific embodiments. It is easy to understand that the following is an explanation of the technical principles of this application and cannot be regarded as a limitation of this application.

[0129] First, it's important to note that in the current consumer market, most mobile smart devices are equipped only with cameras containing imaging sensors, rather than photodetectors. Therefore, indoor visible light positioning systems based on imaging sensors are not only more practical but also have broader market prospects. Currently, most imaging visible light positioning systems rely on the premise that a single image captured by the camera contains at least three LED patterns. However, due to limitations in the camera's field of view and the density of LED light deployment, this requirement is difficult to meet in real-world scenarios. To address this, some imaging visible light positioning systems utilize the mobile terminal's tilt sensor as an auxiliary device to obtain information about the mobile terminal's tilt posture, thereby reducing the number of LED patterns required in the image. For example, positioning a mobile terminal with two and one imaging pattern, respectively, can be achieved. However, these visible light positioning systems all assume that the LED light plane is horizontal, that is, parallel to the ground. If the LED light plane is tilted relative to the ground, the positioning performance of these visible light positioning systems deteriorates, and they are unable to provide normal positioning services.

[0130] To address the shortcomings of existing indoor visible light positioning solutions, this application proposes a visible light positioning method that utilizes a mobile terminal's tilt sensor and camera for positioning. This method addresses the problem that existing solutions fail to consider the tilt of LED lamps and ignore the impact of LED lamp tilt on positioning accuracy. This method, for the first time, utilizes tilted circular LED lamps for visible light positioning and proposes a corresponding positioning algorithm, expanding the application scenarios of visible light positioning.

[0131] In addition, the method proposed in this application can also be used to obtain relatively accurate terminal azimuth information in indoor scenarios. This information can be used for terminal posture detection and control, and can provide effective application support for indoor wearable devices. Current mainstream smart terminals mainly obtain azimuth information through geomagnetic components such as compasses or networks such as GPS satellites and base stations, but there is a problem of poor accuracy in indoor scenarios. However, this application uses tilted circular LED lights to accurately and conveniently calculate the terminal azimuth.

[0132] Consider a Figure 4 In the general indoor environment shown, an LED lamp with a circular lampshade is installed on the ceiling, and the plane of the LED lamp is tilted relative to the ground plane, that is, there is a certain angle between the plane of the LED lamp and the ground plane. Each LED lamp is loaded with a light signal of a different frequency. The receiving end obtains the LED-ID of the lamp by detecting the flashing frequency of different lamps, and then obtains the position, posture and other information of the lamp corresponding to the LED-ID in the database. The built-in camera of the mobile terminal is used to photograph the LED lamp. The image obtained by the photograph contains an image of at least one LED lamp. After being processed by the positioning technology proposed in this application, the precise location information of the mobile terminal can be obtained, thereby realizing the positioning of the user.

[0133] like Figure 2 As shown in the figure, the imaging positioning system in this application can be regarded as a pinhole imaging model, which involves coordinate transformations between three coordinate systems, namely the three-dimensional world coordinate system (WCS), the three-dimensional camera coordinate system (CCS), and the two-dimensional imaging plane coordinate system (IPCS). For ease of description, the letters w, c, and i are used below to represent the relevant parameters of the WCS, CCS, and IPCS coordinate systems, respectively. The origins of the WCS, CCS, and IPCS are 、 and , CCS Axis and IPCS The directions of the axes are the same, and the CCS Axis and IPCS The axes are in the same direction. In addition, It is the center of the camera lens. Completing the positioning of the mobile terminal is equivalent to solving the center point of the camera lens. Coordinates in WCS The center of the camera lens With IPCS origin The distance between them is the focal length of the camera lens .

[0134] According to the principle of coordinate transformation, any point on the lamp WCS coordinates With CCS coordinates The conversion relationship between them is given by formula (1) and formula (2):

[0135] (1)

[0136] (2)

[0137] in: Yes The WCS coordinates, Respectively represent the WCS Axis rotation , around Axis rotation And around Axis rotation The rotation matrix of 、 、 They are given by formula (3), formula (4) and formula (5) respectively:

[0138] (3)

[0139] (4)

[0140] (5)

[0141] And the rotation matrices in equations (3), (4), and (5) satisfy the following properties: , , .

[0142] In general, if the imaging plane is parallel to the plane where the screen of the mobile terminal is located (hereinafter referred to as the “terminal plane”), then the value in equation (2) is They can be equivalent to the roll angle, pitch angle and azimuth angle of the mobile terminal respectively. The combination of can be used to characterize the spatial posture of the mobile terminal, recorded as the feature parameter set Among them, the roll angle and pitch angle Characterizes the tilt state of the mobile terminal. When both values ​​are 0, the terminal plane is in a horizontal state.

[0143] Assume that any point on the lamp , passing through the center of the camera lens The image mapped onto the imaging plane is the image point , then according to the collinearity property of pinhole imaging, any point on the LED light The CCS coordinates of the corresponding image point The relationship between the IPCS coordinates can be given by formula (6):

[0144] (6)

[0145] in Yes The CCS coordinates, Yes IPCS coordinates.

[0146] In the positioning system model of this application, the LED light plane is not horizontal, that is, there is a relative angle between the LED light plane and the horizontal plane. , and the tilt posture of the LED light can be determined by the characteristic parameter set Characterization. Specifically, The LED light plane is formed by the horizontal LED light plane around the WCS first. Axis rotation Then go around Axis rotation And finally around Axis rotation formed later.

[0147] Based on the aforementioned imaging positioning system model, the main operating steps of the mobile terminal positioning solution proposed in this application are as follows: Figure 5 As shown, the following is a detailed introduction:

[0148] Step 1: The mobile terminal takes a photo of the LED light in any position where its built-in camera can take photos of the LED light, obtaining a color image. The color image is then converted into a grayscale image (hereinafter referred to as the "original image") using any grayscale conversion algorithm. The original image must contain at least one image pattern of the LED light.

[0149] Step 2: Use the tilt sensor built into the mobile terminal to measure the roll angle of the mobile terminal and pitch angle .

[0150] Step 3: Decode the ID information of each LED light from the original image using a universal fringe decoding algorithm. This ID information is then matched against a locally stored LED-ID database to obtain a priori information corresponding to the ID information, including the WCS (world coordinate system) coordinates of the LED light's center, the radius of the circular lampshade, and the LED light's posture characteristic parameter set. Specifically, each LED light is loaded with a light signal of a different frequency. The receiver detects the flashing frequency of each light to determine the light's LED-ID. This information can then be retrieved from the database, along with the position and posture information corresponding to the LED-ID. Specifically, the demodulation of the ID information can also be achieved using the method described in "Space-Time-Multiplexed Multi-Image Visible LightPositioning System Exploiting Pseudo-Miller-Coding for Smart Phones."

[0151] Step 4: Reconstruct the original image into an image when the terminal plane is relatively parallel to the LED light plane (hereinafter referred to as the "parallel reconstructed image"), and solve the azimuth angle of the mobile terminal This step is further described below.

[0152] Generally, the terminal plane is not parallel to the lamp plane, so the image of the LED lamp in the original image is elliptical. Since it is very difficult to obtain the projection point corresponding to the center of the LED lamp in the elliptical LED lamp image, this application proposes to first reconstruct the original image into a parallel reconstructed image. Since the image of the LED lamp in the parallel reconstructed image is circular, and the projection point corresponding to the center of the LED lamp is the center of the LED lamp image, the least squares circle fitting method or other circle fitting methods can be used to obtain the center of the circular image.

[0153] Based on this idea, the core problem is how to obtain a parallel reconstructed image of the original image. This application proposes a technical approach to solve this problem by traversing several reconstructed image data sets corresponding to different azimuth angles of the terminal, calculating the parallel reconstructed image corresponding to the inclination angle of the LED light and simultaneously obtaining the azimuth angle of the terminal. The specific technical approach is as follows:

[0154] First, take any point on the edge of the LED light , assuming The imaging point on the imaging plane is the image point . Assume that the original image is in the mobile terminal The image is taken in a tilted state, based on the roll angle of the mobile terminal and pitch angle , using the coordinate transformation principle in equations (1), (2) and (6), the image point in the original image is The IPCS (image coordinate system, image plane coordinate system or imaging plane coordinate system) coordinates with dot The relationship between the WCS coordinates (i.e., the first position relationship) can be expressed by equations (7) and (8):

[0155] (7)

[0156] (8)

[0157] in, Yes On mobile terminals CCS coordinates in the tilted state.

[0158] Secondly, the assumption point The corresponding points in the parallel reconstructed image are points Since the parallel reconstructed image is an image obtained when the terminal plane is relatively parallel to the LED light plane, the tilt postures of the mobile terminal and the LED light should be the same. Assume that the LED light is in The parallel reconstructed image can be regarded as the tilt state of the mobile terminal. Based on the obtained LED lamp posture feature parameter set , using the coordinate transformation principle in equations (1), (2) and (6), we can get IPCS coordinates and The relationship between the WCS coordinates (i.e., the second position relationship) can be expressed by equations (9) and (10):

[0159] (9)

[0160] (10)

[0161] in, Yes On mobile terminals CCS coordinates in the tilted state.

[0162] Furthermore, according to equations (7), (8), (9) and (10), when the azimuth angle of the mobile terminal is When the corresponding image point coordinates in the reconstructed image The coordinates of the image points in the original image The corresponding relationship between them (i.e. the first transformation relationship) is given by equations (11) and (12):

[0163] (11)

[0164] (12)

[0165] Due to the azimuth of the mobile terminal The value range is 0 to , so the azimuth The value range is Step length traversal .for For each value of , use Equation (11) and Equation (12) to reconstruct the edge pixels of the LED light imaging in the original image, and after obtaining the corresponding reconstructed image, any image edge extraction technology can be used to extract the edge pixels of the LED light imaging in the reconstructed image to calculate the major axis length and minor axis length of the elliptical LED light imaging. When a certain value of makes the long axis length and short axis length of the LED light imaging equal, it means that the LED light imaging in the reconstructed image is circular. traverse and record the current The value of is the azimuth angle of the mobile terminal, and the corresponding reconstructed image is equivalent to the image taken when the terminal plane is parallel to the LED light plane, that is, the parallel reconstructed image.

[0166] The parallel reconstructed image obtained through the above operation can help obtain the terminal coordinate information in the subsequent steps; and the azimuth angle obtained , which can be used to assist in detecting the terminal’s direction, posture, etc., and provide assistance for applications such as wearable terminal control.

[0167] Step 5: Through the parallel reconstructed image obtained in the previous step, the circular imaging of the circular LED light in the parallel reconstructed image can be obtained. Therefore, any image edge extraction technology can be used to obtain the edge pixel points of the LED light imaging in the parallel reconstructed image, and then the least squares fitting circle method or other fitting circle methods can be used to obtain the coordinates of the LED light imaging center in the parallel reconstructed image in the IPCS. And the radius of LED light imaging .

[0168] Step 6: Get the distance between the center of the camera lens and the center of the LED light plane .like Figure 3 As shown, the image captured when the terminal plane is parallel to the LED light plane is a parallel reconstructed image, and the terminal plane and the imaging plane are parallel to each other, so the LED light plane and the imaging plane of the parallel reconstructed image are parallel to each other. Figure 3 , take any point on the edge of the LED light , set up a point The imaging point on the imaging plane is the image point , LED light center point The imaging point on the imaging plane is the image point .Depend on Figure 3 It can be seen that since the LED light plane and the imaging plane of the parallel reconstructed image are parallel to each other, the triangle With triangle Similar. According to the proportional properties of the side lengths of similar triangles, the distance between the center of the lens and the center of the LED light plane is It can be given by formula (13):

[0169] (13)

[0170] Step 7: Use the geometric properties of the imaging to estimate the position of the mobile terminal. By obtaining the WCS coordinates of the center of the LED light , LED light posture feature parameter set , and the coordinates of the LED light imaging center in the IPCS in the obtained parallel reconstructed image , using the coordinate transformation principle in equations (1), (2) and (6), the imaging equation between the center of the LED light and the imaging center of the LED light in the parallel reconstructed image can be obtained as follows:

[0171] (14)

[0172] (15)

[0173] Among them: Formula (14) contains two independent equations, and Formula (15) contains three unknowns Therefore, Equations (14) and (15) constitute an underdetermined system of equations, and the general solution can be obtained by using the method of solving underdetermined systems of equations as follows:

[0174] (16)

[0175] in: and is a three-dimensional vector with a specific value, and is a parameter to be solved.

[0176] On the other hand, using the WCS coordinates of the LED light center And the distance between the center of the lens and the center of the LED light plane , substitute The definition of , we can get:

[0177] (17)

[0178] Substituting equation (16) into equation (17), we can solve There are two solutions and ,therefore There are also two corresponding solutions, given by equations (18) and (19):

[0179] (18)

[0180] (19)

[0181] Only one of the solutions is correct. and Only one of them can meet the reasonable value range of mobile terminal height , then you can take its corresponding As the position coordinates of the mobile terminal in the WCS, the positioning of the mobile terminal is completed.

[0182] In order to more fully illustrate the beneficial effects of the present application, the effectiveness and advancement of the present application are further illustrated below in combination with the test results and analysis of specific embodiments.

[0183] In this example, the test system uses a typical indoor room model. A tilted circular LED light is mounted on the ceiling, and the center of the floor serves as the WCS origin. The test area within the room is designed to have 81 test points evenly spaced within the area, with 15 cm between adjacent test points. The actual coordinates of each test point are recorded. The relevant system parameter settings during the test are shown in Table 1.

[0184] Table 1

[0185]

[0186] During the test, the height of the mobile terminal was set to 1.2m from the ground. Photos of the circular LED light on the ceiling were taken at 81 test points. At the same time, the roll and pitch angles of the mobile terminal measured by the built-in tilt sensor of the mobile terminal were recorded. Finally, the estimated position coordinates of the mobile terminal were calculated using a computer. Figure 6 The estimated position and the corresponding real position of 81 test points are shown. According to the estimated position coordinates and the real position coordinates of 81 test points, the positioning error of each test point is calculated. Figure 7 The Cumulative Distribution Function (CDF) graph of positioning error is shown.

[0187] right Figure 6After statistics of the test results in Table 2, the average, maximum and minimum values ​​of the positioning error in the test results are given:

[0188] Table 2

[0189]

[0190] according to Figure 6 and Figure 7 The positioning system has a small positioning error at most test points, with an average positioning error of around 6 cm. The positioning error for more than 90% of the test points is less than 11 cm. This shows that the positioning system has high positioning accuracy, fully demonstrating the effectiveness of the method proposed in this application.

[0191] In summary, the visible light positioning method proposed in this application can be used for visible light positioning scenarios where the LED light plane is tilted. It has the advantages of high positioning accuracy and requires fewer LED lights. It is suitable for mobile terminals with built-in cameras and tilt sensors and has broad application prospects. In addition, the terminal azimuth acquisition method based on this application can provide accurate azimuth information for posture detection of mobile phones, wearable devices, etc., thereby facilitating precise terminal control.

[0192] See also Figure 8 The present application also provides a visible light positioning device 900, which can implement the above-mentioned visible light positioning method. The device includes:

[0193] The first module 910 is configured to obtain an original image captured by a mobile terminal; the original image includes an imaging pattern of at least one LED lamp with a circular lampshade installed; the LED lamp is at an angle to a horizontal plane;

[0194] The second module 920 is configured to obtain angle information of the mobile terminal based on a tilt sensor of the mobile terminal;

[0195] The third module 930 is used to demodulate the ID information of the LED light according to the original image using a fringe decoding algorithm; and obtain the position and posture information of the LED light from the database based on the ID information;

[0196] The fourth module 940 is used to perform coordinate transformation on the original image based on the position and attitude information and the angle information to obtain a parallel reconstructed image in which the plane of the mobile terminal is relatively parallel to the plane of the LED light;

[0197] A fifth module 950 is configured to obtain the central coordinates of the imaging center of the LED lamp in the parallel reconstructed image in the image plane coordinate system and the imaging radius of the LED lamp;

[0198] A sixth module 960 is configured to obtain a target distance from the center of the LED light to the center of the camera lens of the mobile terminal based on the center coordinates and the imaging radius according to the imaging principle and the proportionality property of the side lengths of similar triangles;

[0199] The seventh module 970 is used to obtain the position coordinates of the mobile terminal through coordinate transformation processing according to the position posture information, center coordinates and target distance.

[0200] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0201] The present application also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-described visible light positioning method when executing the computer program. The electronic device can be any smart terminal, such as a tablet computer or an in-vehicle computer.

[0202] It can be understood that the contents of the above method embodiments are applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0203] See also Figure 9 , Figure 9 The hardware structure of an electronic device 1000 according to another embodiment is shown. The electronic device includes:

[0204] The processor 1001 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0205] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called by the processor 1001 to execute the visible light positioning method of the embodiments of this application.

[0206] Input / output interface 1003, used to implement information input and output;

[0207] Communication interface 1004, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0208] Bus 1005 , which transmits information between various components of the device (e.g., processor 1001 , memory 1002 , input / output interface 1003 , and communication interface 1004 );

[0209] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0210] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned visible light positioning method is implemented.

[0211] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0212] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0213] The embodiments of the present application provide a visible light positioning method, a visible light positioning device, an electronic device, and a storage medium. The method comprises: obtaining an original image captured by a mobile terminal; the original image includes an imaging pattern of at least one LED lamp with a circular lampshade installed; the LED lamp has an angle with the horizontal plane; obtaining angle information of the mobile terminal based on a tilt sensor of the mobile terminal; demodulating the ID information of the LED lamp using a stripe decoding algorithm based on the original image; obtaining the position and posture information of the LED lamp from a database based on the ID information; performing a coordinate transformation on the original image based on the position and posture information and the angle information to obtain a parallel reconstructed image in which the plane of the mobile terminal and the plane of the LED lamp are relatively parallel; obtaining the center coordinates of the imaging center of the LED lamp in the image plane coordinate system and the imaging radius of the LED lamp in the parallel reconstructed image; obtaining the target distance from the center of the LED lamp to the center of the camera lens of the mobile terminal based on the center coordinates and the imaging radius according to the imaging principle and the proportional property of the side lengths of similar triangles; and obtaining the position coordinates of the mobile terminal through coordinate transformation based on the position and posture information, the center coordinates, and the target distance. The embodiments of the present application provide a visible light positioning method for positioning using a tilt sensor and a camera of a mobile terminal. This method addresses the problem that existing solutions do not consider the tilt of LED lamps and ignore the impact of LED lamp tilt on positioning accuracy. It is the first to use tilted circular LED lamps for visible light positioning and proposes a corresponding positioning algorithm, expanding the application scenarios of visible light positioning. The embodiments of this application can achieve visible light positioning efficiently and conveniently.

[0214] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0215] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0216] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0217] Those skilled in the art will appreciate that all or some of the steps, devices, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0218] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the numbers used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0219] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0220] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0221] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0222] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0223] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0224] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A visible light positioning method, characterized in that: The method comprises: Acquire an original image captured by a mobile terminal; the original image includes an imaging pattern of at least one LED lamp with a circular lampshade installed; the LED lamp is at an angle to a horizontal plane; obtaining angle information of the mobile terminal based on a tilt sensor of the mobile terminal; According to the original image, the ID information of the LED lamp is demodulated by a fringe decoding algorithm; based on the ID information, the position and posture information of the LED lamp is obtained from a database; Based on the position and posture information and the angle information, coordinate transformation is performed on the original image to obtain a parallel reconstructed image in which the plane of the mobile terminal is relatively parallel to the plane of the LED lamp; The position and posture information includes a set of posture feature parameters of the LED lamp; the angle information includes a roll angle and a pitch angle; and the coordinate transformation of the original image based on the position and posture information and the angle information to obtain a parallel reconstructed image in which the plane of the mobile terminal is relatively parallel to the plane of the LED lamp includes: Based on the roll angle and the pitch angle, a first positional relationship between any point in the original image and a corresponding point of the LED lamp is obtained by processing using a coordinate transformation principle; Based on the posture feature parameter set, a second positional relationship between any point in the parallel reconstructed image to be reconstructed and a corresponding point of the LED lamp is obtained by processing using a coordinate transformation principle; Obtaining a first transformation relationship for converting any point in the LED light into the parallel reconstructed image according to the first position relationship and the second position relationship; Based on the first transformation relationship, the coordinate transformation of the original image is performed in combination with a preset azimuth angle to obtain an alternative reconstructed image; the preset azimuth angle is traversed based on a preset step size; When the major axis and the minor axis of the imaging of the LED light in the candidate reconstructed image are equal, determining that the candidate reconstructed image is the parallel reconstructed image in which the plane of the mobile terminal is relatively parallel to the plane of the LED light; Wherein, when the plane of the mobile terminal is not parallel to the plane of the LED lamp, the imaging of the LED lamp in the alternative reconstructed image is an ellipse; Obtaining the central coordinates of the imaging center of the LED lamp in the image plane coordinate system and the imaging radius of the LED lamp in the parallel reconstructed image; Based on the center coordinates and the imaging radius, a target distance from the center of the LED light to the center of the camera lens of the mobile terminal is obtained according to the imaging principle and the proportional property of the side lengths of similar triangles; The position coordinates of the mobile terminal are obtained through coordinate transformation processing according to the position posture information, the center coordinates and the target distance.

2. The method according to claim 1, characterized in that The obtaining of the original image captured by the mobile terminal includes: The mobile terminal captures an image containing the LED light to obtain a color image; The color image is subjected to grayscale transformation processing by a grayscale transformation algorithm to obtain the original image.

3. The method according to claim 1, characterized in that Different LED lights are set with different light signal frequencies; and according to the original image, the ID information of the LED lights is demodulated by a fringe decoding algorithm, including: According to the original image, the light signal frequency of the LED lamp is demodulated by a fringe decoding algorithm; The ID information of the LED lamp is determined based on the optical signal frequency matching.

4. The method according to claim 1, wherein The obtaining of the central coordinates of the imaging center of the LED lamp in the image plane coordinate system and the imaging radius of the LED lamp in the parallel reconstructed image includes: Performing edge extraction on the imaging of the LED light in the parallel reconstructed image to obtain an imaging edge; According to the imaging edge, the center coordinates of the imaging center of the LED lamp in the image plane coordinate system and the imaging radius of the LED lamp in the parallel reconstructed image are obtained by least squares fitting circle method.

5. The method according to claim 1, wherein The position and posture information includes the physical radius of the circular lampshade of the LED lamp; the target distance from the center of the LED lamp to the center of the camera lens of the mobile terminal is obtained based on the center coordinates and the imaging radius according to the imaging principle and the proportional property of the side lengths of similar triangles, including: Based on the center coordinates and the imaging radius, combined with the entity radius and the focal length of the camera lens, a target distance from the center of the LED light to the center of the camera lens of the mobile terminal is obtained according to the imaging principle and the proportional property of the side lengths of similar triangles; The target distance is expressed as follows: Where, Indicates the target distance, represents the center coordinates, represents the entity radius, represents the imaging radius, Indicates the focal length of the camera lens.

6. The method according to claim 1, characterized in that The position and posture information includes a posture feature parameter set of the LED lamp and a WCS coordinate of the center of the LED lamp; and obtaining the position coordinates of the mobile terminal through coordinate transformation processing according to the position and posture information, the center coordinates, and the target distance includes: According to the WCS coordinates, the posture feature parameter set and the center coordinates, a second transformation relationship between the center of the LED lamp and the imaging center is obtained by using the coordinate transformation principle, and then a general solution is obtained by using a method of solving an underdetermined system of equations; Based on the general solution, the WCS coordinates and the target distance are introduced to obtain the position coordinates of the mobile terminal in the world coordinate system.

7. A visible light positioning device, characterized in that: The device comprises: The first module is configured to obtain an original image captured by a mobile terminal; the original image includes an imaging pattern of at least one LED lamp with a circular lampshade installed; the LED lamp is at an angle to a horizontal plane; A second module is configured to obtain angle information of the mobile terminal based on a tilt sensor of the mobile terminal; The third module is configured to demodulate the ID information of the LED lamp using a fringe decoding algorithm according to the original image; and obtain the position and posture information of the LED lamp from a database based on the ID information; A fourth module is configured to perform coordinate transformation on the original image based on the position and posture information and the angle information to obtain a parallel reconstructed image in which the plane of the mobile terminal is relatively parallel to the plane of the LED lamp; The position and posture information includes a set of posture feature parameters of the LED lamp; the angle information includes a roll angle and a pitch angle; and the coordinate transformation of the original image based on the position and posture information and the angle information to obtain a parallel reconstructed image in which the plane of the mobile terminal is relatively parallel to the plane of the LED lamp includes: Based on the roll angle and the pitch angle, a first positional relationship between any point in the original image and a corresponding point of the LED lamp is obtained by processing using a coordinate transformation principle; Based on the posture feature parameter set, a second positional relationship between any point in the parallel reconstructed image to be reconstructed and a corresponding point of the LED lamp is obtained by processing using a coordinate transformation principle; Obtaining a first transformation relationship for converting any point in the LED light into the parallel reconstructed image according to the first position relationship and the second position relationship; Based on the first transformation relationship, the coordinate transformation of the original image is performed in combination with a preset azimuth angle to obtain an alternative reconstructed image; the preset azimuth angle is traversed based on a preset step size; When the major axis and the minor axis of the imaging of the LED light in the candidate reconstructed image are equal, determining that the candidate reconstructed image is the parallel reconstructed image in which the plane of the mobile terminal is relatively parallel to the plane of the LED light; Wherein, when the plane of the mobile terminal is not parallel to the plane of the LED lamp, the imaging of the LED lamp in the alternative reconstructed image is an ellipse; A fifth module is used to obtain the central coordinates of the imaging center of the LED lamp in the image plane coordinate system and the imaging radius of the LED lamp in the parallel reconstructed image; A sixth module is configured to obtain a target distance from the center of the LED lamp to the center of the camera lens of the mobile terminal based on the center coordinates and the imaging radius and according to the imaging principle and the proportional property of the side lengths of similar triangles; The seventh module is used to obtain the position coordinates of the mobile terminal through coordinate transformation processing according to the position posture information, the center coordinates and the target distance.

8. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images