Camera - Photodiode Hybrid Localization Method and System under Arbitrary - shaped Light Sources

Abstract

The present invention discloses a camera - photodiode hybrid positioning method and system under a light source of any shape, belonging to the field of optical positioning. By modulating the LED light source, visible light communication information is transmitted, and a photodiode array is used to receive the visible light communication information, and the polar angle and azimuth angle of each LED light source are calculated based on the visible light communication information; the camera synchronously captures the image of the LED light source, extracts the LED shape features through image processing and calculates its polar angle and azimuth angle; a matching algorithm is used to match the data of the photodiode array and the camera; finally, the visible light communication information and the visual features of the camera are comprehensively utilized for positioning. The present invention combines the advantages of photodiodes and cameras, solves the positioning problem of LED light sources of any shape in complex scenarios, and is applicable to scenarios such as indoor optical communication, optical positioning, and computer vision.

Description

Technical Field

[0001] The present invention relates to the technical field of optical positioning, and particularly relates to a camera - photodiode hybrid positioning method and system under an arbitrarily shaped light source. Background Art

[0002] Visible Light Positioning (VLP) utilizes the high - frequency modulation characteristics of LED light sources and combines optical sensors to calculate the target position. It has technical advantages such as high positioning accuracy, no electromagnetic pollution, and natural compatibility with lighting facilities, showing broad application prospects in fields such as indoor navigation, intelligent warehousing, and industrial robots. Currently, mainstream optical positioning systems can be divided into positioning schemes based on photodiode arrays and positioning schemes based on computer vision according to different technical routes at the receiving end.

[0003] The photodiode array has a strong dependence on the model, and most indoor LEDs do not conform to the Lambert model; relying only on light intensity parameters, it is impossible to effectively utilize additional information such as the geometric characteristics of LEDs. The positioning scheme based on computer vision relies on prior information and requires a physical coordinate database of LED light sources to be established in advance; existing algorithms usually assume that LEDs are regular circular light sources and are difficult to handle point light sources and other non - regular - shaped LEDs.

[0004] Therefore, there is an urgent need for a new type of optical positioning system that can be compatible with arbitrarily shaped LED light sources and has both high precision and strong robustness to meet the positioning requirements in complex dynamic scenarios. Summary of the Invention

[0005] The present invention provides a camera - photodiode hybrid positioning method and system under an arbitrarily shaped light source. Aiming at the strong dependence of existing LED positioning technology on the Lambert light source model and regular shapes, as well as the insufficient anti - interference ability in complex environments, the present invention combines the visible light communication ability of the photodiode array with the spatial resolution advantage of the camera to achieve high - precision and high - robustness positioning of arbitrarily shaped LED light sources.

[0006] The first - aspect embodiment of the present invention provides a camera - photodiode hybrid positioning method under an arbitrarily shaped light source, including the following steps:

[0007] Periodically transmit the visible light communication information of the LED light source by using an arbitrarily shaped LED light source group;

[0008] Use a photodiode array to receive the visible light communication information sent by the LED light source, and calculate the polar angle and azimuth angle of each LED light source according to the visible light communication information;

[0009] Take an image of the LED light source group by a camera, extract the shape information of the LED light source according to the captured image, and calculate the polar angle and azimuth angle of each LED light source based on the pixel coordinates of the LED light source;

[0010] Match the polar angle and azimuth angle of each LED light source calculated by the photodiode array with the polar angle and azimuth angle of each LED light source calculated by the camera to obtain the correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information;

[0011] Construct a positioning problem using the correspondence between the 3D coordinates and 2D coordinates of the LED light source, and solve for the pose of the camera.

[0012] Optionally, in an embodiment of the present invention, the visible light communication information includes the shape, size, ID, and spatial layout information of the LED light source.

[0013] Optionally, in an embodiment of the present invention, calculate the polar angle of each LED light source according to the visible light communication information and azimuth angle as:

[0014] , ;

[0015] where , is the normalized vector of the incident vector, and its direction is from the photodiode array to the LED light source, is the maximum received power of the photodiode array, is the matrix composed of the normal vectors of the photodiodes, is the received signal of the photodiode array.

[0016] Optionally, in an embodiment of the present invention, denote the geometric center pixel coordinates of the LED light source as (u, v), the focal length of the camera as f, and the camera internal parameter matrix is obtained through camera calibration, where f x , f y are the lengths of the focal lengths of the camera in the x and y axis directions, in pixels u 0 and v 0 are the positions of the camera principal point. First, calculate the coordinates (x, y) of the LED light source in the camera plane:

[0017] ;

[0018] Calculate the polar angle and azimuth angle according to the coordinates of the LED light source in the camera plane is:

[0019] , .

[0020] Optionally, in an embodiment of the present invention, the polar angle and azimuth angle of each LED light source calculated by the photodiode array are matched with the polar angle and azimuth angle of each LED light source calculated by the camera to obtain the correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information, including:

[0021] Removing the unpaired and uniquely paired elements in the polar angle and azimuth angle vectors of each LED light source calculated by the photodiode array and the camera;

[0022] Calculating the matching cost between the i th LED light source estimated by the photodiode array and the j th LED light source estimated by the camera among the remaining elements, inputting the remaining elements of the polar angle and azimuth angle of each LED light source, the matching cost, and a set cost threshold into a matching algorithm for matching, combining the uniquely paired elements to obtain the polar angle and azimuth angle of each paired LED light source, and obtaining the correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information according to the polar angle and azimuth angle of each paired LED light source.

[0023] Optionally, in an embodiment of the present invention, a positioning problem is constructed by using the correspondence between the 3D coordinates and 2D coordinates of the LED light source to solve the pose of the camera, including:

[0024] According to the correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information, obtaining the world coordinates of each LED light source feature point calculated by the photodiode array and the pixel coordinates of the corresponding LED light source calculated by the camera;

[0025] Using the classical PnP algorithm or the EPnP algorithm to solve the world coordinates of each LED light source feature point and the pixel coordinates of the corresponding LED light source calculated by the camera according to the number of LED light source points to obtain the pose of the camera.

[0026] An embodiment of the second aspect of the embodiments of the present invention provides a camera - photodiode hybrid positioning system under an arbitrary - shaped light source for the camera - photodiode hybrid positioning method under an arbitrary - shaped light source in the above - mentioned embodiment. The system includes:

[0027] An LED light source group of arbitrary shape, which is used to periodically send the visible light communication information of the LED light source;

[0028] A photodiode array for receiving visible light communication information sent by an LED light source and calculating the polar angle and azimuth angle of each LED light source according to the visible light communication information;

[0029] A camera for taking an image of the LED light source group, extracting the shape information of the LED light source according to the taken image, and calculating the polar angle and azimuth angle of each LED light source according to the pixel coordinates of the LED light source;

[0030] A data matching module for matching the polar angle and azimuth angle of each LED light source calculated by the photodiode array with the polar angle and azimuth angle of each LED light source calculated by the camera to obtain the corresponding relationship between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information;

[0031] A positioning module for constructing a PnP problem by using the corresponding relationship between the 3D coordinates and 2D coordinates of the LED light source and solving the pose of the camera.

[0032] Optionally, in an embodiment of the present invention, the photodiode array is composed of a plurality of photodiodes with different direction gains, and different photodiodes have different tilting angles and are composed of lenses and gratings with different optical parameters.

[0033] Optionally, in an embodiment of the present invention, the photodiode array and the camera are integrally arranged and are in the same spatial position.

[0034] The camera-photodiode hybrid positioning method and system under an arbitrarily shaped light source according to the embodiment of the present invention combine the visible light communication ability of the photodiode array with the spatial resolution advantage of the camera to achieve high-precision and high-robustness positioning of arbitrarily shaped LED light sources, avoid the dependence on the Lambert model and the limitation of specific LED shapes in the traditional method, and significantly improve the indoor positioning problem based on various LEDs in a complex environment.

[0035] Additional aspects and advantages of the present invention will be given in part in the following description, will become apparent in part from the following description, or will be understood through the practice of the present invention. Description of the Drawings

[0036] The above and / or additional aspects and advantages of the present invention will become apparent and be easily understood from the following description of the embodiments in conjunction with the drawings, wherein:

[0037] Figure 1 It is a schematic flowchart of a camera-photodiode hybrid positioning method under an arbitrarily shaped light source according to an embodiment of the present invention;

[0038] Figure 2 Schematic diagram of a camera - photodiode hybrid positioning system under an arbitrary - shaped light source according to an embodiment of the present invention;

[0039] Figure 3 Schematic diagram of a combination of a photodiode array and a camera according to an embodiment of the present invention;

[0040] Figure 4 Flowchart of an improved KM matching algorithm according to an embodiment of the present invention;

[0041] Figure 5 Flowchart of a positioning algorithm according to an embodiment of the present invention. Detailed implementation manners

[0042] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present invention, and should not be construed as a limitation to the present invention.

[0043] Figure 1 Flow - schematic diagram of a method for camera - photodiode hybrid positioning under an arbitrary - shaped light source according to an embodiment of the present invention.

[0044] As Figure 1 shown, the method for camera - photodiode hybrid positioning under an arbitrary - shaped light source includes the following steps:

[0045] Step 1: Use an arbitrary - shaped LED light - source group to periodically send visible - light communication information of the LED light source.

[0046] Step 2: Use a photodiode array to receive the visible - light communication information sent by the LED light source, and calculate the polar angle and azimuth angle of each LED light source according to the visible - light communication information.

[0047] Step 3: Use a camera to capture an image of the LED light - source group, extract the shape information of the LED light source according to the captured image, and calculate the polar angle and azimuth angle of each LED light source according to the pixel coordinates of the LED light source.

[0048] Step 4: Match the polar angle and azimuth angle of each LED light source calculated by the photodiode array with the polar angle and azimuth angle of each LED light source calculated by the camera to obtain the corresponding relationship between the 3D coordinates of the LED light source in the visible - light communication information and the 2D coordinates of the LED light source in the image information.

[0049] Step 5: Use the corresponding relationship between the 3D coordinates and 2D coordinates of the LED light source to construct a positioning problem and solve for the camera pose.

[0050] By modulating the LED light source, the LED light source can periodically transmit visible light communication (VLC) information such as the shape, size, ID, and spatial layout of the LED. In an embodiment of the present invention, the LED light source can be of any shape, and the visible light communication information does not necessarily include shape information. Only carrying the ID information can also achieve optical positioning.

[0051] The LED light source in the embodiment of the present invention can be of any shape and is not limited to regular shapes. For example, Figure 2 as shown, there are light sources of various shapes on the ceiling, including cylindrical fluorescent tubes, rectangular LEDs, circular LEDs, and dot LEDs.

[0052] In the LED light source modulation section, the ID information, shape parameters, and other characteristic data of the LED light source are encoded into an optical communication signal through a driving circuit. Pulse width modulation (PWM) or frequency shift keying (FSK) technology is used to embed the data into the optical signal, and the information is transmitted by utilizing the high-frequency blinking characteristic of the LED. Common signal processing units with serial ports are microcontrollers (such as Atmega328P, 8051, STM3) or field programmable gate arrays (FPGAs). They receive instructions from the host computer through serial communication, generate corresponding modulation waveforms and output them to the LED driving module to ensure stable emission of the signal in the visible light band.

[0053] In an embodiment of the present invention, the photodiode array is composed of multiple photodiodes with different directional gains. The different photodiodes have different tilting angles and are composed of lenses and gratings with different optical parameters. The photodiode array and the camera module can be integrally arranged and closely combined, and can be considered to be at the same spatial position. For example, Figure 3 as shown, the photodiode array-camera combination is used as the receiving end.

[0054] In an embodiment of the present invention, the photodiode array receives the modulated optical signal emitted by the LED light source and its driving circuit to obtain an optoelectronic signal vector, demodulates various VLC information contained in the optical signal, and calculates the polar angle and azimuth angle of each LED light source as follows:

[0055] , ;

[0056] wherein, , is the normalized vector of the incident vector, and its direction is from the photodiode array to the LED light source. is the maximum received power of the photodiode array. is the matrix composed of the normal vectors of the photodiodes. The received signal of the photodiode array. And:

[0057] .

[0058] In an embodiment of the present invention, the camera captures an image of the LED light source, just one image, and processes the image, including binarization, contour extraction, dilation and erosion operations, shape estimation, etc. Then, an edge detection algorithm is used to detect the LED contour and estimate the LED shape. Denote the geometric center pixel coordinates of the LED light source as (u, v), the focal length of the camera as f, and the internal parameter matrix of the camera Obtained through camera calibration, where, f x 、 f y Are the lengths of the focal lengths of the camera in the x and y axis directions, in pixels u 0 and v 0 are the positions of the principal points of the camera. First, calculate the coordinates (x, y) of the LED light source in the camera plane:

[0059] ;

[0060] Calculate the polar angle 、azimuth angle as:

[0061] , .

[0062] In an embodiment of the present invention, the polar angle and azimuth angle of each LED light source calculated by the photodiode array are matched with the polar angle and azimuth angle of each LED light source calculated by the camera to obtain the correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information, including:

[0063] Remove the unpaired and uniquely paired elements in the polar angle and azimuth angle vectors of each LED light source calculated by the photodiode array and the camera;

[0064] Calculate the matching cost between the i th LED light source estimated by the photodiode array and the j th LED light source estimated by the camera among the remaining elements. Input the remaining elements of the polar angle and azimuth angle of each LED light source, the matching cost, and the set cost threshold into the matching algorithm for matching, and combine the uniquely paired elements to obtain the polar angle and azimuth angle of each paired LED light source. According to the polar angle and azimuth angle of each paired LED light source, obtain the correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information.

[0065] The matching method of the embodiment of the present invention is to use the shape information and the angle information, and use the improved KM algorithm to match the VLC information and the visual features in steps 2 and 3.

[0066] The flowchart of the matching algorithm is as Figure 4 shown. Denote the shape information demodulated by the photodiode array in step 2 as vector u:

[0067] ;

[0068] where u i represents the shape of the i-th light source demodulated by the photodiode array. Among them u i =1 is denoted as a point light source, u i =2 is denoted as a circular light source, u i =3 is denoted as a rectangular light source, u i =4 is a cylindrical light source, etc.

[0069] Denote the polar angle azimuth angle estimated by the photodiode array in step 2 as matrix A:

[0070] ;

[0071] where represents the polar angle of the i-th light source calculated by the photodiode array, represents the azimuth angle of the i-th light source estimated.

[0072] Denote the direction information estimated by the camera in step 3 as vector v:

[0073] ;

[0074] where v j represents the shape of the j-th light source estimated by the camera. u j The shape ratio of u i has one more 0, and 0 represents an uncertain shape, which can represent any shape.

[0075] Denote the polar angle azimuth angle estimated and calculated in step 3 as matrix B:

[0076] ;

[0077] where represents the polar angle of the j-th light source calculated by the camera, represents the azimuth angle of the j-th light source estimated.

[0078] The order and quantity of the LEDs obtained in Step 2 and Step 3 do not necessarily match, and the shapes of some of the LEDs estimated by the camera are uncertain. Therefore, the results of Step 2 and Step 3 need to be matched. The problem is formulated as a minimum-cost matching problem of a constrained weighted bipartite graph.

[0079] First, handle the constraints. Compare the elements in vectors u and v, filter out and remove the unpaired elements. Next, list separately the elements that can be uniquely paired and do not participate in the subsequent matching algorithm. Set a cost threshold h. If it is higher than the threshold, it also indicates an incorrect pairing.

[0080] Update the new shape vectors u, v, and the angle matrices A, B after removing the unpaired and uniquely paired elements.

[0081] Calculate the cost matrix C of the remaining elements according to the polar angle and azimuth angle. For the matching cost between the i-th LED estimated by the photodiode array and the j-th LED estimated by the camera in the cost matrix , the calculation process is as follows:

[0082] ;

[0083] where inf represents positive infinity, indicating an impossible match. According to the cost threshold h, assign the value of higher than h to inf as well.

[0084] Input the updated shape vectors u, v, the angle matrices A, B, and the cost matrix C into the KM algorithm to solve the pairing. Thus, the matching is successful. Generally, the number and types of light sources indoors are not many, and the complexity of the matching algorithm is very low.

[0085] In the embodiment of the present invention, after the matching is completed, based on the LED feature points, solve the camera pose. The positioning problem can be solved by classical algorithms such as PnP (perspective-n-point) or EPnP (Efficient PnP), and the photodiode array is used to assist in screening the unique solution in the case of multiple solutions. The flow chart of the positioning algorithm is as Figure 5 shown.

[0086] In the embodiment of the present invention, utilize the correspondence between the 3D coordinates and 2D coordinates of the LED light source to construct a positioning problem and solve the camera pose, including:

[0087] According to the correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information, obtain the world coordinates of each LED light source feature point calculated by the photodiode array and the pixel coordinates of the corresponding LED light source calculated by the camera;

[0088] According to the number of LED light source points, the world coordinates of each LED light source feature point and the pixel coordinates of the corresponding LED light source calculated by the camera are solved using the classical PnP algorithm or the EPnP algorithm to obtain the pose of the camera.

[0089] Specifically, through VLC, the layout of the LEDs is known. Denote the world coordinates of the light source feature points demodulated in VLC as , i The quantity of which is determined by the information in VLC. The above matching algorithm has already matched the VLC information with the feature points captured by the camera. Denote the pixel coordinates of the corresponding feature points in the camera as .

[0090] The 2D-3D one-to-one coordinate relationship of i points is obtained. According to the quantity and symmetry relationship of the feature points, it can be divided into three cases. (1) If i < 3, or all the feature points are collinear, then obviously there are infinite solutions and positioning cannot be achieved. (2) If i = 3 or the feature points are axisymmetric about a certain axis, then there are multiple solutions. For example, in the classical P3P problem, there are at most 4 real solutions, and the unique solution can be screened based on the received strength signal ratio (RSSR) of PD. Through calculation, the distance d i from the camera to the i-th LED and the distance d j from the camera to the j-th LED can be obtained. The ratio between them and the relationship with RSSR are as follows:

[0091] ;

[0092] Among them, P i represents the received intensity of the i-th LED; represents the polar angle of the i-th LED, which can be obtained from either step 2 or step 3; m represents the Lambert coefficient of the LED and can be obtained through measurement.

[0093] If the situation in (1) occurs, no solution can be obtained; if the situation in (2) occurs, PnP (Perspective-n-Point problem, a classical algorithm in the prior art for solving the camera pose based on 3D points and corresponding 2D image points) or EPnP (Efficient Perspective-n-Point problem, a prior art that improves the calculation efficiency through a linearization method based on PnP) first solves the pose, and then screens the unique solution according to RSSR; if neither of them occurs, it is the situation in (3), and the prior art of PnP or EPnP is directly used to solve, and the unique camera pose solution can be obtained.

[0094] Secondly, a camera-photodiode hybrid positioning system under a light source of any shape according to an embodiment of the present invention is described with reference to the accompanying drawings.

[0095] The camera - photodiode hybrid positioning system under an arbitrarily - shaped light source according to an embodiment of the present invention is used for the camera - photodiode hybrid positioning method under an arbitrarily - shaped light source in the above - mentioned embodiment. The camera - photodiode hybrid positioning system under an arbitrarily - shaped light source includes:

[0096] An arbitrarily - shaped LED light - source group for periodically transmitting visible - light communication information of the LED light source;

[0097] A photodiode array for receiving the visible - light communication information transmitted by the LED light source and calculating the polar angle and azimuth angle of each LED light source according to the visible - light communication information;

[0098] A camera for taking an image of the LED light - source group, extracting the shape information of the LED light source according to the taken image, and calculating the polar angle and azimuth angle of each LED light source according to the pixel coordinates of the LED light source;

[0099] A data - matching module for matching the polar angle and azimuth angle of each LED light source calculated by the photodiode array with the polar angle and azimuth angle of each LED light source calculated by the camera to obtain the corresponding relationship between the 3D coordinates of the LED light source in the visible - light communication information and the 2D coordinates of the LED light source in the image information;

[0100] A positioning module for constructing a PnP problem by using the corresponding relationship between the 3D coordinates and 2D coordinates of the LED light source and solving the pose of the camera.

[0101] For the specific layout, reference can be made to Figure 2 as shown. Among them, the data - matching module and the positioning module are not shown and can be implemented by using a mobile terminal. The sending and receiving of data can adopt conventional methods, and no specific limitation is made here.

[0102] In an embodiment of the present invention, the photodiode array is composed of multiple photodiodes with different direction gains. Different photodiodes have different tilting angles and are composed of lenses and gratings with different optical parameters.

[0103] In an embodiment of the present invention, the photodiode array and the camera are integrally arranged and are in the same spatial position.

[0104] It should be noted that the foregoing explanation of the embodiment of the camera - photodiode hybrid positioning method under an arbitrarily - shaped light source is also applicable to the camera - photodiode hybrid positioning system under an arbitrarily - shaped light source in this embodiment, and will not be elaborated here.

[0105] The camera-photodiode hybrid positioning method and system under an arbitrary-shaped light source according to an embodiment of the present invention transmit visible light communication information by modulating an LED light source, receive the visible light communication information by using a photodiode array, and calculate the polar angle and azimuth angle of each LED light source based on the visible light communication information; the camera synchronously captures an image of the LED light source, extracts the LED shape feature through image processing and calculates its polar angle and azimuth angle; a matching algorithm is adopted to match the data of the photodiode array and the camera; finally, positioning is performed by comprehensively using the visible light communication information and the visual feature of the camera. The present invention combines the advantages of a photodiode and a camera, solves the positioning problem of an arbitrary-shaped LED light source in a complex scenario, and is applicable to scenarios such as indoor optical communication, optical positioning, and computer vision.

[0106] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples.

[0107] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

Claims

1. A camera-photodiode hybrid positioning method under a light source of arbitrary shape, characterized in that Including the following steps: Periodically transmit the visible light communication information of the LED light source by using an LED light source of any shape; Receive the visible light communication information transmitted by the LED light source by using a photodiode array, and calculate the polar angle and azimuth angle of each LED light source according to the visible light communication information; Take an image of the LED light source by using a camera, extract the shape information of the LED light source according to the captured image, and calculate the polar angle and azimuth angle of each LED light source according to the pixel coordinates of the LED light source; Match the polar angle and azimuth angle of each LED light source calculated by the photodiode array with the polar angle and azimuth angle of each LED light source calculated by the camera to obtain the correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the shape information, specifically including: removing the unpaired and uniquely paired elements in the polar angle and azimuth angle of each LED light source calculated by the photodiode array and the camera; calculating the matching cost between the i th LED light source estimated by the photodiode array and the j th LED light source estimated by the camera among the remaining elements, input the remaining elements of the polar angle and azimuth angle of each LED light source, the matching cost, and the set cost threshold into the matching algorithm for matching, combine the uniquely paired elements to obtain the paired polar angle and azimuth angle of each LED light source, and obtain the correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the shape information according to the paired polar angle and azimuth angle of each LED light source; Construct a positioning problem by using the correspondence between the 3D coordinates and 2D coordinates of the LED light source, and solve the pose of the camera. Specifically, it includes: obtaining the world coordinates of each LED light source feature point calculated by the photodiode array and the pixel coordinates of the corresponding LED light source calculated by the camera according to the correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the shape information; solving the world coordinates of each LED light source feature point and the pixel coordinates of the corresponding LED light source calculated by the camera by using the classical PnP algorithm or EPnP algorithm according to the number of LED light source points to obtain the pose of the camera.

2. The method according to claim 1, wherein The visible light communication information includes the shape, size, ID, and spatial layout information of the LED light source.

3. The method according to claim 1, characterized in that, Calculate the polar angle of each LED light source according to the visible light communication information and the azimuth angle is as follows: ， ； Among them, , is the normalized vector of the incident vector, and its direction is from the photodiode array to the LED light source, is the maximum received power of the photodiode array, is the matrix composed of the normal vectors of the photodiodes, is the received signal of the photodiode array.

4. The method according to claim 1, wherein Record the geometric center pixel coordinates of the LED light source as (u, v), and the focal length of the camera is f. The camera internal parameter matrix is obtained through camera calibration, where f x , f y are the lengths of the focal lengths of the camera in the x and y axis directions, in pixels u 0 and v 0 are the positions of the camera principal point. First, calculate the coordinates (x, y) of the LED light source in the camera plane: ； Calculate the polar angle based on the coordinates of the LED light source in the camera plane , azimuth angle as follows: ， 。 5. A camera - photodiode hybrid positioning system under an arbitrary - shaped light source, for the camera - photodiode hybrid positioning method under an arbitrary - shaped light source according to any one of claims 1 - 4, characterized in that, The system includes: An LED light source of any shape, which is used to periodically transmit the visible light communication information of the LED light source; A photodiode array, which is used to receive the visible light communication information transmitted by the LED light source, and calculate the polar angle and azimuth angle of each LED light source according to the visible light communication information; A camera, which is used to take an image of the LED light source, extract the shape information of the LED light source according to the captured image, and calculate the polar angle and azimuth angle of each LED light source according to the pixel coordinates of the LED light source; A data matching module, which is used to match the polar angle and azimuth angle of each LED light source calculated by the photodiode array with the polar angle and azimuth angle of each LED light source calculated by the camera to obtain the correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the shape information; A positioning module, which is used to construct a PnP problem by using the correspondence between the 3D coordinates and 2D coordinates of the LED light source and solve the pose of the camera.

6. The system according to claim 5, characterized in that, The photodiode array is composed of multiple photodiodes with different direction gains, and different photodiodes have different tilting angles and are composed of lenses and gratings with different optical parameters.

7. The system according to claim 5, wherein The photodiode array and the camera are integrally arranged and are in the same spatial position.

Images