Camera-photodiode hybrid positioning method and system under light source of any shape

Abstract

The invention discloses a camera-photodiode hybrid positioning method and system under a light source in any shape, and belongs to the field of light positioning. The visible light communication information is transmitted by modulating the LED light sources, the visible light communication information is received by using the photodiode array, and the polar angle and the azimuth angle of each LED light source are calculated based on the visible light communication information; the camera synchronously shoots an LED light source image, LED shape features are extracted through image processing, and the polar angle and the azimuth angle of the LED shape features are calculated; matching the data of the photodiode array and the camera by adopting a matching algorithm; and finally, positioning by comprehensively utilizing visible light communication information and visual features of the camera. The method combines the advantages of the photodiode and the camera, solves the positioning problem of the LED light source in any shape in a complex scene, and is suitable for indoor optical communication, optical positioning, computer vision and other scenes.

Description

Technical Field

[0001] The present invention relates to the field of optical positioning technology, and in particular to a camera-photodiode hybrid positioning method and system under a light source of arbitrary shape. Background Art

[0002] Visible Light Positioning (VLP) technology uses the high-frequency modulation characteristics of LED light sources and combines optical sensors to achieve target position calculation. It has technical advantages such as high positioning accuracy, no electromagnetic pollution, and natural compatibility with lighting facilities. It has shown broad application prospects in indoor navigation, smart warehousing, industrial robots and other fields. At present, the mainstream optical positioning system can be divided into positioning solutions based on photodiode arrays and positioning solutions based on computer vision according to the different technical routes of the receiving end.

[0003] Photodiode arrays are highly dependent on models, and most indoor LEDs do not conform to the Lambertian model; they rely only on light intensity parameters and cannot effectively utilize additional information such as the geometric characteristics of LEDs. Positioning solutions based on computer vision rely on prior information and require the establishment of a physical coordinate database of LED light sources in advance; existing algorithms usually assume that LEDs are regular circular light sources, and it is difficult to handle point light sources and other irregular-shaped LEDs.

[0004] Therefore, there is an urgent need for a new optical positioning system that is compatible with LED light sources of any shape and has both high precision and strong robustness to meet the positioning requirements in complex dynamic scenes. Summary of the invention

[0005] The present invention provides a camera-photodiode hybrid positioning method and system under a light source of arbitrary shape. In view of the strong dependence of existing LED positioning technology on the Lambertian light source model and regular shape, and the insufficient anti-interference ability in complex environments, the present invention combines the visible light communication capability of the photodiode array with the spatial resolution advantage of the camera to achieve high-precision and high-robustness positioning of LED light sources of arbitrary shapes.

[0006] A first aspect of an embodiment of the present invention provides a camera-photodiode hybrid positioning method under a light source of any shape, comprising the following steps: Using an LED light source group of any shape to periodically send visible light communication information of the LED light source; Using a photodiode array to receive visible light communication information sent by an LED light source, and calculating the polar angle and azimuth angle of each LED light source according to the visible light communication information; Using a camera to capture an image of the LED light source group, extracting shape information of the LED light source according to the captured image, and calculating the polar angle and azimuth angle of each LED light source according to the pixel coordinates of the LED light source; Matching the polar angle and azimuth of each LED light source calculated by the photodiode array with the polar angle and azimuth of each LED light source calculated by the camera to obtain a correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information; The corresponding relationship between the 3D coordinates and the 2D coordinates of the LED light source is used to construct a positioning problem and solve the camera pose.

[0007] Optionally, in one embodiment of the present invention, the visible light communication information includes shape, size, ID and spatial layout information of the LED light source.

[0008] Optionally, in one embodiment of the present invention, the polar angle of each LED light source is calculated according to the visible light communication information. With azimuth for: , ; in, , is the normalized vector of the incident vector, which is directed from the photodiode array to the LED light source, is the maximum received power of the photodiode array, is the matrix of the normal vectors of the photodiode, is the received signal of the photodiode array.

[0009] Optionally, in one embodiment of the present invention, the geometric center pixel of the LED light source is denoted as (u, v), the focal length of the camera is f, and the camera intrinsic parameter matrix is It is obtained by camera calibration, where f x , f y It is the focal length of the camera in the x and y axis directions, in pixels. u 0 and v 0 is the position of the camera's principal point. First calculate the coordinates (x, y) of the LED light source on the camera plane: ; Calculate the polar angle based on the coordinates of the LED light source on the camera plane , azimuth for: , .

[0010] Optionally, in one embodiment of the present invention, the polar angle and azimuth of each LED light source calculated by the photodiode array are matched with the polar angle and azimuth of each LED light source calculated by the camera to obtain a correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information, including: Remove unpaired and uniquely paired elements from the polar angle and azimuth angle vectors of each LED light source calculated by the photodiode array and the camera; Calculate the remaining elements in the photodiode array estimated by i The LED light source is connected to the camera estimated j The matching cost of each LED light source is calculated, and the remaining elements of the polar angle and azimuth angle of each LED light source, the matching cost, and the set cost threshold are input into the matching algorithm for matching, and the polar angle and azimuth angle of each paired LED light source are obtained by combining the unique paired elements. According to the polar angle and azimuth angle of each paired LED light source, the corresponding relationship between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information is obtained.

[0011] Optionally, in one embodiment of the present invention, a positioning problem is constructed using the correspondence between the 3D coordinates and the 2D coordinates of the LED light source to solve the camera pose, including: According to the correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information, obtaining the world coordinates of each LED light source feature point calculated by the photodiode array and the pixel coordinates of the corresponding LED light source calculated by the camera; According to the number of LED light source points, the world coordinates of each LED light source feature point and the pixel coordinates of the corresponding LED light source calculated by the camera are solved using the classic PnP algorithm or the EPnP algorithm to obtain the position and posture of the camera.

[0012] A second aspect of an embodiment of the present invention provides a camera-photodiode hybrid positioning system under a light source of any shape, which is used in the camera-photodiode hybrid positioning method under a light source of any shape in the above embodiment. The system includes: An LED light source group of any shape, used to periodically send visible light communication information of the LED light source; A photodiode array, used to receive visible light communication information sent by the LED light source, and calculate the polar angle and azimuth angle of each LED light source according to the visible light communication information; A camera, used to capture an image of the LED light source group, extract shape information of the LED light source according to the captured image, and calculate the polar angle and azimuth angle of each LED light source according to the pixel coordinates of the LED light source; A data matching module, used to match the polar angle and azimuth of each LED light source calculated by the photodiode array with the polar angle and azimuth of each LED light source calculated by the camera, to obtain a corresponding relationship between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information; The positioning module is used to construct a PnP problem using the correspondence between the 3D coordinates and the 2D coordinates of the LED light source to solve the camera posture.

[0013] Optionally, in one embodiment of the present invention, the photodiode array is composed of a plurality of photodiodes with different directional gains, different photodiodes have different inclination angles, and are composed of lenses and gratings with different optical parameters.

[0014] Optionally, in one embodiment of the present invention, the photodiode array and the camera are integrated and located in the same spatial position.

[0015] The camera-photodiode hybrid positioning method and system under arbitrary shape light sources of the embodiments of the present invention achieve high-precision and high-robustness positioning of LED light sources of arbitrary shapes by combining the visible light communication capability of the photodiode array with the spatial resolution advantage of the camera, avoiding the traditional method's reliance on the Lambertian model and restrictions on specific LED shapes, and significantly improving indoor positioning problems based on various LEDs in complex environments.

[0016] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which: Figure 1 A schematic diagram of the flow of a camera-photodiode hybrid positioning method under a light source of any shape provided according to an embodiment of the present invention; Figure 2 A schematic diagram of a camera-photodiode hybrid positioning system under a light source of arbitrary shape according to an embodiment of the present invention; Figure 3 is a schematic diagram of a photodiode array and a camera combination according to an embodiment of the present invention; Figure 4 A flowchart of an improved KM matching algorithm according to an embodiment of the present invention; Figure 5 The figure is a flow chart of a positioning algorithm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0019] Figure 1 The present invention is a flowchart of a camera-photodiode hybrid positioning method under a light source of arbitrary shape provided by an embodiment of the present invention.

[0020] like Figure 1 As shown, the camera-photodiode hybrid positioning method under an arbitrary shape light source includes the following steps: Step 1: Use an LED light source group of any shape to periodically send visible light communication information of the LED light source.

[0021] Step 2: Use a photodiode array to receive visible light communication information sent by the LED light source, and calculate the polar angle and azimuth angle of each LED light source based on the visible light communication information.

[0022] Step 3, using a camera to capture an image of the LED light source group, extracting shape information of the LED light source based on the captured image, and calculating the polar angle and azimuth angle of each LED light source based on the pixel coordinates of the LED light source.

[0023] Step 4, match the polar angle and azimuth of each LED light source calculated by the photodiode array with the polar angle and azimuth of each LED light source calculated by the camera to obtain the correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information.

[0024] Step 5: Use the correspondence between the 3D coordinates and 2D coordinates of the LED light source to construct a positioning problem and solve the camera pose.

[0025] By modulating the LED light source, the LED light source can periodically send visible light communication (VLC) information such as the shape, size, ID, and spatial layout of the LED. In the embodiment of the present invention, the LED light source can be of any shape, and the visible light communication information does not have to include shape information. Light positioning can be achieved by only carrying ID information.

[0026] The LED light source of the embodiment of the present invention can be of any shape and is not limited to a regular shape. Figure 2As shown, there are light sources of various shapes on the ceiling, including cylindrical tubes, rectangular LEDs, circular LEDs, and point-shaped LEDs.

[0027] In the LED light source modulation part, the ID information, shape parameters and other characteristic data of the LED light source are encoded into optical communication signals through the driving circuit, and the data is embedded in the optical signal using pulse width modulation (PWM) or frequency shift keying (FSK) technology, and the high-frequency flickering characteristics of the LED are used to realize information transmission. Common signal processing units with serial ports are single-chip microcomputers (such as Atmega328P, 8051, STM3) or large-scale programmable gate arrays (Field Programmable Gate Array, FPGA), which receive host computer instructions through serial port communication, generate corresponding modulation waveforms and output them to the LED driver module to ensure stable signal transmission in the visible light band.

[0028] In the embodiment of the present invention, the photodiode array is composed of photodiodes with different directional gains, different photodiodes have different inclination angles, and are composed of lenses and gratings with different optical parameters. The photodiode array and the camera module can be integrated and tightly combined, and can be considered to be in the same spatial position, such as Figure 3 As shown, the photodiode array-camera combination serves as the receiving end.

[0029] In an embodiment of the present invention, the photodiode array receives the modulated light signal emitted by the LED light source and its driving circuit to obtain a photoelectric signal vector, demodulates various VLC information contained in the light signal, and calculates the polar angle of each LED light source according to the visible light communication information. With azimuth for: , ; in, , is the normalized vector of the incident vector, which is directed from the photodiode array to the LED light source, is the maximum received power of the photodiode array, is the matrix of the normal vectors of the photodiode, is the received signal of the photodiode array. And: .

[0030] In the embodiment of the present invention, the camera takes a single image of the LED light source and processes the image, including binarization, contour extraction, dilation and corrosion operations, shape estimation, etc. Then the edge detection algorithm is used to detect the LED contour and estimate the LED shape. The geometric center pixel of the LED light source is denoted as (u, v), the focal length of the camera is f, and the camera internal parameter matrix is It is obtained by camera calibration, where f x , f y It is the focal length of the camera in the x and y axis directions, in pixels. u 0 and v 0 is the position of the camera's principal point. First calculate the coordinates (x, y) of the LED light source on the camera plane: ; Calculate the polar angle based on the coordinates of the LED light source on the camera plane , azimuth for: , .

[0031] In an embodiment of the present invention, the polar angle and azimuth of each LED light source calculated by the photodiode array are matched with the polar angle and azimuth of each LED light source calculated by the camera to obtain the corresponding relationship between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information, including: Remove the unpaired and uniquely paired elements from the polar angle and azimuth angle vectors of each LED light source calculated by the photodiode array and the camera; Calculate the estimated first element of the photodiode array in the remaining i The LED light source and the camera estimate j The matching cost of each LED light source is calculated, and the remaining elements of the polar angle and azimuth angle of each LED light source, the matching cost, and the set cost threshold are input into the matching algorithm for matching. The polar angle and azimuth angle of each paired LED light source are obtained by combining the unique paired elements. According to the polar angle and azimuth angle of each paired LED light source, the corresponding relationship between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information is obtained.

[0032] The matching method of the embodiment of the present invention utilizes shape information and angle information and uses an improved KM algorithm to match the VLC information and visual features in step 2 and step 3.

[0033] The flowchart of the matching algorithm is as follows Figure 4 As shown. The shape information demodulated by the photodiode array in step 2 is recorded as vector u: ; in, u i represents the shape of the i-th light source demodulated by the photodiode array. u i =1 is a point light source,u i =2 is a circular light source, u i =3 is recorded as a rectangular light source, u i =4 cylindrical light sources, etc.

[0034] The polar angle azimuth estimated by the photodiode array in step 2 is recorded as matrix A: ; in, represents the polar angle of the ith light source calculated by the photodiode array, Represents the estimated azimuth of the i-th light source.

[0035] The direction information estimated by the camera in step 3 is recorded as vector v: ; in, v j Represents the shape of the j-th light source estimated by the camera. u j Shape ratio u i One more 0, 0 represents an uncertain shape, which can represent any shape.

[0036] The polar angle azimuth estimated in step 3 is recorded as matrix B: ; in, represents the polar angle of the jth light source calculated by the camera, Represents the estimated azimuth of the j-th light source.

[0037] The order and number of LEDs obtained in step 2 and step 3 do not necessarily match, and the shapes of some LEDs estimated by the camera are uncertain. Therefore, the results of step 2 and step 3 need to be matched. The problem is constructed as a constrained weighted bipartite graph minimum cost matching problem.

[0038] First, process the constraints. Compare the elements in vectors u and v, filter out unpaired elements and remove them. Next, list the elements that can be uniquely paired separately and do not participate in the subsequent matching algorithm. Set a cost threshold h. If it is higher than the threshold, it means it is not a correct pairing.

[0039] Update and remove the new shape vectors u, v, and angle matrices A, B that have no pairings and the only pairings.

[0040] The cost matrix C of the remaining elements is calculated based on the polar angle and azimuth angle. For the cost matrix, the matching cost of the i-th LED estimated by the photodiode array and the j-th LED estimated by the camera is , the calculation process is as follows: ; Inf means positive infinity, which means that there is no match. The value of is also assigned to inf.

[0041] The updated shape vectors u, v, angle matrices A, B, and cost matrix C are input into the KM algorithm to solve the pairing. The matching is successful. Usually, the number and types of indoor light sources are not large, and the complexity of the matching algorithm is very low.

[0042] In the embodiment of the present invention, after the matching is completed, the camera pose is solved based on the LED feature points. The positioning problem can be solved using algorithms such as classic PnP (pespective-n-point) or EPnP (Efficient PnP), and a photodiode array is used to assist in screening the only solution in the case of multiple solutions. The flowchart of the positioning algorithm is as follows: Figure 5 shown.

[0043] In an embodiment of the present invention, the corresponding relationship between the 3D coordinates and the 2D coordinates of the LED light source is used to construct a positioning problem and solve the camera posture, including: According to the correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information, the world coordinates of each LED light source feature point calculated by the photodiode array and the pixel coordinates of the corresponding LED light source calculated by the camera are obtained; According to the number of LED light source points, the world coordinates of each LED light source feature point and the pixel coordinates of the corresponding LED light source calculated by the camera are solved using the classic PnP algorithm or EPnP algorithm to obtain the camera's position and posture.

[0044] Specifically, the layout of the LED is known through VLC, and the world coordinates of the light source feature points demodulated in VLC are recorded as , i The number of is determined by the information in VLC. The above matching algorithm has matched the VLC information with the feature points captured by the camera. The pixel coordinates of the corresponding feature points in the camera are recorded as .

[0045] The 2D-3D coordinate relationship of i points is obtained. According to the number and symmetry of feature points, it can be divided into three cases. (1) If i < 3, or all feature points are collinear, then there are obviously infinite solutions and positioning is impossible. (2) If i = 3 or the feature points are symmetric about an axis, then there are multiple solutions. For example, there are at most 4 real solutions in the classic P3P problem. The unique solution can be selected based on the received intensity signal ratio (RSSR) of the PD. After calculation, the distance from the camera to the i-th LED can be obtained. d i Distance from camera to the jth LED d j Ratio , and the relationship between RSSR: ; in, P i represents the receiving intensity of the i-th LED; represents the polar angle of the ith LED, which can be obtained from step 2 or step 3; m represents the Lambert coefficient of the LED, which can be obtained by measurement.

[0046] If the situation in (1) occurs, the solution cannot be obtained. If the situation in (2) occurs, PnP (Perspective n-point Problem, a classic algorithm in the prior art for solving the camera pose based on 3D points and corresponding 2D image points) or EPnP (Efficient Perspective n-point Problem, an existing technology that improves computational efficiency through linearization methods based on PnP) is used to solve the pose first, and then the unique solution is screened based on RSSR. If neither of the two is true, the situation in (3) is obtained, and the existing PnP or EPnP technology is directly used to solve the problem, and a unique camera pose solution can be obtained.

[0047] Next, the camera-photodiode hybrid positioning system under a light source of arbitrary shape proposed in an embodiment of the present invention is described with reference to the accompanying drawings.

[0048] The camera-photodiode hybrid positioning system under a light source of any shape according to an embodiment of the present invention is used for the camera-photodiode hybrid positioning method under a light source of any shape according to the above embodiment. The camera-photodiode hybrid positioning system under a light source of any shape includes: An LED light source group of any shape, used to periodically send visible light communication information of the LED light source; A photodiode array is used to receive visible light communication information sent by the LED light source, and calculate the polar angle and azimuth angle of each LED light source according to the visible light communication information; A camera is used to capture an image of the LED light source group, extract shape information of the LED light source according to the captured image, and calculate the polar angle and azimuth angle of each LED light source according to the pixel coordinates of the LED light source; A data matching module is used to match the polar angle and azimuth of each LED light source calculated by the photodiode array with the polar angle and azimuth of each LED light source calculated by the camera, so as to obtain the corresponding relationship between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information; The positioning module is used to construct the PnP problem using the correspondence between the 3D coordinates and 2D coordinates of the LED light source to solve the camera pose.

[0049] For specific layout, please refer to Figure 2 As shown, the data matching module and the positioning module are not shown, and can be implemented using a mobile terminal. The sending and receiving of data can be carried out in a conventional manner, and no specific limitation is made to this.

[0050] In an embodiment of the present invention, the photodiode array is composed of a plurality of photodiodes with different directional gains, different photodiodes have different inclination angles, and are composed of lenses and gratings with different optical parameters.

[0051] In an embodiment of the present invention, the photodiode array and the camera are integrated and located at the same spatial position.

[0052] It should be noted that the aforementioned explanation of the embodiment of the camera-photodiode hybrid positioning method under a light source of any shape is also applicable to the camera-photodiode hybrid positioning system under a light source of any shape of this embodiment, and will not be repeated here.

[0053] According to the camera-photodiode hybrid positioning method and system under an arbitrary shape light source proposed in an embodiment of the present invention, visible light communication information is transmitted by modulating the LED light source, the visible light communication information is received by a photodiode array, and the polar angle and azimuth of each LED light source are calculated based on the visible light communication information; the camera synchronously captures the LED light source image, extracts the LED shape features through image processing and calculates its polar angle and azimuth; a matching algorithm is used to match the data of the photodiode array and the camera; finally, the visible light communication information and the visual features of the camera are comprehensively used for positioning. The present invention combines the advantages of photodiodes and cameras, solves the positioning problem of LED light sources of arbitrary shapes in complex scenes, and is suitable for scenes such as indoor optical communications, optical positioning, and computer vision.

[0054] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0055] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

Claims

1. A camera-photodiode hybrid positioning method under a light source of arbitrary shape, characterized in that: The following steps are involved: Using an LED light source group of any shape to periodically send visible light communication information of the LED light source; Using a photodiode array to receive visible light communication information sent by an LED light source, and calculating the polar angle and azimuth angle of each LED light source according to the visible light communication information; Using a camera to capture an image of the LED light source group, extracting shape information of the LED light source according to the captured image, and calculating the polar angle and azimuth angle of each LED light source according to the pixel coordinates of the LED light source; Matching the polar angle and azimuth of each LED light source calculated by the photodiode array with the polar angle and azimuth of each LED light source calculated by the camera to obtain a correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information; The corresponding relationship between the 3D coordinates and the 2D coordinates of the LED light source is used to construct a positioning problem and solve the camera pose.

2. The method according to claim 1, characterized in that The visible light communication information includes the shape, size, ID and spatial layout information of the LED light source.

3. The method according to claim 1, characterized in that Calculate the polar angle of each LED light source according to the visible light communication information With azimuth for: ， ； in, , is the normalized vector of the incident vector, which is directed from the photodiode array to the LED light source, is the maximum received power of the photodiode array, is the matrix of the normal vectors of the photodiode, is the received signal of the photodiode array.

4. The method according to claim 1, characterized in that: The geometric center pixel of the LED light source is (u, v), the focal length of the camera is f, and the camera internal parameter matrix is It is obtained by camera calibration, where f x , f y It is the focal length of the camera in the x and y axis directions, in pixels. u 0 and v 0 is the position of the camera's principal point. First calculate the coordinates (x, y) of the LED light source on the camera plane: ； Calculate the polar angle based on the coordinates of the LED light source on the camera plane , azimuth for: ， 。 5. The method according to claim 1, characterized in that Matching the polar angle and azimuth of each LED light source calculated by the photodiode array with the polar angle and azimuth of each LED light source calculated by the camera to obtain a correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information, including: Remove unpaired and uniquely paired elements from the polar angle and azimuth angle vectors of each LED light source calculated by the photodiode array and the camera; Calculate the remaining elements in the photodiode array estimated by i The LED light source is connected to the camera estimated j The matching cost of each LED light source is calculated, and the remaining elements of the polar angle and azimuth angle of each LED light source, the matching cost, and the set cost threshold are input into the matching algorithm for matching, and the polar angle and azimuth angle of each paired LED light source are obtained by combining the unique paired elements. According to the polar angle and azimuth angle of each paired LED light source, the corresponding relationship between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information is obtained.

6. The method according to claim 1, characterized in that The corresponding relationship between the 3D coordinates and the 2D coordinates of the LED light source is used to construct a positioning problem and solve the camera pose, including: According to the correspondence between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information, obtaining the world coordinates of each LED light source feature point calculated by the photodiode array and the pixel coordinates of the corresponding LED light source calculated by the camera; According to the number of LED light source points, the world coordinates of each LED light source feature point and the pixel coordinates of the corresponding LED light source calculated by the camera are solved using the classic PnP algorithm or the EPnP algorithm to obtain the position and posture of the camera.

7. A camera-photodiode hybrid positioning system under a light source of arbitrary shape, used in the camera-photodiode hybrid positioning method under a light source of arbitrary shape according to any one of claims 1 to 6, characterized in that: The system includes: An LED light source group of any shape, used to periodically send visible light communication information of the LED light source; A photodiode array, used to receive visible light communication information sent by the LED light source, and calculate the polar angle and azimuth angle of each LED light source according to the visible light communication information; A camera, used to capture an image of the LED light source group, extract shape information of the LED light source according to the captured image, and calculate the polar angle and azimuth angle of each LED light source according to the pixel coordinates of the LED light source; A data matching module, used to match the polar angle and azimuth of each LED light source calculated by the photodiode array with the polar angle and azimuth of each LED light source calculated by the camera, to obtain a corresponding relationship between the 3D coordinates of the LED light source in the visible light communication information and the 2D coordinates of the LED light source in the image information; The positioning module is used to construct a PnP problem using the correspondence between the 3D coordinates and the 2D coordinates of the LED light source to solve the camera posture.

8. The system according to claim 7, characterized in that The photodiode array is composed of photodiodes with different directional gains, different photodiodes have different inclination angles, and are composed of lenses and gratings with different optical parameters.

9. The system according to claim 7, characterized in that The photodiode array and the camera are integrated and located at the same spatial position.

Images