A method for calibration-free camera indoor localization using aruco codes

Abstract

The application discloses a kind of methods for indoor positioning of uncalibrated camera using Aruco code.The application uses a special two-dimensional code mark-Aruco code to carry out the self-calibration of camera, and the mapping and camera positioning of indoor environment.Through the corresponding relationship of a group of Aruco code corner points under different viewing angles, the distortion coefficient of the camera is solved using epipolar geometry constraint.The camera intrinsic parameters are calculated using the special properties of essential matrix after removing the distortion of the image.In an indoor scene with a group of Aruco codes pasted on a wall and a ceiling, the positional relationship between all codes is calculated using the pose relationship between a single code and the camera, thereby establishing an indoor map and completing the indoor positioning task of the camera.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robot vision positioning. In particular, it relates to a method for indoor positioning of a non-calibrated camera using Aruco codes. BACKGROUND

[0002] With the rapid development of automatic driving, intelligent agriculture, warehouse logistics and other fields, the industry has put forward higher requirements for efficient and accurate indoor positioning of robots. Among the robot positioning algorithms, the visual positioning method has become the most important research direction in the industry due to its high accuracy. As one of the most important devices in computer vision, the acquisition of camera intrinsic parameters is the most critical prerequisite for scene three-dimensional reconstruction. The main methods for camera intrinsic parameter calibration at present are:

[0003] 1. The traditional calibration method needs to use a known shape, and usually a very regular 3D or 2D calibration template.

[0004] 2. The active vision calibration method does not require a precise calibration template, but requires the camera to make some special movements for calibration, such as pure rotation, pure translation, etc.

[0005] 3. The self-calibration method uses the correspondence between points in different views of the same scene, i.e. the epipolar geometry constraint, to solve the camera intrinsic parameters.

[0006] In method 1, the calibration template needs to be precisely machined, and the camera intrinsic parameters are solved by establishing the correspondence between the three-dimensional coordinates of the calibration template and the two-dimensional coordinates of the image pixels. The traditional calibration method represented by Zhang Zhengyou calibration method is the most accurate among the three methods, but the calibration template is not always available, which is the biggest obstacle of the traditional calibration method. The active vision calibration method in method 2 is often not complex in calculation, but the requirement for precise camera movement is relatively harsh, and such harsh calibration conditions are rarely met in use. In the case of online calibration without prior access to the calibration template, the self-calibration method in method 3 is the most ideal calibration method, but the current self-calibration research rarely includes solving the camera distortion coefficients.

[0007] In the aspects of mapping and positioning, SFM and SLAM have become a research hotspot in recent years, but their limitations lie in that if the indoor scene is extremely simple and cannot provide rich feature point information, it is difficult to perform feature matching and it is impossible to reconstruct the indoor scene. SUMMARY

[0008] In view of the deficiencies of the prior art, the present application provides a method for indoor positioning of a non-calibrated camera using Aruco codes.

[0009] To achieve the above technical purposes, the technical scheme of the present application is:

[0010] A method for indoor positioning of an uncalibrated camera using Aruco codes includes the following steps:

[0011] S101: Aruco codes are affixed to the walls and ceiling of the indoor experimental setting at preset intervals;

[0012] S102: Take photos of a certain part of the scene from at least three angles. Each photo must show the complete images of three common Aruco codes, and the corner points of each Aruco code must appear in the image. Use the local adaptive binarization algorithm to detect Aruco codes and obtain the IDs and corner coordinates of the Aruco codes in all images.

[0013] S103: Calculate the distortion coefficients using the epipolar geometric constraints provided by the corner coordinates obtained in S102, and use this distortion to perform distortion removal processing on all subsequent images.

[0014] S104: Using the distortion-removed image obtained in S103, recalculate the fundamental matrix, and then use the relationship between the fundamental matrix, the essential matrix, and the camera intrinsic parameters, as well as the special properties of the essential matrix, to solve for the camera intrinsic parameters;

[0015] S105: Capture video so that all Aruco codes in the experimental scene appear in at least one frame of the video, and construct a set of keyframes from the video stream;

[0016] S106: Construct a pairwise pose relationship graph of Aruco markers from the keyframe set as the initial pose graph; the initial pose relationship is not unique.

[0017] S107: Calculate the pose graph from the initial pose graph and find the optimal pose relationship between two co-visible Aruco codes;

[0018] S108: Select the best Aruco code as the origin of the world coordinate system;

[0019] S109: Perform pose graph optimization to obtain the final indoor scene map;

[0020] S110: When a new camera needs to participate in the localization task, proceed to steps S102-S104 to calculate its distortion coefficients and intrinsic parameters, and use the indoor scene map provided in S109, as well as the pose relationship between the camera and the Aruco markers within its field of view, to determine its own coordinates.

[0021] Further, in step S101, a suitable bit number of Aruco code set is selected according to the required number of Aruco codes: if the scene is not large, the required Aruco codes are less, and the 16-bit set can be selected; if the scene is larger, the required Aruco codes are more, and the 36-bit set can be selected. The Aruco codes need to be pasted with a reasonable distance (the camera can see no less than two Aruco codes at the same time).

[0022] Further, in step S102, three Aruco codes in the photos are in different planes; the Bernsen algorithm is applied to the obtained image for binaryzation processing.

[0023] Further, in step S103, two pictures are selected from step S102, and the distortion coefficient is calculated according to the corner point coordinates of the corresponding Aruco codes by applying the division model formula and the fundamental matrix constraint condition.

[0024]

[0025] wherein p is the coordinate of the pixel point under the non-distortion condition, x is the coordinate of the pixel point under the real condition, ||x|| is the distance of the pixel point to the distortion center, and λ is the distortion coefficient required to be solved in this step. 2

[0026] The fundamental matrix constraint condition is:

[0027] p′ T Fp=0

[0028] p is the pixel point coordinate of the first image, p' is the pixel point coordinate of the corresponding point of p in the second image, and F is the fundamental matrix of the two views; the distortion formula is substituted into the fundamental matrix constraint condition, and the equation group is established by using the corner point coordinates of all Aruco codes in the view, so that the distortion coefficient λ can be solved.

[0029] Further, in step S104, first, the distortion coefficient λ solved in step S103 is used to remove the distortion of all images, and the fundamental matrix is recalculated; the relationship among the fundamental matrix, the essential matrix and the camera intrinsic parameter is as follows:

[0030] F=K -T EK -1

[0031] wherein E is the essential matrix, and K represents the camera intrinsic parameter; and the essential matrix has the particularity that two eigenvalues are equal, and the other eigenvalue is 0, and accordingly, the camera intrinsic parameter K can be solved by using the optimization algorithm such as Levenberg-Marquarelt.

[0032] ​Further, in step S105, for each frame in the video stream, the distortion is removed using the distortion coefficients solved in step S103. Then the following flow is judged:

[0033] If the number of detected markers is less than 2, the current frame is abandoned.

[0034] Otherwise, the pose of the camera relative to each Aruco code is calculated using the PNP algorithm and the camera intrinsic solved in step S104. If an Aruco code with a certain id is detected for the first time, a KD-tree (a data structure used for space partitioning and information retrieval) for this id is constructed. The pose of the camera relative to this Aruco code at this time is added to its KD-tree; if an Aruco code with a certain id is not detected for the first time, but the current camera pose is significantly different from any pose in the KD-tree, the current camera pose is added to the KD-tree.

[0035] If any KD-tree for any id has changed in the current frame, the current frame is determined as a key frame and added to the key frame set frameSet.

[0036] Further, in step S106, from the key frame set obtained in step S105, the pose relationship between all Aruco codes that can be co-visible in the same frame view is calculated, and the calculation formula is:

[0037]

[0038] Where {m} represents the set of all used Aruco codes, {f ij represents the key frame subset that can simultaneously see the Aruco code with id i and the Aruco code with id j. If there is no key frame that can simultaneously see the marker i and the marker j, H ji is empty, otherwise, Where t represents the t-th frame of the key frame subset, H ji represents the transformation matrix between the marker i and the marker j, H cj represents the transformation matrix (camera extrinsic) calculated using the PNP algorithm that converts the coordinate system of the marker j to the camera coordinate system; H ci represents the transformation matrix that converts the coordinate system of the marker i to the camera coordinate system. Since t≥1, H ji is not unique. All matrices are stored and referred to as the initial pose graph.

[0039] Further, the purpose of step S107 is to calculate the best transformation matrix between the marker i and the marker j from the initial pose graph obtained in step S106; the definition of the re-projection error is as follows:

[0040]

[0041] Where Ψ represents the camera projection formula, and δ includes the camera intrinsic parameters and distortion coefficients. Let w be the transformation matrix of the camera relative to the marked coordinate system at keyframe t. k This represents the coordinates of the four corner points of the Aruco code in its own coordinate system (world coordinates, with the origin at the center of the code), and the actual side length is l. These represent the coordinates (image coordinates) of the four corner points of marker i at keyframe t;

[0042]

[0043]

[0044] To solve for the optimal transformation matrix between markers, the transformation reprojection error formula is defined as follows:

[0045]

[0046] The world coordinates of marker i are used, along with the t-th coordinates. ′ The transfer matrix in the keyframe is transferred to the coordinate system of marker j, then projected using the extrinsic parameters of marker j, the projected coordinates are calculated, and then the reprojection error is calculated; for example... Therefore, it is assumed that when representing the pose relationship between marker i and marker j, The effect is better than That is, the relationship between marker i and marker j calculated based on the 4th keyframe is better than that in the 3rd keyframe; when a certain If the sum of the transformation reprojection errors of all keyframes sharing the same view marker i and marker j is minimized, then the H is determined to be... ji This represents the optimal pose between marker i and marker j;

[0047]

[0048]

[0049] Store all The matrix is ​​called a pose graph.

[0050] Furthermore, in step S108, based on the pose graph obtained in S107, a loss graph describing the magnitude of the transformation reprojection error between the markers is established, with the node being id and the edge representing the magnitude of the transformation reprojection error. If there is no co-visibility relationship between two markers, the value of the edge is infinite.

[0051] The Floyd algorithm is used to find a mark, so that the path to all other marks is the shortest, that is, the sum of the conversion re-projection error is the smallest, and the mark is taken as the origin of the world coordinate system, the pose of all marks relative to the origin is calculated, and the pose graph is updated.

[0052] The pose of each key frame camera relative to the origin of the world coordinate system is calculated to obtain a camera pose graph.

[0053] Further, in step S109, according to the pose graph and the camera pose graph obtained in step S108, the gtsam tool is used to optimize the pose graph, and the final pose graph of the Aruco mark is obtained, and the whole mapping step is completed.

[0054] Further, in step S110, when a new camera participates in the positioning task, steps S102-S104 are performed to calculate the distortion coefficient and the intrinsic parameter, and the indoor scene map provided in S109 and the pose relationship between the Aruco mark in the field of view (at this time, the number of marks in the field of view is greater than or equal to one) are used to solve the coordinates in the world coordinate, and realize the positioning of the camera.

[0055] The beneficial technical effects of the present application are:

[0056] The method provided by the present application can use a camera with unknown intrinsic parameters for indoor positioning, without the need for manual camera calibration in advance, and has great popularization value in situations where a calibration device cannot be obtained. Through experimental testing, in an indoor room of 5.5m*3.5m, the average positioning error is 6cm, which can meet the accuracy requirements of most positioning scenes. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 The flowchart of the method embodiment of the present application is shown in the figure;

[0058] Figure 2 The experimental test scene diagram of the method of the present application is shown in the figure;

[0059] Figure 3 The schematic diagram of the key frame set in the method embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0060] In order to more specifically describe the present application, the technical solutions of the present application are further described in detail in combination with the drawings and specific embodiments.

[0061] The present application faces the scene requirements of indoor positioning, and proposes a method for indoor positioning using a non-calibration camera with unknown intrinsic parameters and Aruco code.

[0062] The basic theory of the application is the imaging principle of a monocular camera and epipolar geometry theory. A pinhole camera model is a projection model for describing mapping of a three-dimensional space point to a two-dimensional plane pixel point. A projection equation of the camera is as follows:

[0063]

[0064] For specific derivation, refer to Hartley R, Zisserman A. Multiple view geometry in computer vision [M]. Cambridge university press, 2003. Wherein, P w represents coordinates of a point in space in a world coordinate system, p represents pixel coordinates of P w after camera imaging, R and t describe a position of the camera in the world coordinate system, and are referred to as camera extrinsic parameters, and K is an inherent parameter of the camera, and is referred to as a camera intrinsic parameter, and K is shaped as follows:

[0065]

[0066] In most cases, camera pixels are square pixels, that is, f x =f y , and u0 and v0 are referred to as principal points. A process of solving the camera intrinsic parameter K is referred to as camera calibration.

[0067] A complete flowchart of the application is shown in Figure 1 .

[0068] In step S101, first, an indoor environment without occlusion needs to be selected, and an Aruco code is pasted. For an introduction of the Aruco code, refer to Garrido-Jurado S, , Salinas R, Madrid-Cuevas F J, et al. Automatic generation and detection of highly reliable fiducial markers under occlusion [J]. Pattern Recognition, 2014, 47(6): 2280-2292. A test experimental scene is shown in Figure 2 .

[0069] In step S102, the photo of a certain part of the scene is taken from at least three angles, and each photo needs to show the complete image of three common Aruco codes, and the corner points of each Aruco code appear in the image. In addition, these Aruco codes cannot be coplanar. When all Aruco codes are coplanar, the fundamental matrix will be degenerated, and the camera intrinsic parameters cannot be accurately calibrated. For the problem of fundamental matrix degeneration, see Hartley R, Zisserman A. Multiple view geometry in computer vision [M]. Cambridge university press, 2003.

[0070] The first step of Aruco code detection is image binarization, and then the black and white bit blocks in the image are extracted for decoding. The Bernsen algorithm is used for binarization processing in the present application. Compared with the global threshold method, the local adaptive binarization method represented by the Bernsen algorithm can still achieve ideal results in the uneven light environment. Compared with the general local adaptive binarization method, the biggest advantage is that the selection of the window size has less influence on the binarization result.

[0071] For detailed introduction of the Bernsen algorithm, see the paper J. Bernsen, “Dynamic Thresholding of Gray Level Image,” ICPR `86: Proceedings of International Conference on Pattern Recognition, Berlin, 1986, pp. 1251-1255.

[0072] Through experiments, it is found that selecting the parameters k = 50, S = 25, and TT = 100 of the Bernsen algorithm can achieve relatively ideal binarization effect under most daily lighting conditions. The black and white bit blocks in the image are extracted for decoding, and the id and corner point coordinates of the Aruco code in the image are obtained.

[0073] In step S103, two images are selected from step S102. The distortion coefficients are calculated by applying the distortion division model formula and the fundamental matrix constraints based on the corner coordinates of the corresponding Aruco codes. The division model formula is (see Fitzgibbon AW. Simultaneous linear estimation of multiple view geometry and lens distortion[C] / / Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.CVPR 2001.IEEE,2001,1:II.):

[0074]

[0075] Where p represents the coordinates of a pixel in an image under ideal, distortion-free conditions, and x represents the coordinates of a pixel in real-world conditions, ||x|| 2 λ represents the distance from the pixel to the distortion center, and λ is the distortion coefficient that needs to be solved in this step. Current research shows that although the distortion center and the image center do not completely coincide, their distance on the pixel plane is very small, and assuming the distortion center is at the image center has minimal impact on distortion removal. Therefore, this invention assumes the distortion center is the image center and only solves for λ. Regarding the differences and relationships between the image center, principal point, and distortion center, refer to the paper Willson RG, Shafer S A. What is the center of the image? [J]. JOSAA, 1994, 11(11): 2946-2955.

[0076] The fundamental matrix constraints are:

[0077] p′ T Fp = 0

[0078] Let p be the pixel coordinates of the first image, p′ be the pixel coordinates of the corresponding point of p in the second image, and F be the fundamental matrix of the two views. Representing p and p′ in homogeneous coordinates as [p,q,1] and [p′,q′,1] respectively, and expanding F into a column vector f, we obtain:

[0079] [p′p,p′q,p′,q′p,q′q,q′,p,q,1]·f=0

[0080] Representing x using homogeneous coordinates as [x, y, 1], according to the division model, we get:

[0081]

[0082] Substitute it into the fundamental matrix constraint condition, and obtain:

[0083] [x'x, x'y, x', y'x, y'y, y', x, y, 1] · f + λ [0, 0, x'r 2 , 0, 0, y'r 2 , xr' 2 , yr' 2 , r 2 + r' 2 ] · f + λ 2 [0, 0, 0, 0, 0, 0, 0, 0, r'r 2 r 2 ] · f = 0

[0084] Thus, a quadratic polynomial eigenvalue problem of the form (D1 + λD2 + λ 2 D3) · f = 0 is obtained, and the distortion coefficient λ is obtained by solving the quadratic polynomial eigenvalue problem.

[0085] In step S104, first, the distortion of all images is removed by using the distortion coefficient λ solved in step S103. The fundamental matrix is recalculated. The relationship among the fundamental matrix, the essential matrix and the camera intrinsic parameter is as follows:

[0086] F = K -T EK -1

[0087] The essential matrix and the fundamental matrix are both 3x3 matrices with rank 2. The essential matrix has a special property that one singular value is 0 and the other two singular values are equal. Based on this, the objective function C is established

[0088]

[0089] Wherein, n represents the number of images used for calibration, n(n-1) / 2 fundamental matrices can be obtained from n images, and trace is the trace of the matrix. The particle swarm algorithm is used to find the value of the camera intrinsic parameter K that makes C minimum, and the camera calibration step is completed.

[0090] In step S105, for each frame of image in the video stream, the distortion is removed by using the distortion coefficient solved in step S103. Then, the following flow judgment is performed:

[0091] 1) If the number of markers detected is less than 2, the constraint relationship between the markers cannot be provided, and the current frame is abandoned;

[0092] 2) If the number of detected markers is greater than or equal to 2, use the PNP algorithm and the camera intrinsic parameters solved in step S104 to calculate the pose of the camera relative to each Aruco code.

[0093] For the introduction of PNP algorithm, see the paper Collins T, Bartoli A. Infinitesimal plane-based pose estimation [J]. International journal of computer vision, 2014, 109(3): 252-286.

[0094] For each Aruco marker in the image, the following judgment is made:

[0095] a) If an Aruco code with a certain id is detected for the first time, construct the KD tree of this id. KD tree is a data structure used for space partitioning and information retrieval. See Bentley J L. Multidimensional binary search trees used for associative searching [J]. Communications of the ACM, 1975, 18(9):

[0096] 509-517 for details, and add the pose of the camera relative to this Aruco code at this time to its KD tree;

[0097] b) If an Aruco code with a certain id is not detected for the first time, but the current camera pose is significantly different from any pose in the KD tree, add the current camera pose to the KD tree.

[0098] If the KD tree of any id has changed in the current frame, determine that the current frame is a key frame and add it to the key frame set frameSet.

[0099] After traversing each frame of image in the video stream, the key frame set frameSet is obtained. A schematic diagram of a set of key frame set is shown in Figure 3 .

[0100] In step S106, from the key frame set obtained in step S105, the pose of each frame of image relative to each Aruco code in the image is calculated, and the rotation matrix and translation vector are and The superscript t and subscript i represent the pose of the t-th key frame camera relative to the Aruco code with id i, and and are combined into a 4x4 transition matrix:

[0101]

[0102] The pose relationship between all the Aruco codes that can be co-visible by the same frame view is calculated by the pose of the camera relative to the Aruco code, and the calculation formula is as follows:

[0103]

[0104] Where {m} represents the set of all Aruco codes used, {f ij} represents all the key frame subsets that can simultaneously see the Aruco code with id i and the Aruco code with id j. If there is no key frame that can simultaneously see the marker i and the marker j, H ji is empty; otherwise, Where t represents the calculation using the tth frame of the key frame subset, H ji represents the transformation matrix between the marker i and the marker j. Since t≥1, H ji is not unique. Storing all the matrices is called the initial pose graph.

[0105] In step S107, the purpose is to calculate the best transformation matrix between the marker i and the marker j from the initial pose graph obtained in step S106. The definition of the re-projection error is as follows:

[0106]

[0107] Where represents the camera projection formula, and δ contains the camera intrinsic parameters and distortion coefficients, is the transformation matrix (extrinsic parameters) of the camera relative to the marker i coordinate system under the tth key frame, w k represents the coordinates of the four corner points of the Aruco code in its own coordinate system (world coordinates, the origin of the coordinate system is at the center of the code), and the real side length is l, represents the four corner point coordinates of the marker i under the tth key frame (image coordinates).

[0108]

[0109]

[0110] In order to solve the best transformation matrix between the markers, the conversion re-projection error formula is defined as follows:

[0111]

[0112] The world coordinates of the marker i are converted to the coordinate system of the marker j using the conversion matrix under the tth ′ key frame, and then the projection coordinates are calculated by using the extrinsic parameters of the marker j, and the re-projection error is calculated. For example,​ The pose relationship between marker i and marker j is represented by H The effect of H That is, the pose relationship between marker i and marker j calculated according to the fourth key frame is better than that of the third key frame. When a certain The H ji The H

[0113]

[0114]

[0115] The matrix of all The matrix of all

[0116] In step S108, a loss graph describing the size of the transformation reprojection error between markers is established according to the pose graph obtained in S107, with the node being id and the edge being the size of the transformation reprojection error. If there is no co-view relationship between two markers, the value of the edge is infinite. Floyd algorithm is used to find a marker that has the shortest path to all other markers, i.e., the smallest sum of transformation reprojection errors, and it is used as the origin of the world coordinate system. For details of the Floyd algorithm, see Hougardy S. The Floyd-Warshall algorithm on graphs with negative cycles [J]. Information Processing Letters.

[0117] The pose of all markers relative to the origin is calculated, and the pose graph is updated. The pose of each key frame camera relative to the origin of the world coordinate system is calculated to obtain the camera pose graph.

[0118] In step S109, according to the pose graph and the camera pose graph obtained in step S108, gtsam tools are used for pose graph optimization, and the final pose graph of the Aruco marker is obtained, completing the entire mapping step.

[0119] In step S110, when a new camera needs to be positioned, steps S102-S104 are performed to calculate its distortion coefficient and intrinsic parameters, and the indoor scene map provided in S109 is used to solve the coordinates in the world coordinate system according to the pose relationship between the Aruco markers in its field of view, and the coordinates in the world coordinate system are obtained to realize its own positioning.

[0120] The above merely provides the specific embodiments of the present application, and cannot be used to limit the scope of the present application. Any equivalent changes made by those skilled in the art according to the present application, and the changes well known to those skilled in the art, shall still fall within the scope of the present application.

Claims

1. A method for calibration-free camera indoor localization using Aruco codes, characterized in that, The method comprises the following steps: S101: paste Aruco codes on the walls and ceiling of the indoor experimental scene at a preset interval; S102: take a photo of a certain part of the scene from at least three angles, and require that each photo shows the complete image of three common Aruco codes, and the corner points of each Aruco code appear in the image; perform Aruco code detection on the obtained images by using a local adaptive binary algorithm to obtain the id and corner point coordinates of the Aruco codes in the images; S103: calculate the distortion coefficient by using the epipolar geometry constraint provided by the corner point coordinates obtained in S102, and perform distortion removal processing on all subsequent images by using the distortion; S104: re-calculate the fundamental matrix by using the images obtained in S103, and then solve the camera intrinsic parameter by using the relationship among the fundamental matrix, the essential matrix and the camera intrinsic parameter, and the special properties of the essential matrix; S105: take a video so that all Aruco codes in the experimental scene appear in at least one frame of the video, and construct a key frame set from the video stream; S106: construct a pose relationship graph between two Aruco markers from the key frame set, as an initial pose graph; S107: calculate the pose graph from the initial pose graph, and find the best pose relationship between two Aruco codes that can be co-observed; S108: select the best Aruco code as the origin of the world coordinate system; establish a loss graph that describes the conversion reprojection error size between markers according to the pose graph obtained in S107, the node is id, the edge is the conversion reprojection error size, and if there is no co-observation relationship between two markers, the value of the edge is infinite; find a marker by using the Floyd algorithm so that the path from the marker to all other markers is the shortest, take the marker as the origin of the world coordinate system, calculate the pose of all markers relative to the origin, and update the pose graph; calculate the pose of each key frame camera relative to the origin of the world coordinate system to obtain a camera pose graph; S109: optimize the pose graph to obtain a final indoor scene map; S110: when a new camera needs to participate in the positioning task, perform steps S102-S104 to calculate the distortion coefficient and intrinsic parameter, and determine the own coordinate by using the indoor scene map provided in S109 and the pose relationship between the Aruco markers in the own field of view.

2. The method for calibration-free camera indoor positioning with Aruco codes according to claim 1, characterized in that, In step S102, the three common Aruco codes in the photo are on different planes; the Bernsen algorithm is applied to the obtained images for binary processing.

3. The method for calibration-free camera indoor positioning using Aruco codes according to claim 1, characterized in that, In step S103, two images are selected from step S102, the distortion division model formula and the fundamental matrix constraint condition are applied according to the corner point coordinates of the corresponding Aruco codes, and the distortion coefficient is calculated; the division model formula is: wherein, is the coordinate of the pixel point under the non-distortion condition, is the coordinate of the pixel point under the real condition, represents the distance from the pixel point to the distortion center, is the distortion coefficient required to be solved in this step; the fundamental matrix constraint condition is: is the pixel coordinate of the first image, is the pixel coordinate of the second image, is the pixel coordinate of the second image, is the fundamental matrix of the two views; by substituting the distortion formula into the constraint condition of the fundamental matrix and establishing an equation group using the corner coordinates of all Aruco codes in the view, the distortion coefficients can be solved .

4. The method for calibration-free camera indoor positioning with Aruco codes according to claim 3, characterized in that, In step S104, the distortion coefficients solved in step S103 are first used to remove the distortion of the image Remove the distortion of all images, and recalculate the fundamental matrix; the relationship among the fundamental matrix, the essential matrix, and the camera intrinsic parameters is as follows: wherein is the essential matrix, represents the camera intrinsic parameters; the essential matrix has the particularity that two eigenvalues are equal and the other eigenvalue is 0, and the camera intrinsic parameters are solved by using a nonlinear least square optimization algorithm .

5. The method for calibration-free camera indoor positioning using Aruco codes according to claim 1, characterized in that, The purpose of step S107 is to calculate the best transformation matrix between marker i and marker j from the initial pose graph obtained in step S106; the definition of the re-projection error is as follows: wherein, represents a camera projection formula, contains camera intrinsic parameters and distortion coefficients, is the first is the transformation matrix of the camera under the key frame relative to the coordinate system of the marker i, represents the coordinates of the four corner points of the Aruco code under its own coordinate system, and the real side length is , represents the fourth four corner point coordinates of the marker i under the key frame; In order to solve the best transformation matrix between markers, the conversion re-projection error formula is defined as follows: The world coordinates of marker i are used, using the... The transition matrix in the keyframe is transferred to the coordinate system of marker j, then projected using the extrinsic parameters of marker j, the projected coordinates are calculated, and then the reprojection error is calculated; when a certain If the sum of the transformation reprojection errors of all keyframes sharing the same view marker i and marker j is minimized, then the keyframe is determined to be... This represents the optimal pose between marker i and marker j; stores all The matrix of all is called pose graph.

Images