Fusion laser positioning long-distance two-dimensional code map creation and positioning method and system

Abstract

The application provides a long-distance two-dimensional code map creation and positioning method and system fusing laser positioning, wherein the map creation method comprises the following steps: setting a long-distance two-dimensional code at a look-up position of the top of a mapping area relative to a camera; establishing a laser map of the mapping area and obtaining laser positioning information; obtaining an image containing the long-distance two-dimensional code through the camera to extract the spatial pose of the long-distance two-dimensional code in a camera coordinate system; when it is judged that the image observed by the camera contains a first observed long-distance two-dimensional code, calculating the spatial position of the first observed long-distance two-dimensional code in the laser map and adding it to the long-distance two-dimensional code map. In this way, the creation and positioning of the long-distance two-dimensional code map fusing laser positioning are realized, and the adaptability of the positioning scene is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to indoor space mapping and positioning technology, and in particular to a long-distance two-dimensional code map creation and positioning method and system fusing laser positioning. BACKGROUND

[0002] Currently, the mainstream mobile robot mapping and positioning scheme usually adopts laser mapping technology, such as using the current laser point cloud to match the point cloud map to realize laser positioning. However, in the actual implementation process, due to the existence of scene change factors, such as in the process of stacking, unstacking, narrow channel, dynamic scene, etc., the environment and the blocking and interference of moving objects are easy to be affected, the scanning of the laser radar is blocked, and the robot movement mechanism often has the phenomenon of slipping, so that the pure laser positioning has certain limitations in the actual long-time operation and deployment.

[0003] Therefore, the inventors consider that the long-distance two-dimensional code positioning has the particularity that the long-distance two-dimensional code can be set in the unobstructed and open environment of the indoor ceiling, at this time, the camera is not easy to be affected by the ground environment in the process of shooting the long-distance two-dimensional code, and if the long-distance two-dimensional code is set in the area where the positioning is easy to be lost in the scene to assist in positioning in the artificial marking mode, the occurrence of the foregoing problems can be reduced or avoided, so as to obtain the positioning advantage in the scene. SUMMARY

[0004] Therefore, the main purpose of the present application is to provide a long-distance two-dimensional code map creation and positioning method and system fusing laser positioning, so as to realize the scheme proposed in the background and obtain the positioning advantage in the scene.

[0005] In order to achieve the above-mentioned purpose, according to the first aspect of the present application, a long-distance two-dimensional code map creation method fusing laser positioning is provided, and the steps include:

[0006] Step S100 sets the long-distance two-dimensional code at the upward looking position of the top of the mapping area relative to the camera; establishes the laser map of the mapping area, and obtains the laser positioning information;

[0007] Step S200 obtains the image containing the long-distance two-dimensional code through the camera, so as to extract the spatial pose of the long-distance two-dimensional code in the camera coordinate system;

[0008] Step S300 calculates the spatial position of the first observed long-distance two-dimensional code in the laser map when it is judged that the image observed by the camera contains the first observed long-distance two-dimensional code, and adds it to the long-distance two-dimensional code map.

[0009] In the possible preferred embodiment, in step S300, the step of calculating the spatial position of the first observed long-distance two-dimensional code in the laser map includes:

[0010] Step S310 fuses the laser positioning information and the spatial pose of the long-distance two-dimensional code in the camera coordinate system at all time points in the form of a factor graph to construct a maximum a posteriori problem to optimize the spatial position of each long-distance two-dimensional code in the laser map.

[0011] In a possible preferred embodiment, in step S310, the step of constructing a maximum a posteriori problem to optimize the spatial position of each long-distance two-dimensional code in the laser map comprises:

[0012] Step S311 sets the position of all long-distance two-dimensional codes in the laser map as , uses the measurement value to constrain the to-be-optimized variable , models the Bayesian network as , and solves the optimized state variable by maximum a posteriori:

[0013]

[0014] wherein the measurement value is a set of measurement values:

[0015] represents the observation of the camera on the th position of the mobile robot;

[0016] represents the th position of the mobile robot in the laser map;

[0017] represents the installation position of the camera in the vehicle body coordinate system.

[0018] In a possible preferred embodiment, the long-distance two-dimensional code comprises a positioning code and an information code, wherein the positioning code is arranged in a plurality of non-crossing directions around a center point by a plurality of direction points, a plurality of quadrant regions are divided at the angles between adjacent direction points, the information code is arranged in the corresponding quadrant region according to a preset code table to form a two-dimensional lattice with the positioning code, and the positioning code and the information code are provided with a reflective layer on the surface.

[0019] In a possible preferred embodiment, in step S200, the step of extracting the spatial pose of the long-distance two-dimensional code in the camera coordinate system comprises:

[0020] Step S210 performs edge extraction after performing binaryzation on the image containing the long-distance two-dimensional code information.

[0021] Step S220 performs ellipse fitting detection according to the extracted edge information to obtain the original code and the center coordinate thereof.​

[0022] Step S230 locates the distribution position of the positioning code corresponding to each long-distance two-dimensional code in the original code according to the geometric relationship of the positioning code, to screen the information code in the original code, and acquires the ID information of the long-distance two-dimensional code according to the general code table;

[0023] Step S240, when judging that the long-distance two-dimensional code under the ID appears for the first time, calculates the spatial pose of the corresponding long-distance two-dimensional code in the camera coordinate system according to the center coordinates of the positioning code in the image and binds it with the ID information.

[0024] In a possible preferred embodiment, the step of calculating the spatial pose of the long-distance two-dimensional code in the camera coordinate system in step S240 comprises:

[0025] Step S241 sets the center coordinates of the positioning code as , and establishes the homogeneous matrix

[0026]

[0027] Solve the homography matrix H based on the SVD method, wherein u and v represent the pixel coordinates of the center of the positioning code, represent the coordinates of the point in the coordinate system of the long-distance two-dimensional code, and s is the equivalent distance scale factor;

[0028] Step S242, according to the relationship between the homography matrix and the conversion matrix of the long-distance two-dimensional code in the camera coordinate system

[0029]

[0030] Obtain the rotation and translation matrix, wherein P is the camera projection matrix, E is the truncated extrinsic matrix, , are the focal lengths of the camera direction and direction respectively, , is the camera center point coordinate, are the first two columns in the rotation matrix, , , respectively represent the position of the center of the long-distance two-dimensional code in the camera coordinate system, is a 3x3 homographic projection matrix.

[0031] In a possible preferred embodiment, step S240 further comprises:

[0032] Step 243 solves the minimum error function

[0033]

[0034] Ri, ti = optimizeRotationAndTranslation(R, t) where R denotes rotation and t denotes translation

[0035]

[0036]

[0037] denotes the i-th element in the translation vector, denotes the i-th column in the rotation matrix; where denotes the center of the i-th far-distance QR code in the image ; denotes the center of the i-th far-distance QR code in the image ; denotes the center of the i-th far-distance QR code in the image ; denotes the center of the i-th far-distance QR code in the image ; denotes the center of the i-th far-distance QR code in the image

[0038] where the constraint condition is

[0039]

[0040]

[0041] In a possible preferred embodiment, the step of extracting the spatial pose of the far-distance QR code in the camera coordinate system in step S200 further comprises:

[0042] Step S250 constrains the spatial pose obtained in step S240 according to time filtering and prior pose constraints, wherein the time filtering step comprises: filtering out when it is judged that the spatial pose result has a mutation; and the prior pose constraint step comprises: filtering out when it is judged that the far-distance QR code coordinate system and the camera coordinate system in the spatial pose result are not parallel.

[0043] In order to achieve the above-mentioned purpose, corresponding to the above-mentioned map creation method, the second aspect of the present application also provides a far-distance QR code map positioning method fusing laser positioning, the steps of which comprise:

[0044] Step S400, when the laser map and the far-distance QR code map are mixed in the positioning area, the accuracy of the positioning information is described by using the confidence, the laser positioning confidence and the far-distance QR code positioning confidence are set respectively, and the positioning information is obtained by weighted fusion according to the size of the respective positioning confidence:

[0045] ​

[0046]

[0047] wherein is the position at the time t, represents the confidence of the current hybrid positioning, is the position of the previous time positioning, is the position of the current time positioning using the camera, is the position of the current time positioning using the laser;

[0048] When the position is in the remote two-dimensional code map positioning area, the ID information of all remote two-dimensional codes in the current scene image is extracted by the camera to match the map to obtain the positioning information.

[0049] In order to achieve the above purpose, corresponding to the above map creation method and positioning method, the third aspect of the present application also provides a remote two-dimensional code map creation and positioning system fused with laser positioning, which comprises:

[0050] A storage unit is used to store the programs including the steps of the remote two-dimensional code-based map creation method and the programs of the remote two-dimensional code map positioning method fused with laser positioning, so as to be executed by the control unit, the processing unit and the laser positioning unit in time;

[0051] A control unit is used to control the laser radar and the camera to be time-space synchronized, and control the infrared camera to shoot the image to be processed containing the remote two-dimensional code, and control the laser radar to perform mapping scanning;

[0052] A laser positioning unit is used to establish the laser map of the mapping area according to the mapping scanning data of the laser radar, and obtain the laser positioning information;

[0053] A processing unit is used to extract the spatial pose of the remote two-dimensional code in the camera coordinate system, calculate the spatial position of the first observed remote two-dimensional code in the laser map when it is judged that the image observed by the camera contains the first observed remote two-dimensional code, and add it to the remote two-dimensional code map; when the position is in the hybrid positioning area of the laser map and the remote two-dimensional code map, the accuracy of the positioning information is described by using the confidence, and the laser confidence and the remote two-dimensional code positioning confidence are weighted and fused to obtain the positioning information according to the size of the positioning confidence of each position; when the position is in the remote two-dimensional code map positioning area, the ID information of all remote two-dimensional codes in the current scene image is extracted by the camera to match the map to obtain the positioning information.

[0054] The fusion laser positioning remote two-dimensional code map creation and positioning method and system provided by the application ingeniously utilizes the mapping and positioning advantages and characteristics of the remote two-dimensional code, supplements and fuses the shortcomings of laser mapping and positioning, thereby solving the problem of positioning loss in traditional single laser mapping and positioning, and obtaining positioning advantages in more scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0055] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application, illustrate the preferred embodiments of the application and assist in

[0056] Figure 1 A fusion laser positioning remote two-dimensional code map creation method and system provided by the application ingeniously utilizes the mapping and positioning advantages and characteristics of the remote two-dimensional code, supplements and fuses the shortcomings of laser mapping and positioning, thereby solving the problem of positioning loss in traditional single laser mapping and positioning, and obtaining positioning advantages in more scenarios.

[0057] Figure 2 A fusion laser positioning remote two-dimensional code map creation method and system provided by the application ingeniously utilizes the mapping and positioning advantages and characteristics of the remote two-dimensional code, supplements and fuses the shortcomings of laser mapping and positioning, thereby solving the problem of positioning loss in traditional single laser mapping and positioning, and obtaining positioning advantages in more scenarios.

[0058] Figure 3 A fusion laser positioning remote two-dimensional code map creation method and system provided by the application ingeniously utilizes the mapping and positioning advantages and characteristics of the remote two-dimensional code, supplements and fuses the shortcomings of laser mapping and positioning, thereby solving the problem of positioning loss in traditional single laser mapping and positioning, and obtaining positioning advantages in more scenarios.

[0059] Figure 4 A fusion laser positioning remote two-dimensional code map creation method and system provided by the application ingeniously utilizes the mapping and positioning advantages and characteristics of the remote two-dimensional code, supplements and fuses the shortcomings of laser mapping and positioning, thereby solving the problem of positioning loss in traditional single laser mapping and positioning, and obtaining positioning advantages in more scenarios.

[0060] Figure 5 A fusion laser positioning remote two-dimensional code map creation method and system provided by the application ingeniously utilizes the mapping and positioning advantages and characteristics of the remote two-dimensional code, supplements and fuses the shortcomings of laser mapping and positioning, thereby solving the problem of positioning loss in traditional single laser mapping and positioning, and obtaining positioning advantages in more scenarios.

[0061] Figure 6 A fusion laser positioning remote two-dimensional code map creation method and system provided by the application ingeniously utilizes the mapping and positioning advantages and characteristics of the remote two-dimensional code, supplements and fuses the shortcomings of laser mapping and positioning, thereby solving the problem of positioning loss in traditional single laser mapping and positioning, and obtaining positioning advantages in more scenarios.

[0062] Figure 7 A fusion laser positioning remote two-dimensional code map creation method and system provided by the application ingeniously utilizes the mapping and positioning advantages and characteristics of the remote two-dimensional code, supplements and fuses the shortcomings of laser mapping and positioning, thereby solving the problem of positioning loss in traditional single laser mapping and positioning, and obtaining positioning advantages in more scenarios.

[0063] Figure 8 A fusion laser positioning remote two-dimensional code map creation method and system provided by the application ingeniously utilizes the mapping and positioning advantages and characteristics of the remote two-dimensional code, supplements and fuses the shortcomings of laser mapping and positioning, thereby solving the problem of positioning loss in traditional single laser mapping and positioning, and obtaining positioning advantages in more scenarios.

[0064] Figure 9 A fusion laser positioning remote two-dimensional code map creation method and system provided by the application ingeniously utilizes the mapping and positioning advantages and characteristics of the remote two-dimensional code, supplements and fuses the shortcomings of laser mapping and positioning, thereby solving the problem of positioning loss in traditional single laser mapping and positioning, and obtaining positioning advantages in more scenarios.

[0065] Figure 10 A fusion laser positioning remote two-dimensional code map creation method and system provided by the application ingeniously utilizes the mapping and positioning advantages and characteristics of the remote two-dimensional code, supplements and fuses the shortcomings of laser mapping and positioning, thereby solving the problem of positioning loss in traditional single laser mapping and positioning, and obtaining positioning advantages in more scenarios.

[0066] Figure 11 a schematic diagram of a binary image in the map creation method based on a long-distance two-dimensional code of the present application;

[0067] Figure 12 a schematic diagram of a pose singularity phenomenon;

[0068] Figure 13 a schematic diagram of a long-distance two-dimensional code map creation and positioning system structure of the present application. DETAILED DESCRIPTION

[0069] In order for those skilled in the art to better understand the technical solutions of the present application, the specific technical solutions of the present application will be described in detail below in conjunction with the embodiments, so as to help those skilled in the art to further understand the present application. Obviously, the embodiments described in the present application are only a part of the embodiments of the present application, not all the embodiments. It should be pointed out that, for those skilled in the art, the embodiments in the present application and the features in the embodiments can be combined with each other without departing from the concept of the present application and without being in conflict with each other. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts should belong to the disclosure and protection scope of the present application.

[0070] In addition, the terms "first", "second", "S100", "S200" and the like in the specification and claims and drawings of the present application are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that described herein. At the same time, the terms "include" and "have" and any variations thereof in the present application are intended to cover non-exclusive inclusion. Unless otherwise expressly specified and limited, the terms "provide", "arrange", "mount", "connect", "connect" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be connected inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances and in combination with the prior art.

[0071] In order to make up for the short board of the existing laser positioning technology in some scene adaptability, please refer to Figures 1 to 12 The present application provides a long-distance two-dimensional code map creation method combined with laser positioning, and the steps thereof include:

[0072] Step S100 sets a long-distance two-dimensional code at a look-up position of a top of a mapping area relative to a camera; a laser map of the mapping area is established, and laser positioning information is acquired.

[0073] Specifically, in the field of traditional laser mapping and positioning, a robot uses current laser point cloud to match a point cloud map to achieve laser positioning. However, due to the existence of scene changes, mobile robots also often occur in actual situations such as skidding, shielding, and interference of moving objects, so that laser positioning has certain limitations in actual use. Therefore, as shown in Figure 1 , the present example preferably adopts an artificial marking method to set a long-distance two-dimensional code on the ceiling of the scene where the laser positioning loss area is prone to occur, so as to establish a positioning recognition relationship to make up for the deficiency of laser positioning.

[0074] Step S200 acquires an image containing a long-distance two-dimensional code through a camera to extract the spatial pose of the long-distance two-dimensional code in the camera coordinate system.

[0075] Specifically, referring to Figures 6 to 11 , the long-distance two-dimensional code of the present example includes a positioning code and an information code, wherein the positioning code is arranged around a center point in a plurality of non-crossing directions by a plurality of direction points, and a plurality of quadrant regions are divided at the included angles of each adjacent direction point, and the information code is arranged in the corresponding quadrant region according to a preset code table to form a two-dimensional dot array with the positioning code.

[0076] Specifically, the inventors found that a circle (including an ellipse, an elliptical ring, and a circular ring) would produce an affine transformation when the viewing angle changes, and it would be in the form of an ellipse in the camera plane, so its ability to cope with viewing angle transformation is relatively weak. Although the corner points of the traditional square pixel two-dimensional code have stronger robustness under different viewing angles, the weakness is that the corner points are prone to shift at a long distance viewing angle. Based on this discovery, the inventors designed a two-dimensional code in the form of a dot array.

[0077] As shown in Figures 6 to 8 , in the present example, the positioning code in the long-distance two-dimensional code is arranged around a center point according to the four directions of east, south, west, and north by a plurality of direction points to divide four quadrant regions at the included angles of each adjacent direction point as shown in Figure 8 , and the information code can be arranged in the corresponding quadrant region according to a preset code table to form a two-dimensional dot array with the positioning code as shown in Figure 6 , and in the present example, the direction points, the center point, and the information points are preferably in any one of a circle, an ellipse, an elliptical ring, and a circular ring. Figure 7

[0078] ​This setup not only reduces the difficulty of processing and producing the long-range QR code, as it only has one basic pattern (dots), but also allows for significant variations in the arrangement of any two QR codes within this dot matrix, reducing the probability of false detection and supporting different height distances.

[0079] In addition, in order to enhance the layout characteristics of the positioning code, such as Figure 8 As shown in this preferred example, there is at least one main direction point among the plurality of direction points (such as Figure 7 The distance between point A (in the figure) and the center point B is different from that between the other secondary direction points. For example, the distance between A and B is half the distance between C, D, and F and B. However, the secondary direction points are set at the same distance from the center point. This can be used as a constraint for long-range QR code recognition, further improving recognition accuracy.

[0080] Furthermore, considering the influence of various comprehensive factors such as ambient light, the background color of the indoor ceiling, and suspended interference objects, which will interfere with the image captured by the camera and affect the recognition rate of the long-distance QR code, in this example, it is preferred to set a reflective layer on the surface of the positioning code and information code of the long-distance QR code, such as using bright silver reflective cloth, and the corresponding camera is preferably set as an infrared camera, with an infrared filter installed between the camera lens and CMOS to allow infrared light within a certain band to pass through, absorb or reflect visible light and ultraviolet light, and optionally add an infrared light to the camera. After this setting, if Figure 9 As shown, the infrared camera can easily capture images that clearly contain long-range QR code information.

[0081] On the other hand, the long-range QR code corresponding to the above example, such as Figure 10 As shown, in step S200, the step of extracting the spatial pose of the long-range QR code in the camera coordinate system includes:

[0082] Step S210 obtains an image containing the long-range two-dimensional code information, performs binarization processing, and then performs edge extraction.

[0083] Specifically, according to the above example of a long-range QR code, when an infrared camera captures an image containing a long-range QR code, the image obtained is as follows: Figure 8 As shown in the figure, except for the positioning code and information code dots which are white, the rest are black. At this time, the pixel values ​​of the image can be inverted. The inverted image is binarized to convert the grayscale image into a black and white image, as shown in the figure. Figure 11 shown.

[0084] Afterwards, the black and white image is subjected to the Canny edge detection algorithm to extract edges. Due to the special characteristics of the aforementioned infrared camera, there are fewer edges in the black and white image at this time, which can greatly speed up the edge extraction process.

[0085] Step S220 performs ellipse fitting detection based on the extracted edge information to obtain the original code and its center coordinates.

[0086] Specifically, a circle is a special shape of an ellipse. Due to different viewing angles, the points in a long-range QR code are generally approximate ellipses in the imaging plane. Ellipse fitting is often used in feature extraction, scene modeling, camera calibration, and other problems. Given a set of image measurement data from a certain ellipse, The set of all edge points, which contains all the edge points of the ellipse. Represents each edge point, whose coordinates consist of pixel positions, where N represents the number of pixels that make up the edge. The image point edge point information is used here.

[0087]

[0088] Since the measurement data inevitably contains errors, the problem becomes to recover the corresponding ellipse information from the erroneous data. The equation of the ellipse is It can be written as: Represents a set of ellipse parameters. There are 6 ellipse parameters in total.

[0089]

[0090] Because the edge points of each ellipse are on the ellipse, all edge points are on the ellipse. Ellipse fitting is transformed into a linear least squares problem:

[0091]

[0092] To ensure that the solution of the least squares is an ellipse, constraints must be added , , ,

[0093]

[0094] in:

[0095]

[0096] Thus, a nonlinear optimization problem with constraints is constructed:

[0097]

[0098] For this nonlinear optimization problem, the Levenberg-Marquardt algorithm can be used to solve the corresponding ellipse parameter information.

[0099] Based on the above ellipse detection scheme, after all the edge information is extracted, each edge information is respectively fitted based on the least square ellipse. Further, each black ellipse fitting information, i.e. the original code, is obtained, wherein the general equation of the ellipse is as follows . Wherein A, B, C, D, E are parameters of the ellipse equation, which are solved in the ellipse fitting process.

[0100] Through the ellipse parameters, the coordinates of the geometric center of the ellipse are further obtained as:

[0101]

[0102] Wherein the length of the long axis of the ellipse and the length of the short axis b are respectively:

[0103]

[0104] Further, due to the existence of noise points in the image or the influence caused by other reflective materials, each ellipse information needs to be filtered to find the most round ellipse equation.

[0105] Therefore, the step S220 further includes: judging whether the long axis and short axis ratio of each original code conforms to the threshold value according to the long axis and short axis formula of the ellipse equation, and performing the filtering step when it does not conform. For example, when the ratio between the long axis and the short axis is less than 1.3, it is considered that the ellipse (original code) does not conform to the shape of a circle, and further filtering is performed. At the same time, in order to exclude the influence of other noise points, the radius of all circle points belonging to the same long-distance two-dimensional code should be within the same range, and therefore all original codes can be further filtered according to this condition to quickly filter out unqualified original codes.

[0106] Step S230 locates the distribution position of the positioning code corresponding to each long-distance two-dimensional code in the original code according to the geometric relationship of the positioning code.

[0107] Specifically, after obtaining all the original codes that can constitute a long-distance two-dimensional code, since the long-distance two-dimensional code is composed of a positioning code and an information code as described in the above example, the positioning code is composed of five points, i.e. a fixed template as shown in Figure 8 Therefore, the distance between the center circle B and the nearest circle A is exactly times the distance between the other three circles C, D, F and circle B based on the existence of the above example geometric relationship. Therefore, the position information of each long-distance two-dimensional code in the image can be quickly located.

[0108] Therefore, the grouping of each long-range two-dimensional code is obtained. This classification method can greatly speed up the positioning process of long-range two-dimensional codes in the image, and similar to template matching, if multiple long-range two-dimensional codes appear in the image, they can also be distinguished without additional clustering processes.

[0109] In addition, after grouping all long-range two-dimensional codes in an image, the position of each positioning code in the image is obtained. For further verification, the roundness of all positioning codes is checked to ensure that each circle has the same length of major and minor axes. Through the spatial position of the positioning code, the distribution of the information code in the long-range two-dimensional code can be quickly obtained.

[0110] Since the information code carries the ID information of the long-range two-dimensional code, each long-range two-dimensional code has different information codes. After determining the position of the positioning code, the black and white conditions of the pixel blocks in each quadrant can be obtained by calculation, and all the information codes of the long-range two-dimensional code can be obtained. The previous code table determines the spatial position information of each long-range two-dimensional code dot, so by comparing the code table, the ID information of each long-range two-dimensional code can be determined.

[0111] For judgment of whether the long-range two-dimensional code under the ID is first appeared, if not, it means that the spatial pose matrix of the long-range two-dimensional code in the camera coordinate system has been calculated and can be directly called; if it is the first time, the next step of calculating the spatial pose of the long-range two-dimensional code in the camera coordinate system can be performed. Thus, the computing power is saved, and if the position of the long-range two-dimensional code in the environment has been recorded, the recognition of the ID can also be used as the positioning of the mobile robot in the map.

[0112] Step S240 calculates the spatial pose of the corresponding long-range two-dimensional code in the camera coordinate system according to the center coordinates of the positioning code in the image.

[0113] Specifically, the first step is to determine the input label (long-range two-dimensional code) coordinate system coordinates and pixel (image) coordinates.

[0114] For example, a 3x3 homography matrix is calculated to project the 2D circle points in the homogeneous coordinates from the label coordinate system (where the center of the long-range two-dimensional code is located, the label extends one unit in the and directions) to the 2D image coordinate system. The homography is calculated using the direct linear transformation (DLT) algorithm. Note that since the homography projection points are in homogeneous coordinates, only the scale is defined.

[0115] The second step is to establish the relationship between the homography matrix and the conversion matrix.

[0116] ​The position and orientation of the computed long-range QR code, i.e. the extrinsic matrix, requires additional information: the focal length of the camera and the physical size of the tag. The 3 x 3 homography matrix (computed by DLT) can be written as the product of a 3 x 4 camera projection matrix P (assuming it is known) and a 4 x 3 truncated extrinsic matrix E.

[0117] While the extrinsic matrix is usually 4 x 4, each position on the tag is at z = 0 in the tag coordinate system. Therefore, each tag coordinate can be rewritten as a two-dimensional homogeneous point with z implicitly zero, and the third column of the extrinsic matrix can be removed, forming a truncated extrinsic matrix.

[0118] The relationship between the established homography matrix and the transformation matrix is:

[0119]

[0120] where P is the camera projection matrix, whose rotation component is expressed as , E is the truncated extrinsic matrix, whose translation component is expressed as , and s is the equivalent distance scale factor.

[0121] Third, solve the homography matrix.

[0122] First, the homography of a plane is defined as the projection mapping from one plane to another, which is described by the mathematical expression that a point on one plane is multiplied by a projection matrix, and the result is the corresponding point on another plane, point The coordinates of the center point of the detected ellipse are u, v, which represent the pixel coordinates of the center of the ellipse. The point coordinates in the tag coordinate system are represented as

[0123]

[0124] where the projection matrix H is a 3 x 3 matrix

[0125]

[0126] Expand equation (10)

[0127]

[0128] where the format of the equation in the above formula is to find the ratio of the coefficients of each row in the matrix H, that is, the numerator and denominator on the right side of the equation can be multiplied by a scaling factor

[0129]

[0130] The projection matrix is not unique, and the matrix elements can be scaled, so there are only 8 unknowns instead of 9 when solving for the matrix H. Once a value is assigned to any non-zero element in the matrix, the other elements are determined according to the scale.

[0131] In general, the matrix used is obtained by setting the value of to 1 and denoted as In order to make the elements h3 and h6 the values of u and v when x and y are both 0, which is of particular significance in some applications. From the previous derivation, it can be seen that each pair of points on two planes is provided, i.e., two equations based on the coordinates of respectively, and 8 constraint conditions are required to solve the projection matrix H, so 4 pairs of points are required to solve H.

[0132] In this example, an additional point is added to ensure the accuracy of the pose calculation. As can be seen from the ellipse detection example in step S200, each remote two-dimensional code can provide at least five sets of positioning points. Five sets of positioning points provide more sufficient constraints than four pairs of positioning points, thereby ensuring the accuracy of the pose calculation.

[0133] Fourthly, the homography matrix is solved based on SVD.

[0134] (13) can be written as

[0135]

[0136] That is,

[0137]

[0138] where

[0139]

[0140]

[0141] When there are five pairs of points on two planes , the following equation group can be obtained as shown below

[0142]

[0143] where

[0144]

[0145] In order to be more intuitive, the following formula is expanded

[0146]

[0147] Next, solve this equation, the solution of this matrix generally uses singular value decomposition.

[0148] Step 5, solve the conversion matrix by homography matrix.

[0149]

[0150] After obtaining the H matrix by SVD decomposition in the fourth step, since the intrinsic information of the camera can be obtained by camera calibration, the homography matrix and the camera position relationship, i.e. equation (21), can be obtained. The vector , , Since the rotation matrix is orthogonal, we can get . Get the conversion matrix of each remote two-dimensional code relative to the camera coordinate system , and then get the rotation and translation matrix.

[0151] In addition, due to the influence of noise, the rotation and translation matrix calculated by this method is not a global optimal solution, so it is necessary to optimize the result by constructing a nonlinear optimization problem.

[0152] Step 6, solve the minimum error function by iterative optimization method to optimize the rotation and translation matrix.

[0153] Since the result obtained by the direct method is greatly affected by image noise, it is necessary to optimize the result by iterative optimization method according to the distribution of pixel points and spatial points to obtain more accurate remote two-dimensional code pose information. According to the spatial point set and its corresponding image point set determine the camera pose information.

[0154] The rotation of the camera relative to the two-dimensional code coordinate system is represented as and the translation matrix

[0155]

[0156] In the ideal case without noise and other factors, in the case of camera calibration, the spatial point and the point projected into the camera normalized plane coordinate system satisfy the following equation,

[0157]

[0158] where respectively represent the center coordinates of the th positioning code in the remote two-dimensional code, and the coordinates of the point converted into the camera normalized plane according to the rotation and translation matrix , Ci, i = 1, 2, 3, 4, 5, 6 .

[0159] Due to the effect of error, the spatial points and the pixel points do not actually satisfy the above problem, which is converted to finding the optimal parameters Minimizing the error function where R represents rotation, t represents translation. Wherein represents the i-th element in the translation vector, represents the i-th column in the rotation matrix, wherein respectively represent the coordinates of the center of the i-th distant two-dimensional code in the camera normalized coordinate system. represents the center coordinates of all positioning codes. Then the error function is minimized as follows:

[0160]

[0161] In the minimization process, in order to ensure that R is a rotation matrix, therefore, its constraint condition is:

[0162]

[0163] Its optimization function becomes

[0164]

[0165] For this nonlinear optimization problem, the Levenberg-Marquardt algorithm can be used to solve the corresponding accurate rotation and translation matrix.

[0166] Further, as shown in Figure 12 , due to the actual use, if the distant two-dimensional code is small or observed from a very far distance, sometimes the position singularity occurs. The position singularity problem belongs to the basic characteristics of the problem. Geometrically, the two poses correspond to the object flipping around the plane, and the normal line of the plane passes through the line of sight from the camera center to the object center. In such cases, there are usually two pose conditions that can be solved by the above scheme, and the re-projection error of the two solutions is similar, so it is impossible to select the correct pose using the re-projection error.

[0167] There is no reliable algorithm to solve the pose ambiguity problem at present, because it is a natural attribute of the 3D pose recovery problem of the planar target. To solve this problem, more information is needed to constrain the pose of the two-dimensional code.

[0168] To this end, the method of the present example, the step S250 according to time filtering and prior pose constraints, the spatial pose obtained in step S240, wherein the time filtering is when the mobile robot acquires the pose information of the remote two-dimensional code to calculate the current mobile robot pose, to determine whether the mobile robot pose mutation, once the error positioning, then the pose will exist jump, once found, the current remote two-dimensional code position information can be filtered out. And prior pose constraints is to consider the remote two-dimensional code pasted on the ceiling, its coordinate system and the ground, that is, the vehicle body is in parallel plane, so the remote two-dimensional code once the error pose, can be filtered out by the constraint.

[0169] Step S300 as shown in Figure 3 When it is judged that the image observed by the camera contains the first observed remote two-dimensional code, the spatial position of the first observed remote two-dimensional code in the laser map is calculated and added to the remote two-dimensional code map.

[0170] Specifically, because the position of the camera on the mobile robot vehicle body and the position of the laser radar on the vehicle body can be known in advance. Or through the two-dimensional code observation to obtain the position of the two-dimensional code in the camera, and according to the position of the laser in the laser map to know. Thus, through this position relationship, the spatial position of the current remote two-dimensional code in the laser map can be obtained. Therefore, as long as the camera sees the two-dimensional code in real time during laser positioning, the position of the remote two-dimensional code in the laser map can be known in real time.

[0171] On the other hand, to ensure the accuracy of the fusion mapping, in the present example, the step of calculating the spatial position of the observed remote two-dimensional code in the laser map includes:

[0172] Step S310 fuses all the laser positioning information and the spatial pose of the remote two-dimensional code in the camera coordinate system at all times through the factor graph form to construct a maximum a posteriori probability problem to optimize the spatial position of each remote two-dimensional code in the laser map.

[0173] For example, assuming that the position of all remote two-dimensional codes in the laser map is , the measured value is used to constrain the to-be-optimized variable , the Bayesian network modeling is , and the optimized state variable is solved by maximum a posteriori probability. To estimate the position where the maximum a posteriori probability appears, that is, to find a state variable that makes the probability value of the multiplication of each factor maximum, that is, to solve:

[0174]

[0175] Among them, the measured value is a set of measurement values, which includes the following aspects:

[0176] represents the observation of the camera pair at the robot's th position.

[0177] represents the robot's th position in the map.

[0178] represents the camera's mounting position in the vehicle coordinate system, usually obtained by the calibration module.

[0179] First, the problem is modeled using a Bayesian network, which describes a mobile robot moving in the current global map (laser map in static scenes) and performing real-time localization, and observing a series of distant two-dimensional codes in the ceiling during movement:

[0180]

[0181] where represents the probability density function of the pose of the th code in the map given the vehicle's position in the map at the time of all observed th code, where represents the observation of the camera pair at the robot's th position. represents the camera's mounting position in the vehicle coordinate system, i.e., the extrinsic parameter. represents the observation function of a two-dimensional code at a certain position in the map.

[0182] represents the probability density function of the relative position relationship of two two-dimensional codes observed under the condition of seeing two two-dimensional codes in an image.

[0183]

[0184] ,

[0185]

[0186] Next, the maximum a posteriori (MAP) two-dimensional code space position is estimated, i.e., find a state quantity that maximizes the product of the probabilities of each factor, i.e., solve:

[0187]

[0188] Therefore, the maximum a posteriori probability problem is transformed into a least squares problem, that is, the least squares problem is used for solution.

[0189] In this way, the positioning information of the laser provides an absolute coordinate system reference for the positioning of the long-range QR code, while also ensuring the sparse characteristics of the long-range QR code and reducing the deployment density of the long-range QR code. The introduction of the long-range QR code avoids some limitations of laser positioning and makes it possible to quickly update the laser map.

[0190] Further, such as Figures 3 to 4 As shown in the figure, since the camera and laser are in different positioning frames, to ensure the quality of multi-sensor data fusion, a synchronization method is needed to ensure that the data sources of multiple sensors have a unified time and space reference, thereby ensuring the spatiotemporal synchronization of the measurement data of each sensor. Spatial synchronization can be achieved by obtaining the relative position relationship through external parameter calibration technology.

[0191] Time synchronization is to ensure that the information collected by multiple sensors is collected and calculated within a certain time error range. In addition, time synchronization is divided into software synchronization and hardware synchronization. This solution implements both synchronization schemes.

[0192] Hardware synchronization: Both the camera and lidar are connected to the same controller, which provides stabilization time for both. The camera is triggered by a level signal from the controller. The controller triggers camera data acquisition at a specific frequency. However, since the lidar frame rate is affected by its internal rotational mechanism, a unified trigger signal cannot be used. In this solution, the lidar output data includes not only the coordinates of each laser point, but also the hardware timestamp corresponding to each laser point.

[0193] Software synchronization: Compared with radar, laser radar is a slow scanning device. In this solution, the camera is used as the time synchronization benchmark. The positioning information based on the laser map obtained by laser data will be combined with the QR code information through the timestamp information. The camera data collected at the current moment is The timestamp of the laser radar is the laser information of the first point in each frame of laser information. That is, the timestamp of the current positioning Its timestamp information is as follows: Figure 4 shown.

[0194] In order to obtain accurate positioning information corresponding to the camera time, it is necessary to interpolate the positioning information based on the camera time to obtain the corresponding The positioning information at each moment can then be used to complete time synchronization.

[0195] Furthermore, in the process of mapping according to the above-mentioned example, in order to obtain more accurate spatial coordinate information of long-range QR codes, it is preferred to perform multiple non-repetitive observations under each QR code mark to be observed, thereby improving the accuracy of the long-range QR code posture in the map.

[0196] Maps are the foundation of positioning. Unlike pure laser maps and pure QR code maps, fused maps are derived from measurements taken by multiple sensor types. Fusion maps include not only laser measurement information but also the pose information of the QR codes in the scene. This map enables both laser positioning and QR code-assisted positioning. It effectively addresses positioning challenges even in the event of a single sensor failure.

[0197] Therefore, corresponding to the above-mentioned mapping method, the present invention also provides a long-distance QR code map positioning method integrated with laser positioning, which comprises the following steps:

[0198] Step S400: When the laser map and the long-range QR code map are mixed in the positioning area, the confidence level is used to describe the accuracy of the positioning information. The laser positioning confidence level is set separately. Reliability of long-distance QR code positioning , the positioning information is obtained by weighted fusion of the positioning confidence of each position.

[0199] Specifically, in this example, the long-range QR code map integrated with laser positioning (herein referred to as the fused map) consists of three components: a 2D spatial laser grid map, a 3D spatial QR code map, and positioning block configuration information. This positioning block configuration includes four areas: a hybrid positioning area, a QR code positioning area, and a laser positioning area. The entire fused map is positioned using different positioning methods in different areas through the positioning block configuration, ensuring effective positioning in complex scenarios.

[0200] For the laser positioning area, the position information is provided by using a Monte Carlo positioning algorithm on the laser map.

[0201] As for the mixed positioning area, it is a positioning area used by multiple positioning methods. Multiple positioning methods can be weighted and fused according to the size of their respective position confidence.

[0202] Therefore, in the hybrid positioning area positioning process of the fusion map, in order to determine the accuracy of the positioning position, the confidence level is first used to describe the accuracy of the positioning information:

[0203] The confidence of laser positioning is expressed as:

[0204]

[0205] in, represents the confidence of each laser measurement in the current map position. The above equation describes the degree of laser point cloud and occupancy grid map recombination, the greater the laser and map recombination, the higher the confidence, and if there is no corresponding point in the map, the confidence is 0.

[0206] And the long-distance two-dimensional code positioning confidence is represented as:

[0207]

[0208] represents the number of positioning circles in space, respectively represent the pixel coordinates of the center of each positioning code in the current two-dimensional code, represents the center coordinates of the camera, which is usually obtained by camera calibration. The distribution of a two-dimensional code in the current image is described, and the farther the distance from the image center, the lower the confidence. In the robot positioning process, the corresponding positioning confidence is calculated in real time according to the type of positioning adopted.

[0209] When the mobile robot is in a hybrid positioning area, 2D laser positioning and long-distance two-dimensional code positioning will work simultaneously, and 2D laser will output the current robot position information in the laser map based on the Monte Carlo positioning algorithm. The top-view camera will also capture the two-dimensional code information in the ceiling and output the corresponding positioning information. Both kinds of positioning information not only contain the spatial position of the robot, but also contain the positioning confidence of the current positioning method. When the positioning confidence of a certain method is lower than the threshold, the positioning method is discarded.

[0210] For example, the vehicle positioning position at the last moment is set as , the positioning confidence is set as , the vehicle positioning position using camera positioning at the current moment is set as , and the positioning confidence is set as . The vehicle position using laser positioning at the current moment is set as , and the corresponding positioning confidence is set as .

[0211] The fusion positioning result is:

[0212]

[0213]

[0214] wherein represents the current robot position, represents the confidence of the current hybrid positioning.

[0215] Step S500, when the robot is in the remote two-dimensional code map positioning area, the ID information of all remote two-dimensional codes in the current scene image is extracted by the camera to match the map to obtain the positioning information.

[0216] Specifically, the two-dimensional code positioning area only uses camera positioning, and when the two-dimensional code cannot be observed, the current pose information of the robot can be updated by the wheel mileage information of the robot.

[0217] The remote two-dimensional code map contains the spatial position information of all two-dimensional codes in the current scene. The mobile robot captures images in the scene in real time and extracts corresponding two-dimensional code information during movement, and realizes positioning by matching the two-dimensional code information with the map.

[0218] For example, the pixel coordinates of all two-dimensional code positioning points detected in the current image are represented as:

[0219]

[0220] wherein represents the number of all detected two-dimensional codes in the current image, represents the number of positioning points in each two-dimensional code. The remote two-dimensional code used in the present scheme has five positioning code points.

[0221] Therefore, the position of each two-dimensional code in space can be known from the map, and the spatial coordinates of the center of each positioning code circle in the map can be calculated, wherein represents the spatial pose of the two-dimensional code in the map, represents the position in the two-dimensional code coordinate system, wherein .

[0222]

[0223] The spatial points of the centers of all positioning code circles of the two-dimensional code in the map coordinate system are obtained by the above formula. By using the two-dimensional code ID information, all two-dimensional codes detected in the current image in the map can be found, and then the spatial point set of all centers in the map coordinate system can be extracted .

[0224] Therefore, the image point set and the spatial point set constitute a classic visual positioning problem. Using the pose refinement algorithm proposed in the foregoing example about the scheme for solving the minimum error function by an iterative optimization method, the pose information of the current camera in the remote two-dimensional code map can be solved. The visual positioning belongs to an optimization problem, and an initial value of the camera pose needs to be provided to ensure the effectiveness of the refinement result.

[0225] For example, when only one two-dimensional code is detected in the current image, the initial pose of the camera is given by the following formula:

[0226]

[0227] When multiple two-dimensional codes are detected in the current image, to avoid mutual interference of multiple two-dimensional code positioning, the camera pose initial value is obtained by multiple two-dimensional code Kalman filtering. The covariance of each two-dimensional code positioning result is given by the confidence of each two-dimensional code.

[0228] To avoid singular solutions of two-dimensional code pose caused by image noise and scene light, the present scheme adopts a secondary result evaluation mechanism to delete incorrect two-dimensional code positioning information, and the flowchart thereof is shown in Figure 5 Therefore, through the secondary result evaluation mechanism, the influence of outliers on positioning can be avoided, and the robustness of positioning is enhanced.

[0229] On the other hand, as shown in Figure 13 Corresponding to the mapping and positioning method described above, the present application also provides a long-distance two-dimensional code map creation and positioning system fused with laser positioning, which comprises:

[0230] A storage unit for storing programs including the steps of the long-distance two-dimensional code-based map creation method according to any one of the above and the steps of the long-distance two-dimensional code map positioning method fused with laser positioning described above, for the control unit, processing unit, and laser positioning unit to execute in time;

[0231] A control unit for controlling the laser radar and the camera to perform time-space synchronization, and controlling the infrared camera to capture the image to be processed containing the long-distance two-dimensional code, and controlling the laser radar to perform mapping scanning

[0232] A laser positioning unit for establishing a laser map of the mapping area according to the mapping scanning data of the laser radar, and obtaining laser positioning information;

[0233] A processing unit for extracting the spatial pose of the long-distance two-dimensional code in the camera coordinate system, when it is judged that the image observed by the camera contains the first observed long-distance two-dimensional code, calculating the spatial position of the first observed long-distance two-dimensional code in the laser map, and adding it to the long-distance two-dimensional code map; when in the mixed positioning area of the laser map and the long-distance two-dimensional code map, using the confidence to describe the accuracy of the positioning information, respectively setting the laser positioning confidence And the long-distance two-dimensional code positioning confidence , and obtaining the positioning information by weighted fusion according to the size of the positioning confidence of each position; when in the long-distance two-dimensional code map positioning area, extracting the ID information of all long-distance two-dimensional codes in the current scene image by the camera, and matching with the map to obtain the positioning information.

[0234] In summary, by the fusion laser positioning long-distance two-dimensional code map creation and positioning method and system provided by the application, the circular geometric shape is ingeniously used, even with the change of the camera view angle, the affine transformation only becomes an ellipse, and the center of the circle still exists, and the circular shape is still robust in the case of blurred detection at a long distance, a two-dimensional code suitable for long-distance detection is designed, and according to the aforementioned characteristics of the long-distance two-dimensional code, the center of the ellipse can be obtained very accurately through ellipse detection to establish the pose transformation matrix with the camera, thereby solving the problem that the two-dimensional code in the detection image captured by the traditional two-dimensional code and the detection camera is easy to present a blurred block shape when the distance is far and the angle is changed, and accurate recognition cannot be performed. On this basis, the application utilizes the advantages and characteristics of long-distance two-dimensional code mapping and positioning to complement and fuse the short board of laser mapping and positioning, thereby solving the problem that the traditional single laser mapping and positioning is easy to lose positioning, and obtaining the positioning advantage in more scenes.

[0235] The preferred embodiments of the application disclosed above are only used to help explain the application. The preferred embodiments do not describe all the details, nor limit the application to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of the present application. The embodiments are selected and described in the present specification in order to better explain the principles and practical applications of the application, so that those skilled in the art can well understand and utilize the application. The application is limited by the claims and their entire scope and equivalents, and any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the application shall be included in the protection scope of the application.

[0236] Those skilled in the art can understand that, in addition to implementing the system, device, unit and each module thereof provided by the application in the form of pure computer readable program code, the same program can also be realized by logically programming the method steps to make the system, device, unit and each module thereof provided by the application in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers. Therefore, the system, device and each module thereof provided by the application can be considered as a hardware component, and the modules included therein for realizing various programs can also be considered as structures in the hardware component; the modules for realizing various functions can also be considered as both software programs for realizing methods and structures in the hardware component.

[0237] Moreover, all or part of the steps of the above-mentioned embodiment methods can be instructed by a program to relevant hardware, the program is stored in a storage medium, including a plurality of instructions to make a single-chip microcomputer, chip or processor (processor) to execute all or part of the steps of the method described in various embodiments of the present application. And the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and various storage program codes.

[0238] In addition, various different embodiments of the embodiments of the present application can also be combined arbitrarily, as long as they do not violate the idea of the embodiments of the present application, they should also be considered as disclosed by the embodiments of the present application.

Claims

1. A method for creating a long-range QR code map by integrating laser positioning, comprising the following steps: Step S100 sets the long-range QR code at the top of the mapping area in an upward position relative to the camera; Establishing a laser map of the mapping area and obtaining laser positioning information; wherein the long-range QR code includes: a positioning code and an information code, wherein the positioning code is composed of a plurality of direction points arranged in a plurality of non-intersecting directions around a center point, and a plurality of quadrant areas are divided at the angles between adjacent direction points, and the information code is arranged in the corresponding quadrant area according to a preset code table to form a two-dimensional dot matrix with the positioning code, wherein the surfaces of the positioning code and the information code are provided with a reflective layer; Step S200: acquiring an image containing a long-range QR code via a camera to extract the spatial pose of the long-range QR code in the camera coordinate system; Step S300: When it is determined that the image observed by the camera contains a long-range QR code observed for the first time, the spatial position of the long-range QR code observed for the first time in the laser map is calculated. The steps include: Step S310 fuses the laser positioning information at all times and the spatial pose of the long-range QR code in the camera coordinate system in the form of a factor graph, constructs a maximum a posteriori probability problem to optimize the spatial position of each long-range QR code in the laser map, and adds it to the long-range QR code map.

2. The method for creating a long-range QR code map integrating laser positioning according to claim 1, wherein in step S310, the step of constructing a maximum a posteriori probability problem to optimize the spatial position of each long-range QR code in the laser map comprises: Step S311 sets the position of all long-range QR codes in the laser map to , using the measured value Constrain the variables to be optimized , modeled using a Bayesian network as , solve the optimized state variables through the maximum a posteriori probability: ； Among them, the measured value is a set of measured values: Indicates that the mobile robot is in the When the camera is in position observations; Indicates the position of the mobile robot in the laser map. positions; Indicates the installation position of the camera in the vehicle coordinate system.

3. The method for creating a long-range QR code map by integrating laser positioning according to claim 1, wherein in step S200, the step of extracting the spatial pose of the long-range QR code in the camera coordinate system comprises: Step S210: obtaining an image containing the long-range QR code information, performing binarization processing, and then performing edge extraction; Step S220 performs ellipse fitting detection based on the extracted edge information to obtain the original code and its center coordinates; Step S230 locates the distribution of the positioning codes corresponding to the remote QR codes in the original code based on the geometric relationship of the positioning codes, thereby filtering out the information code in the original code and obtaining the ID information of the remote QR code according to the pass code table; In step S240, when it is determined that the long-range QR code under the ID appears for the first time, the spatial position of the corresponding long-range QR code in the camera coordinate system is calculated based on the center coordinates of the positioning code in the image and is bound to the ID information.

4. The method for creating a long-range QR code map by integrating laser positioning according to claim 3, wherein the step of calculating the spatial pose of the long-range QR code in the camera coordinate system in step S240 comprises: Step S241 sets the center coordinates of the bit code to , build a homogeneous matrix ； Solve the homography matrix H based on the SVD method, where u and v represent the pixel coordinates of the center of the positioning code circle, Represents the coordinates of a point in the long-distance QR code coordinate system, and s is the equivalent distance scale factor; Step S242 is based on the relationship between the homography matrix and the transformation matrix of the long-range QR code in the camera coordinate system. ； Obtain the rotation and translation matrices, where P is the camera projection matrix and E is the truncated extrinsic matrix. , Camera The focal length in the a and y directions, , is the coordinate of the camera center point, are the first two columns in the rotation matrix, , , They represent the position of the center of the long-range QR code in the camera coordinate system, is a 3×3 homography projection matrix.

5. The method for creating a long-range QR code map by integrating laser positioning according to claim 4, wherein step S240 further comprises: Step S243 solves the minimization error function by iterative optimization method ； To optimize the rotation and translation matrix, where R represents rotation and t represents translation ； ； represents the i-th element in the translation vector, represents the i-th column in the rotation matrix; Represents the first The coordinates of the center of the long-distance QR code projected into the camera's normalized coordinate system ; Indicates the center coordinates of all positioning codes; Respectively represent the first The spatial coordinates of the center of the positioning code are converted to the coordinates of the point in the camera normalized plane according to the rotation and translation matrix ; Indicates the spatial coordinates of the center of each positioning code in the long-range QR code coordinate system ; In the solution process, as a constraint.

6. The method for creating a long-range QR code map by integrating laser positioning according to claim 3, wherein in step S200, the step of extracting the spatial pose of the long-range QR code in the camera coordinate system further comprises: Step S250 constrains the spatial pose obtained in step S240 according to time filtering and prior pose constraints, wherein the time filtering step includes: filtering out when it is determined that there is a sudden change in the spatial pose result; the prior pose constraint step includes: filtering out when it is determined that the long-range QR code coordinate system and the camera coordinate system in the spatial pose result are non-parallel.

7. A long-range QR code map positioning method integrated with laser positioning, for use in positioning a map created by the long-range QR code map creation method integrated with laser positioning as claimed in any one of claims 1 to 6, comprising: Step S400: When the laser map and the long-range QR code map are mixed in the positioning area, the confidence level is used to describe the accuracy of the positioning information. The laser positioning confidence level is set separately. Reliability of long-distance QR code positioning , the positioning information is obtained by weighted fusion through the positioning confidence of each position: ； ； in is the current location, Indicates the confidence of the current hybrid positioning, Position the location at the last moment. The current position of the camera. The current position using laser positioning; In step S500, when the camera is in the long-range QR code map positioning area, the camera extracts the ID information of all long-range QR codes in the current scene image to match them with the map to obtain positioning information.

8. A long-range QR code map creation and positioning system integrated with laser positioning, comprising: A storage unit for storing a program including the steps of the long-range QR code map creation method integrated with laser positioning as described in any one of claims 1 to 6 and the steps of the long-range QR code map positioning method integrated with laser positioning as described in claim 7, so as to be retrieved and executed by the control unit, the processing unit, and the laser positioning unit in a timely manner; A control unit is used to control the LiDAR and camera for time and space synchronization, control the infrared camera to capture the image to be processed containing the long-range QR code, and control the LiDAR for mapping and scanning; The laser positioning unit is used to create a laser map of the mapping area based on the laser radar mapping scan data and obtain laser positioning information; A processing unit is used to extract the spatial position of the long-range QR code in the camera coordinate system. When it is determined that the image observed by the camera contains a long-range QR code observed for the first time, the processing unit is used to calculate the spatial position of the long-range QR code observed for the first time in the laser map and add it to the long-range QR code map; When the laser map and the long-distance QR code map are mixed in the positioning area, the confidence level is used to describe the accuracy of the positioning information. The laser positioning confidence level is set separately. Reliability of long-distance QR code positioning , the positioning information is obtained by weighted fusion of the positioning confidence of each position; when it is in the long-range QR code map positioning area, the ID information of all long-range QR codes in the current scene image is extracted by the camera to match it with the map to obtain positioning information.

Images