An unmanned vehicle positioning method and device based on indoor global vision

Abstract

The application provides a kind of indoor global vision-based unmanned vehicle positioning method and device, the method comprises: step 1, visual sensor is fixed on indoor ceiling, and the ground environment image of the periphery of mobile robot in running is collected;Two two-dimensional code labels are arranged on the top of the mobile robot, and the mobile robot is identified by two two-dimensional codes;Step 2, the ground environment image obtained in step 1 is transformed by image processor Abnormal image is obtained by distortion correction;Step 3, the normal image is extracted to obtain feature points;More than two normal images with feature points captured by different vision sensors are matched;Step 4, the image is fused to obtain the wide-area panoramic image after splicing;Step 5, the pose information of the mobile robot is solved.The application can realize the control and tracking of mobile robot in wide-area complex indoor environment, and realize the accurate navigation of mobile robot.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a method and device for unmanned vehicle positioning based on indoor global vision. Background Technology

[0002] Indoor localization is a prerequisite for indoor navigation in mobile robots. Mobile robots carry relevant sensors for real-time localization and mapping. Since localization in SLAM (Simultaneous Localization and Mapping) is fundamental to robot mapping and navigation, and plays a decisive role in the application and industrialization of mobile robots, it is the biggest obstacle to the industrialization and development of mobile robot applications. However, SLAM cannot be used in changing environments. In such cases, visual localization can be used to solve the localization problem in dynamic indoor environments.

[0003] QR code positioning refers to the technology of detecting the real-time location of QR codes in an indoor scene using image sensing and other means. It is one of the main positioning methods for indoor autonomous vehicles. In a vision-based QR code positioning system, QR code detection and tracking is a basic and necessary function. It can prevent autonomous vehicles from colliding with obstacles, and at the same time, it can provide other important information, including map information and the autonomous vehicle's pose, to the main control system.

[0004] In recent decades, visual measurement technology has become a very popular measurement technique. It uses machine vision information to acquire feature points in images. Visual measurement technology is widely used in modern advanced manufacturing technology research and smart service industries due to its non-contact nature, good measurement accuracy, speed, reasonable price, and wide applicability to various cameras. Furthermore, camera-based global vision measurement systems can measure static cameras or fixed and redundant camera networks without electronic interference, achieving higher accuracy. Therefore, its measurements can be used as the true values ​​for other trajectory evaluation systems for robots. To evaluate the performance of mobile robots, multiple cameras are typically fixed in the test scene in indoor building experiments, and a global vision system is used for robot trajectory tracking. Moreover, the accuracy and stability of mobile robot localization are key to improving mobile robot performance. One of the basic requirements for industrial applications of mobile robots is that the robot can stably and accurately localize, build environmental maps, and perform reliable navigation. Summary of the Invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to address the shortcomings of existing technologies by providing a method for unmanned vehicle localization based on indoor global vision, comprising the following steps:

[0006] Step 1: Fix two or more vision sensors on the indoor ceiling to collect images of the ground environment around the mobile robot as it moves; place two QR code labels on the top of the mobile robot to identify it, and use the two QR codes to identify the mobile robot. The information on the labels includes the pose and speed information of the mobile robot.

[0007] Step 2: The image processor (GK3000 industrial computer) performs distortion correction transformation on the ground environment image obtained in Step 1 to obtain a normal image;

[0008] Step 3: Extract feature points from the normal image and store the feature matrix; then perform feature matching on two or more normal images with feature points captured by different visual sensors.

[0009] Step 4: The images after feature matching in step 3 are fused together to obtain a stitched wide-area panoramic image.

[0010] Step 5: Perform QR code recognition on the wide-area panoramic image to obtain the pose information of the two QR codes; calculate the pose information of the mobile robot based on the placement height of the visual sensor, the size of the QR code, and the center position of the QR code.

[0011] Step 1 includes:

[0012] Step 1-1: First, determine the placement and orientation of the cameras with two or more vision sensors, and adjust the brightness of the matching light source for the cameras and the spacing between the vision sensors according to the actual scene.

[0013] Steps 1-2: Adjust the parameters of the vision sensor according to the current light source brightness, collect QR code information, and compare the variance of the QR code center position measurement with the actual variance to obtain the confidence level of the positioning.

[0014] Steps 1-3: Real-time acquisition of continuous ground environment images around the mobile robot in motion, and transmission of the acquired ground environment images to the image processor.

[0015] Steps 1-2 include: Let d be the actual physical distance between the center points of the two QR codes; e i w is the measured distance between the two QR codes in the current frame i; i =|de i | represents the measurement error of the current frame i. The accuracy of positioning is evaluated by the variance of the measurement error. The variance formula is:

[0016]

[0017] Where s 2This represents the variance of the measurement across the entire video frame, where n is the total number of frames in the video stream. w represents the average measurement error of the n video frames. Variance measures the magnitude of fluctuation in error data; the smaller the variance, the more stable the system and the smaller the error.

[0018] Step 2 includes: the visual sensor is a camera, and the camera is calibrated using the Zhang Zhengyou calibration method to obtain camera parameter information (f). x ,f y ,c x ,c y ), where f x f is the focal length of the camera along the x-axis. y Let c be the focal length along the y-axis of the camera. x It is the x-coordinate of the focal center in the pixel coordinate system, c y This is the y-coordinate of the focal center in the camera coordinate system; the distorted image is known, and the mapping relationship is derived through the distortion model to obtain the undistorted image:

[0019] The relationship between the real image imgR and the distorted image imgD is: imgR(u,v)=imgD(u d ,u v ); imgR(u,v) represents the pixel value of the real image imgR at the x-coordinate u and y-coordinate v in the pixel coordinate system, imgD(u d ,u v The ) represents the x-coordinate u of the distorted image imgD in the pixel coordinate system. d , ordinate u v Pixel value at;

[0020] The formula for converting a real image to a distorted image is:

[0021]

[0022]

[0023]

[0024] (x′, y′) represents the position (u, v) in the real image imgR after projection transformation and coordinate system transformation. (x″, y″) represents the distorted position coordinates of any point in the distorted image with the camera coordinate system as the origin. Here, k1 and k2 are the mirror distortion coefficients, and P1 and P2 are the tangential distortion coefficients. γ is the radius from the current pixel to the center of the circle, and γ2 = (ux′). 2 +(vy′) 2 ;

[0025] After transformation using formulas (1), (2), and (3), the correspondence between the pixels in the real image and the pixels in the distorted image is obtained, thus enabling the mapping of pixels in the distorted image imgD (u d ,v u The pixel values ​​of ) are inserted into imgR(u,v).

[0026] Step 3 includes: In a system composed of two or more vision sensors, if any two cameras have overlapping areas, the images from these two cameras are denoted as image1 and image2 respectively for registration. First, feature point 1 (u1, v1) in image1 and feature point 2 (u2, v2) in image2 are obtained, and the corresponding homography matrix is ​​H, where h ij Let be the parameter in the i-th row and j-th column of matrix H, 1≤i,j≤3; s is the image scale factor, the coordinates of feature point 1 are (u1,v1), and the coordinates of feature point 2 are (u2,v2).

[0027]

[0028] When s takes any value, set... Then matrix H is rewritten as:

[0029]

[0030] After editing, it is written as:

[0031]

[0032] The homography matrix H can be obtained by using more than four sets of feature points.

[0033] Step 5 includes: using the camera intrinsic parameters obtained in Step 2 based on the Zhang Youzheng calibration method and the homography matrix H′ between the camera coordinate system and the world coordinate system, the following conversion formula between theoretical coordinates and actual coordinate points is obtained, namely, the expansion formula of the homography matrix H′:

[0034]

[0035] Where (x2, y2) are the camera image coordinates, and (x1, y1) are the actual physical locations of the camera image coordinates.

[0036] Expanding formula (7) yields:

[0037]

[0038] Then, the coordinate pose of the center point is calculated based on the coordinate information of the four corner points of the label.

[0039] The present invention also provides an unmanned vehicle positioning device based on indoor global vision, comprising:

[0040] A vision sensor, fixed to the indoor ceiling, is used to acquire images of the ground environment around the mobile robot in motion;

[0041] The mobile robot has two QR code labels on its top. The two QR codes identify the mobile robot and contain information about the robot's pose and speed.

[0042] An image processor (industrial computer) is used to receive ground environment images and perform distortion correction transformation on the ground environment images to obtain normal images;

[0043] The feature point matching module is used to extract feature points from normal images and store the feature matrix; then it performs feature matching on two or more normal images with feature points captured by different visual sensors.

[0044] The fusion module is used to fuse the feature-matched images to obtain a stitched wide-area panoramic image;

[0045] The pose information solving module is used to perform QR code recognition on a wide-area panoramic image to obtain the pose information of two QR codes; the pose information of the mobile robot is solved based on the placement height of the visual sensor, the size of the QR code, and the center position of the QR code.

[0046] This paper focuses on global visual localization of mobile robots in multiple indoor rooms, aiming to reduce the impact of interference factors in the indoor environment and achieve low-cost, high-precision localization and navigation for robots in complex indoor environments. Firstly, by comparing the installation of cameras at multiple locations on the ceiling of an indoor environment, a global visual localization and path planning system based on a distributed monocular camera array and dual QR codes is designed. Using a fixed camera mounted on the ceiling of the indoor environment, dual QR codes are affixed to the robot. The camera stably identifies and tracks the robot, estimating its pose and generating its trajectory in real time as it moves forward. However, in complex indoor environments with multiple spaces and large fields of view, relying solely on a monocular camera is often insufficient. Therefore, a distributed camera array is used for real-time stitching to improve the applicability of this method.

[0047] Due to the limitations of camera field of view leading to the limitations of global visual positioning systems, this invention proposes a real-time video stitching method based on multiple cameras. Multiple cameras are distributed and installed to construct a camera network, achieving large-area, multi-spatial, and high-precision positioning. Image stitching technology combines several images with overlapping parts (possibly obtained at different times, from different perspectives, or by different sensors) into a large, seamless, high-resolution image. In this invention, due to the fixed real-time monitoring scenario, while improved feature point extraction and matching can be used to accelerate stitching, and GPU acceleration via the CUDA framework can be achieved, it does not meet real-time requirements. Therefore, this paper proposes a panoramic multi-channel high-resolution video stitching system. This system utilizes a distributed architecture of a camera network, using matched feature points to calculate the homography matrix, i.e., the transformation matrix of pixel coordinates. However, the above process is time-consuming during stitching. Considering the relatively fixed positions of surveillance cameras, in this invention as an example: the cameras are arranged in a fan shape on the top of an indoor space, with protective covers on the outside to prevent external interference. The camera position remains fixed. The homography matrix is ​​calculated once using the image registration algorithm and used as a fixed parameter. It can be directly used in subsequent image stitching, which will greatly reduce the time consumption.

[0048] Compared with the prior art, the present invention has the following advantages and effects:

[0049] This invention comprises an image acquisition module consisting of multiple vision sensors. The vision sensors acquire scene images, an image processor processes and analyzes the images, and a motion control module issues commands to control the operation of the mobile robot. Two QR codes are pre-set on the top of the mobile robot and labeled accordingly. Multiple vision sensors acquire scene images of the surrounding environment, and distortion-free images are obtained through distortion correction. Feature extraction and fusion are performed on the obtained images, and the image processor acquires the pose information of the QR codes. The environmental information and the two QR code information are processed to obtain the pose and motion state information of the mobile robot, enabling navigation. This invention can achieve the control and tracking of a mobile robot in a wide-area, complex indoor environment, enabling precise navigation of the mobile robot. Attached Figure Description

[0050] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0051] Figure 1 This is a schematic diagram of the navigation method for a vision-based mobile robot navigation system.

[0052] Figure 2 This is a schematic diagram of barrel distortion correction for a vision acquisition device.

[0053] Figure 3 This is a schematic diagram of a dual QR code structure.

[0054] Figure 4 This is a schematic diagram of a mobile robot with dual QR code positioning.

[0055] Figure 5 This is a schematic diagram of the distribution of vision sensors.

[0056] Figure 6 This is a flowchart of the video stitching process.

[0057] Figure 7 This is a schematic diagram of a vision-based mobile robot localization and navigation structure. Detailed Implementation

[0058] like Figure 1 As shown, this invention provides a method for unmanned vehicle localization based on indoor global vision, comprising the following steps:

[0059] Step 1: Fix two or more vision sensors on the indoor ceiling to collect images of the ground environment around the mobile robot as it moves; place two QR code labels on the top of the mobile robot to identify it, and use the two QR codes to identify the mobile robot. The information on the labels includes the pose and speed information of the mobile robot.

[0060] Step 2: The image processor (GK3000 industrial computer) performs distortion correction transformation on the ground environment image obtained in Step 1 to obtain a normal image;

[0061] Step 3: Extract feature points from the normal image and store the feature matrix; then perform feature matching on two or more normal images with feature points captured by different visual sensors.

[0062] Step 4: The images after feature matching in step 3 are fused together to obtain a stitched wide-area panoramic image.

[0063] Step 5: Perform QR code recognition on the wide-area panoramic image to obtain the pose information of the two QR codes; calculate the pose information of the mobile robot based on the placement height of the visual sensor, the size of the QR code, and the center position of the QR code.

[0064] Step 1 includes:

[0065] Step 1-1: First, determine the placement and orientation of the cameras with two or more vision sensors, and adjust the brightness of the matching light source for the cameras and the spacing between the vision sensors according to the actual scene;

[0066] Steps 1-2: Adjust the parameters of the vision sensor according to the current light source brightness, collect QR code information, and compare the variance of the QR code center position measurement with the actual variance to obtain the confidence level of the positioning.

[0067] Steps 1-3: Real-time acquisition of continuous ground environment images around the mobile robot in motion, and transmission of the acquired ground environment images to the image processor.

[0068] Steps 1-2 include: Let d be the actual physical distance between the center points of the two QR codes; e i w is the measured distance between the two QR codes in the current frame i; i =|de i | represents the measurement error of the current frame i. The accuracy of positioning is evaluated by the variance of the measurement error. The variance formula is:

[0069]

[0070] Where s 2 This represents the variance of the measurement across the entire video frame, where n is the total number of frames in the video stream. w represents the average measurement error of the n video frames. Variance measures the magnitude of fluctuation in error data; the smaller the variance, the more stable the system and the smaller the error.

[0071] Step 2 includes: the visual sensor is a camera, and the camera is calibrated using the Zhang Zhengyou calibration method to obtain camera parameter information (f). x ,f y ,c x ,c y ), where f x f is the focal length of the camera along the x-axis. y Let c be the focal length along the y-axis of the camera. x It is the x-coordinate of the focal center in the pixel coordinate system, c y This is the y-coordinate of the focal center in the camera coordinate system; the distorted image is known, and the mapping relationship is derived through the distortion model to obtain the undistorted image:

[0072] The relationship between the real image imgR and the distorted image imgD is: imgR(u,v)=imgD(u d ,u v ); imgR(u,v) represents the pixel value of the real image imgR at the x-coordinate u and y-coordinate v in the pixel coordinate system, imgD(u d ,u v The ) represents the x-coordinate u of the distorted image imgD in the pixel coordinate system. d , ordinate u v Pixel value at;

[0073] The formula for converting a real image to a distorted image is:

[0074]

[0075]

[0076]

[0077] (x′, y′) represents the position (u, v) in the real image imgR after projection transformation and coordinate system transformation. (x″, y″) represents the distorted position coordinates of any point in the distorted image with the camera coordinate system as the origin. Here, k1 and k2 are the mirror distortion coefficients, and P1 and P2 are the tangential distortion coefficients. γ is the radius from the current pixel to the center of the circle, and γ2 = (ux′). 2 +(vy′) 2 ;

[0078] After transformation using formulas (1), (2), and (3), the correspondence between the pixels in the real image and the pixels in the distorted image is obtained, thus enabling the mapping of pixels in the distorted image imgD (u d ,v u The pixel values ​​of ) are inserted into imgR(u,v).

[0079] Step 3 includes: In a system composed of two or more vision sensors, if any two cameras have overlapping areas, the images from these two cameras are denoted as image1 and image2 respectively for registration. First, feature point 1 (u1, v1) in image1 and feature point 2 (u2, v2) in image2 are obtained, and the corresponding homography matrix is ​​H, where h ij Let be the parameter in the i-th row and j-th column of matrix H, 1≤i,j≤3; s is the image scale factor, the coordinates of feature point 1 are (u1,v1), and the coordinates of feature point 2 are (u2,v2).

[0080]

[0081] When s takes any value, set... Then matrix H is rewritten as:

[0082]

[0083] After editing, it is written as:

[0084]

[0085] The homography matrix H can be obtained by using more than four sets of feature points.

[0086] Step 5 includes: using the camera intrinsic parameters obtained in Step 2 based on the Zhang Youzheng calibration method and the homography matrix H′ between the camera coordinate system and the world coordinate system, the following conversion formula between theoretical coordinates and actual coordinate points is obtained, namely, the expansion formula of the homography matrix H′:

[0087]

[0088] Where (x2, y2) are the camera image coordinates, and (x1, y1) are the actual physical locations of the camera image coordinates.

[0089] Expanding formula (7) yields:

[0090]

[0091] Then, the coordinate pose of the center point is calculated based on the coordinate information of the four corner points of the label.

[0092] The present invention also provides an unmanned vehicle positioning device based on indoor global vision, comprising:

[0093] A vision sensor, fixed to the indoor ceiling, is used to acquire images of the ground environment around the mobile robot in motion;

[0094] The mobile robot has two QR code labels on its top. The two QR codes identify the mobile robot and contain information about the robot's pose and speed.

[0095] An image processor (industrial computer) is used to receive ground environment images and perform distortion correction transformation on the ground environment images to obtain normal images;

[0096] The feature point matching module is used to extract feature points from normal images and store the feature matrix; then it performs feature matching on two or more normal images with feature points captured by different visual sensors.

[0097] The fusion module is used to fuse the feature-matched images to obtain a stitched wide-area panoramic image;

[0098] The pose information solving module is used to perform QR code recognition on a wide-area panoramic image to obtain the pose information of two QR codes; the pose information of the mobile robot is solved based on the placement height of the visual sensor, the size of the QR code, and the center position of the QR code.

[0099] Example

[0100] like Figure 1 , Figure 7 As shown, this embodiment provides a method for unmanned vehicle localization based on indoor global vision, including:

[0101] Step S100: The vision sensor collects the scene image in the preset environment where the mobile robot is located, and obtains the image for barrel distortion correction, as Figure 2 shown, so that the image is transformed into a distortion-free image. There are multiple vision sensors in the preset environment, and they are arranged on the indoor ceiling, as Figure 5 shown. The two-dimensional code label is arranged on the top of the mobile robot, as Figure 3 shown.

[0102] The above step S100 further includes the following sub-steps:

[0103] Step S101: Arrange the vision sensor, determine the position and attitude of the vision sensor, and adjust the brightness of the forehead light supporting the camera according to the experimental scene. In this embodiment, by measuring the relationship between the distance between the camera and the two-dimensional code label board and the positioning accuracy of the label board, the distance with the highest positioning accuracy is selected within the practical range. The USB camera is used in this embodiment. After measurement, the placement height is 300 cm from the ground vertically, and the shooting angle is that the axis line of the camera is perpendicular to the horizontal ground; because image fusion between cameras is required, there must be a certain overlapping part of the images between the two cameras, and the distance between the vision sensors (USB cameras) should be such that there is an overlapping part within the visible range; after fixing the position and height of the camera, analyze the positioning accuracy of the light source intensity, and select the light intensity with the highest positioning accuracy in this experimental environment.

[0104] Step S102: Adjust the light source brightness according to the current experimental environment so that the camera can collect a clear two-dimensional code image.

[0105] Step S103: Continuously collect clear environmental images in real time, and transmit the collected images to the host computer for the next image processing.

[0106] In this embodiment, the processor model is Intel Core i5-7300HQ CPU@2.50Hz.

[0107] Step S200: Perform noise reduction processing on the image obtained through step S100 and convert it into a grayscale image, and then through feature point extraction and matching fusion processing, finally convert the image into a panoramic image, as Figure 6 shown. e

[0108] In this embodiment, orb-stitch is used for image fusion.

[0109] Step S300: Perform two-dimensional code positioning and recognition on the panoramic image obtained through step S200, and obtain the pose information of the two-dimensional code.

[0110] In this embodiment, the size of the two-dimensional code is set to 20*20 cm, and the center distance between the two two-dimensional codes is 25 cm.

[0111] Step S400: Identify the QR code information obtained in step S300, and calculate the current pose information of the mobile robot using the visual sensor and the pose information of the QR code, such as... Figure 4 As shown. d represents the actual physical distance between the two QR codes; e represents the measured distance between the two QR codes; the accuracy of the system's positioning is evaluated by the variance of the measurement error in multiple sets of experimental data. The variance formula is:

[0112]

[0113] Step 400 also includes the following sub-steps:

[0114] Step S401: Recognize the QR code obtained in S300 to obtain relevant information about the mobile robot.

[0115] In step S402, the pose information of the QR code obtained in step S300 is calculated, and the pose information of the mobile robot is calculated based on the original size of the QR code and the height of the vision sensor, so as to obtain the position and acceleration change information of the mobile robot.

[0116] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the invention's content regarding an indoor global vision-based unmanned vehicle positioning method, as well as some or all of the steps in various embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0117] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MUU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0118] This invention provides a method and apparatus for unmanned vehicle positioning based on indoor global vision. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A method for unmanned vehicle localization based on indoor global vision, characterized in that, Includes the following steps: Step 1: Fix two or more vision sensors on the indoor ceiling to collect images of the ground environment around the mobile robot as it moves; place two QR code labels on the top of the mobile robot to identify it, and use the two QR codes to identify the mobile robot. The information on the labels includes the pose and speed information of the mobile robot. Step 2: The image processor performs distortion correction transformation on the ground environment image obtained in Step 1 to obtain a normal image; Step 3: Extract feature points from the normal image and store the feature matrix; then perform feature matching on two or more normal images with feature points captured by different visual sensors. Step 4: The images after feature matching in step 3 are fused together to obtain a stitched wide-area panoramic image. Step 5: Perform QR code recognition on the wide-area panoramic image to obtain the pose information of two QR codes; calculate the pose information of the mobile robot based on the placement height of the visual sensor, the size of the QR code, and the center position of the QR code. In step 3, the homography matrix H can be obtained using more than four sets of feature points; Step 5 includes: obtaining the camera intrinsic parameters and the homography matrix between the camera coordinate system and the world coordinate system using the Zhang Youzheng calibration method obtained in Step 2. The following conversion formula between theoretical coordinates and actual coordinates is obtained, i.e., the homography matrix. The expansion formula: （7）， in( ) is the camera image coordinate point, ( () represents the actual physical location of the camera image coordinates. In step 5, expanding formula (7) yields: （8）， Then, the coordinate pose of the center point is calculated based on the coordinate information of the four corner points of the label.

2. The method according to claim 1, characterized in that, Step 1 includes: Step 1-1: First, determine the placement and orientation of the cameras with two or more vision sensors, and adjust the brightness of the matching light source for the cameras and the spacing between the vision sensors according to the actual scene. Steps 1-2: Adjust the parameters of the vision sensor according to the current light source brightness, collect QR code information, and compare the variance of the QR code center position measurement with the actual variance to obtain the confidence level of the positioning. Steps 1-3: Real-time acquisition of continuous ground environment images around the mobile robot in motion, and transmission of the acquired ground environment images to the image processor.

3. The method according to claim 2, characterized in that, Steps 1-2 include: Let d be the actual physical distance between the center points of the two QR codes; For the current frame The distance between two QR codes; For the current frame The measurement error is used to assess the accuracy of positioning by measuring the variance of the measurement error. The variance formula is: ， Where s 2 denoted by , where is the variance of the time-lapse measurement for the entire video frame, n is the total number of frames in the video stream, and w represents the average value of the measurement error for the n video frames.

4. The method according to claim 3, characterized in that, Step 2 includes: the visual sensor is a camera, and the camera is calibrated using the Zhang Zhengyou calibration method to obtain camera parameter information (f). x ,f y ,c x ,c y ), where f x f is the focal length of the camera along the x-axis. y Let c be the focal length along the y-axis of the camera. x It is the x-coordinate of the focal center in the pixel coordinate system, c y This is the y-coordinate of the focal center in the camera coordinate system; the distorted image is known, and the mapping relationship is derived through the distortion model to obtain the undistorted image: The relationship between the real image imgR and the distorted image imgD is: imgR(u,v) = imgD(u d ,u v ); imgR(u,v) represents the pixel value of the real image imgR at the x-coordinate u and y-coordinate v in the pixel coordinate system, imgD(u d ,u v The ) represents the x-coordinate u of the distorted image imgD in the pixel coordinate system. d , ordinate u v Pixel value at; The formula for converting a real image to a distorted image is: （1）， （2）， （3）， ) is the location in the real image imgR ( The position after projection transformation and coordinate system transformation. ) represents the distortion position coordinates of any point in the distorted image with the camera coordinate system as the origin, where It is the mirror distortion coefficient. It is the tangential deformation coefficient; It is the radius from the current pixel to the center of the circle. .

5. The method according to claim 4, characterized in that, In step 2, through the transformation using formulas (1), (2), and (3), the correspondence between the pixels of the real image and the pixels of the distorted image is obtained, thereby transforming the distorted image imgD... The pixel values ​​are inserted into imgR .

6. The method according to claim 5, characterized in that, Step 3 includes: In a system consisting of two or more vision sensors, if any two cameras have overlapping areas, the images from these two cameras are taken and denoted as image1 and image2 respectively for registration. First, feature point 1 in image1 is obtained. and feature points 2 in image2 The corresponding homography matrix is ​​H, where The parameter in the i-th row and j-th column of matrix H ; s is the image scale factor, and the coordinates of feature point 1 are... The coordinates of feature point 2 are : （4）， When s takes any value, let s = Then matrix H is rewritten as: （5）， After editing, it is written as: （6）。 7. A localization device for unmanned vehicles based on indoor global vision, implemented using the method described in claim 1, characterized in that, include: A vision sensor, fixed to the indoor ceiling, is used to acquire images of the ground environment around the mobile robot in motion; The mobile robot has two QR code labels on its top. The two QR codes identify the mobile robot and contain information about the robot's pose and speed. An image processor is used to receive ground environment images and perform distortion correction transformation on the ground environment images to obtain normal images; The feature point matching module is used to extract feature points from normal images and store the feature matrix; then it performs feature matching on two or more normal images with feature points captured by different visual sensors. The fusion module is used to fuse the feature-matched images to obtain a stitched wide-area panoramic image; The pose information solving module is used to perform QR code recognition on a wide-area panoramic image to obtain the pose information of two QR codes; the pose information of the mobile robot is solved based on the placement height of the visual sensor, the size of the QR code, and the center position of the QR code.

Images