A visual positioning device, a positioning method and a readable storage medium

Abstract

This application relates to the field of visual positioning, and particularly to a visual positioning device, positioning method, and readable storage medium. The visual positioning device includes a first beacon plane, which includes a first visual QR code and a first checkerboard area; a second beacon plane, which includes a second visual QR code and a second checkerboard area; and a reference plane on which the first and second beacon planes are disposed. Neither the first nor the second beacon plane is on the same plane as the reference plane. The reference plane faces the camera to be calibrated to assist the camera in performing visual positioning based on the first and second beacon planes. By using beacon corner points on different planes, the depth measurement accuracy and rotation detection accuracy of the visual positioning device can be improved, thereby increasing the resolution of the photographed object in the image and thus improving the accuracy of visual positioning of the object.

Description

Technical Field

[0001] This application relates to the field of visual positioning, and in particular to a visual positioning device, positioning method and readable storage medium. Background Technology

[0002] In robot vision applications, cameras are generally used for size detection, while visual calibration boards are used to set camera parameters for the machine vision system and improve image segmentation and object recognition performance. In related technologies, localization requires visual beacons to determine the positional relationship between the robot's camera and the beacon. Commonly used visual beacons include AprilTag, ARTag, STag, and aruco. However, these markers are generally planar, leading to lower calibration accuracy. Accuracy is typically improved by increasing the number of calibration boards for more precise position estimation. However, this technique results in a large field of view for the localization device, requiring multiple images from different locations for calibration. When the lens focal length is large or the camera resolution is insufficient, the resulting calibration accuracy error becomes unacceptable for the robotic arm. Summary of the Invention

[0003] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0004] This application aims to at least address one of the technical problems existing in the prior art. To this end, embodiments of this application provide a visual positioning device, a positioning method, and a readable storage medium, which helps to solve the problem that the visual positioning device has a large field of view and requires multiple photos for calibration. This application sets a first beacon plane and a second beacon plane on a reference plane, with the reference plane facing the camera to be calibrated to assist the camera in performing visual positioning based on the first and second beacon planes. Only one photo needs to be taken for calibration, reducing the field of view of the positioning device and improving the resolution of the captured image, thereby further improving the accuracy of visual positioning.

[0005] In a first aspect, embodiments of this application provide a visual positioning device, comprising:

[0006] A first beacon plane, the first beacon plane including a first visual QR code and a first checkerboard area;

[0007] The second beacon plane includes a second visual QR code and a second checkerboard area;

[0008] A reference plane is provided, on which the first beacon plane and the second beacon plane are disposed. The first beacon plane and the second beacon plane are not on the same plane as the reference plane, nor are they on the same plane. The reference plane is oriented toward the camera to be calibrated to assist the camera in visual positioning based on the first beacon plane and the second beacon plane.

[0009] The technical solution of the first aspect of this application has at least one of the following advantages or beneficial effects: By setting a first beacon plane and a second beacon plane on a reference plane, and ensuring that the first and second beacon planes are not on the same plane, and that the reference plane is not on the same plane as either the first or second beacon plane, a three-dimensional visual positioning device is provided, improving the reliability of the visual positioning device. Simultaneously, by setting the reference plane to face the camera to be calibrated to assist the camera in visual positioning based on the first and second beacon planes, the depth measurement accuracy and rotation detection accuracy of the visual positioning device are improved, resulting in higher resolution of the photographed object in the image, thereby improving the accuracy of visual positioning. Furthermore, this application only requires taking one photograph for calibration, reducing the field of view of the positioning device and simultaneously increasing the resolution of the captured image, further improving the accuracy of visual positioning.

[0010] Furthermore, a visual clearance groove is provided between the first beacon plane and the second beacon plane. The visual clearance groove is used to prevent the first beacon plane from obscuring the second beacon plane when the camera is not perpendicular to the reference plane.

[0011] Furthermore, it also includes a positioning pin mounting seat for mounting a positioning pin, and the visual relief groove also includes a positioning pin mounting hole, the positioning pin mounting seat and the positioning pin mounting hole cooperating with each other.

[0012] Secondly, embodiments of this application provide a visual positioning method applied to a camera, wherein the camera is used to capture images toward the visual positioning device described in the first aspect of the technical solution; the visual positioning method includes:

[0013] The camera captures an image of the visual positioning device.

[0014] On the captured image, a first visual QR code beacon is determined in the first beacon plane based on the first beacon identifier, and a second visual QR code beacon is determined in the second beacon plane based on the second beacon identifier;

[0015] The pose of the reference plane relative to the camera is estimated based on the position and pose of the first visual QR code beacon and the position and pose of the second visual QR code beacon;

[0016] Based on the posture information of the QR code, the first region of interest and the second region of interest on the captured image are rotated and transformed, wherein the first region of interest corresponds to the first checkerboard area and the second region of interest corresponds to the second checkerboard area.

[0017] In the first region of interest, a first target corner point corresponding to the first beacon plane is identified; in the second region of interest, a second target corner point corresponding to the second beacon plane is identified.

[0018] The pose of the camera from the first beacon plane is obtained based on the first target corner point, and the pose of the camera from the second beacon plane is obtained based on the second target corner point.

[0019] The technical solution of the second aspect of this application has at least one of the following advantages or beneficial effects: By determining the first visual QR code beacon in the first beacon plane based on the first beacon representation and the second visual QR code beacon in the second beacon plane based on the second beacon identifier on the captured image by the camera, the visual positioning device is accurately located for calibration. The pose of the reference plane relative to the camera is estimated based on the position and pose of the first and second visual QR codes. The region of interest on the captured image is determined through the pose information of the QR codes, the corresponding target corner point is identified in the region of interest, and the pose from the beacon plane to the camera is obtained based on the target corner point, thereby achieving position calibration. This calibration method improves the resolution of the captured object in the image, thereby improving the accuracy of visual positioning of the object. In addition, in this application, only one image of the visual positioning device needs to be captured by the camera for calibration, reducing the field of view of the positioning device and further improving the accuracy of visual positioning.

[0020] Further, the step of determining the first visual QR code beacon in the first beacon plane based on the first beacon identifier includes:

[0021] Obtain the pose from the camera coordinates to the first beacon plane coordinates and the rotation matrix from the first chessboard area to the camera coordinates;

[0022] A coordinate system is created based on the pose from the camera coordinates to the first beacon plane coordinates and the rotation matrix from the first checkerboard area to the camera coordinates;

[0023] Obtain the relative coordinates of the first checkerboard area to the reference plane under the first beacon plane coordinates;

[0024] The coordinates from the first beacon marker to the camera are obtained based on the relative coordinates of the first checkerboard area to the reference plane under the first beacon plane coordinates;

[0025] The corresponding first visual QR code beacon is determined based on the coordinates of the first beacon identifier to the camera.

[0026] Furthermore, after estimating the pose of the reference plane relative to the camera based on the position and pose of the first visual QR code beacon and the position and pose of the second visual QR code beacon, the method further includes:

[0027] The dividing line is determined based on the posture information of the QR code;

[0028] The first region of interest in the first beacon plane and the second region of interest in the second beacon plane are determined based on the dividing lines.

[0029] Furthermore, after determining the first region of interest in the first beacon plane and the second region of interest in the second beacon plane based on the dividing line, the method further includes:

[0030] By performing rotational projection transformations on rectangular frames of the first and second preset sizes respectively, a first oblique rectangular frame and a second oblique rectangular frame are obtained;

[0031] The first checkerboard region on the first beacon plane in the captured image is identified as the first region of interest using the first oblique rectangle.

[0032] The second checkerboard region on the second beacon plane in the captured image is identified as the second region of interest using the second oblique rectangle.

[0033] Furthermore, identifying the first target corner point corresponding to the first beacon plane in the first region of interest includes:

[0034] Obtain the displacement marker of the i-th corner point of the first chessboard area relative to the center of the first beacon plane;

[0035] Determine the first target corner point in the first beacon plane that corresponds to the displacement mark.

[0036] Furthermore, determining the dividing line based on the posture information of the QR code includes:

[0037] Determine the path whose distance from the first visual QR code to the midpoint coordinates is equal to the distance from the second visual QR code to the midpoint coordinates;

[0038] The path is determined as the posture information of the QR code;

[0039] The dividing line is determined based on the posture information of the QR code.

[0040] Thirdly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which are used to execute the visual positioning method described in any one of the technical solutions of the second aspect. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the structure of a visual positioning device provided in an embodiment of this application;

[0042] Figure 2 This is a schematic diagram of another visual positioning device provided in an embodiment of this application;

[0043] Figure 3 This is a schematic diagram of another visual positioning device provided in an embodiment of this application;

[0044] Figure 4 This is a schematic diagram of the structure of a positioning pin mounting base provided in an embodiment of this application;

[0045] Figure 5 This is a visual positioning method provided in an embodiment of this application;

[0046] Figure 6 yes Figure 5 Flowchart of steps in S200;

[0047] Figure 7 This is another visual positioning method provided in the embodiments of this application;

[0048] Figure 8 This is another visual positioning method provided in the embodiments of this application;

[0049] Figure 9 yes Figure 5 Flowchart of the steps in the S500;

[0050] Figure 10 yes Figure 7 Flowchart of steps in S310;

[0051] Figure 11 This is a schematic diagram of the structure of a controller provided in an embodiment of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] In the description of this application, "multiple" refers to two or more. The use of "first" and "second" is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance, or implicitly indicating the number of technical features indicated, or the order in which the technical features are indicated.

[0054] In robot vision applications, cameras are generally used for size detection, while visual calibration boards are used to set camera parameters for the machine vision system and improve image segmentation and object recognition performance. In related technologies, localization requires visual beacons to determine the positional relationship between the robot's camera and the beacon. Commonly used visual beacons include AprilTag, ARTag, STag, and aruco. However, these markers are generally planar, leading to lower calibration accuracy. Accuracy is typically improved by increasing the number of calibration boards for more precise position estimation. However, this technique results in a large field of view for the localization device, requiring multiple images from different locations for calibration. When the lens focal length is large or the camera resolution is insufficient, the resulting calibration accuracy error becomes unacceptable for the robotic arm.

[0055] To address this issue, this application provides a visual positioning device, a positioning method, and a readable storage medium. This addresses the problem of visual positioning devices having a large field of view and requiring multiple images for calibration. By setting a first beacon plane and a second beacon plane on a reference plane, with the reference plane facing the camera to be calibrated, this application assists the camera in visual positioning based on the first and second beacon planes. This improves the depth measurement and rotation detection accuracy of the visual positioning device, increasing the resolution of the photographed object in the image and thus enhancing the accuracy of visual positioning. Furthermore, this application only requires taking one image for calibration, reducing the field of view of the positioning device while simultaneously increasing the resolution of the captured image, further improving the accuracy of visual positioning.

[0056] Reference Figure 1 and Figure 2 , Figure 1This is a schematic diagram of the structure of a visual positioning device 1000 provided in an embodiment of this application. It includes a first beacon plane 100, a second beacon plane 200, and a reference plane 300. The first beacon plane 100 includes a first visual QR code 110 and a first checkerboard area 120. The second beacon plane 200 includes a second visual QR code 210 and a second checkerboard area 220. The first beacon plane 100 and the second beacon plane 200 are disposed on the reference plane 300. The first beacon plane 100 and the second beacon plane 200 are not on the same plane as the reference plane 300, and the first beacon plane 100 and the second beacon plane 200 are not on the same plane. The reference plane 300 faces the camera (not shown in the figure) to be calibrated to assist the camera in performing visual positioning based on the first beacon plane 100 and the second beacon plane 200.

[0057] By setting a first beacon plane 100 and a second beacon plane 200 on a reference plane 300, and ensuring that the first beacon plane 100 and the second beacon plane 200 are not on the same plane, and that the reference plane 300 is also not on the same plane as the first beacon plane 100 and the second beacon plane 200, a three-dimensional visual positioning device 1000 is provided. This improves the reliability of the visual positioning device 1000. Furthermore, by setting the reference plane 300 to face the camera to be calibrated (not shown in the figure) to assist the camera in visual positioning based on the first beacon plane 100 and the second beacon plane 200, the depth measurement accuracy and rotation detection accuracy of the visual positioning device 1000 are improved, resulting in higher resolution of the photographed object in the image, thereby improving the accuracy of visual positioning. In addition, this application only requires taking one photograph for calibration, reducing the field of view of the positioning device and increasing the resolution of the captured image, further improving the accuracy of visual positioning.

[0058] It should be noted that in the embodiments of this application, the first beacon plane 100 and the reference plane 300 are not on the same plane, and the first beacon plane 100 and the reference plane 300 directly form a first included angle, wherein the first included angle is less than 20°.

[0059] It should be noted that in this embodiment, the second beacon plane 200 and the reference plane 300 are not on the same plane, and the second beacon plane 200 and the reference plane 300 directly form a second included angle, wherein the second included angle is less than 20°.

[0060] It should be noted that, in this embodiment of the application, by setting a first included angle and a second included angle, the reference plane 300 and the first beacon plane 100 and the second beacon plane 200 are not on the same plane, thus providing a three-dimensional visual positioning device 1000 and improving the realism of visual positioning. This embodiment of the application does not limit the size of the first included angle and the second included angle.

[0061] It should be noted that, in this embodiment of the application, by setting a first visual QR code and a second visual QR code, the redundancy of the data is increased, ensuring that the second visual QR code can still be recognized even if the first visual QR code fails to be recognized, thereby improving the accuracy of the visual positioning method.

[0062] Reference Figure 3 and Figure 4 , Figure 3 This is a schematic diagram of another visual positioning device 1000 provided in an embodiment of this application. Figure 4 This is a schematic diagram of a positioning pin mounting base 500 provided in an embodiment of this application. A visual clearance groove 400 is provided between the first beacon plane 100 and the second beacon plane 200. The visual clearance groove 400 is used to prevent the first beacon plane 100 from obstructing the second beacon plane 200, and to prevent the second beacon plane 200 from obstructing the first beacon plane 100, when the camera is not perpendicular to the reference plane 300. The visual positioning device also includes a positioning pin mounting base 500 for mounting a positioning pin 510. The visual clearance groove 400 also includes a positioning pin mounting hole 410, and the positioning pin mounting base 500 and the positioning pin mounting hole 410 cooperate with each other.

[0063] A visual clearance groove 400 is provided between the first beacon plane 100 and the second beacon plane 200. When the camera is not perpendicular to the reference plane 300, the visual clearance groove 400 prevents the first beacon plane 100 from obstructing the second beacon plane 200, ensuring that the calibration of the first and second beacon planes does not interfere with each other, thus guaranteeing the integrity of the visual positioning device 1000 and improving the positioning accuracy. Simultaneously, the positioning pin 510 is mounted on a positioning pin mounting seat 500, and the positioning pin mounting seat 500 and the positioning pin mounting hole 410 cooperate with each other, further improving the calibration accuracy of the visual positioning device 1000.

[0064] Reference Figure 5 , Figure 5 This application provides a visual positioning method applied to a camera, which is used to take pictures of a visual positioning device. The visual positioning method includes steps S100 to S600, specifically...

[0065] S100: The camera captures an image of the visual positioning device;

[0066] S200: On the captured image, a first visual QR code beacon in the first beacon plane is determined based on the first beacon identifier, and a second visual QR code beacon in the second beacon plane is determined based on the second beacon identifier;

[0067] S300: Estimate the pose of the reference plane relative to the camera based on the position and pose of the first visual QR code beacon and the second visual QR code beacon;

[0068] S400: Rotate and transform the first region of interest and the second region of interest on the captured image according to the posture information of the QR code. The first region of interest corresponds to the first checkerboard area, and the second region of interest corresponds to the second checkerboard area.

[0069] S500: Identify a first target corner point corresponding to a first beacon plane in a first region of interest, and identify a second target corner point corresponding to a second beacon plane in a second region of interest;

[0070] S600: Obtain the pose from the first beacon plane to the camera based on the first target corner point, and obtain the pose from the second beacon plane to the camera based on the second target corner point.

[0071] By identifying a first visual QR code beacon in a first beacon plane based on a first beacon identifier and a second visual QR code beacon in a second beacon plane based on a second beacon identifier in an image captured by a camera, the visual positioning device is accurately located for calibration. The pose of the reference plane relative to the camera is estimated based on the position and pose of the first and second visual QR codes. The region of interest (ROI) in the captured image is determined using the QR code's pose information. Corresponding target corner points are identified within the ROI, and the pose from the beacon plane to the camera is obtained based on these target corner points, thus achieving position calibration. This calibration method improves the resolution of the captured object in the image, thereby enhancing the accuracy of visual positioning. Furthermore, this application only requires the camera to capture one image of the visual positioning device for calibration, reducing the field of view of the positioning device while simultaneously increasing the resolution of the captured image, further improving the accuracy of visual positioning.

[0072] It should be noted that the first beacon identifier and the second beacon identifier are different, thus effectively distinguishing the first beacon plane and the second beacon plane; the first visual QR code is determined based on the first beacon identifier to determine the location as the first beacon plane, and the second visual QR code is determined based on the second beacon identifier to determine the location as the second beacon plane, thereby improving the accuracy of the visual positioning method.

[0073] It should be noted that after identifying the first target corner point corresponding to the first beacon plane in the first region of interest, and identifying the second target corner point corresponding to the second beacon plane in the second region of interest, the pose of the first and second target corner points relative to the camera is solved by the PNP algorithm.

[0074] Reference Figure 6 , Figure 6 yes Figure 5The flowchart of steps S200 includes steps S210 to S250, specifically...

[0075] S210: Obtain the pose from the camera coordinates to the first beacon plane coordinates and the rotation matrix from the first checkerboard area to the camera coordinates;

[0076] S220: Create a coordinate system based on the pose from the camera coordinates to the first beacon plane coordinates and the rotation matrix from the first checkerboard area to the camera coordinates;

[0077] S230: Obtain the relative coordinates of the first checkerboard area to the reference plane under the first beacon plane coordinates;

[0078] S240: Obtain the coordinates from the first beacon marker to the camera based on the relative coordinates from the first checkerboard area to the reference plane under the first beacon plane coordinates;

[0079] S250: Determine the corresponding first visual QR code beacon based on the coordinates from the first beacon identifier to the camera.

[0080] In one embodiment, a coordinate system is created based on the pose from the camera coordinates to the first beacon plane coordinates and the rotation matrix from the first checkerboard area to the camera coordinates. The coordinate system is obtained by the following formula.

[0081] board R cam = board R left left R cam ,

[0082] in, board R cam This refers to the pose from the camera coordinates to the coordinates of the first beacon plane. left R cam It refers to the rotation matrix from the first chessboard area to the camera coordinates.

[0083] Obtain the relative coordinates of the first checkerboard area to the reference plane under the first beacon plane coordinates, and obtain the coordinates of the first beacon identifier to the camera based on the relative coordinates of the first checkerboard area to the reference plane under the first beacon plane coordinates; determine the corresponding first visual QR code beacon based on the coordinates of the first beacon identifier to the camera, and determine the current position as the first beacon plane based on the first visual QR code.

[0084] Reference Figure 7 , Figure 7 This application provides another visual positioning method. After estimating the pose of the reference plane relative to the camera based on the position and pose of the first visual QR code beacon and the second visual QR code beacon, the positioning method further includes steps S310 and S320. Specifically,

[0085] S310: Determine the dividing line based on the posture information of the QR code;

[0086] S320: Determine the first region of interest in the first beacon plane and the second region of interest in the second beacon plane based on the dividing line.

[0087] The segmentation lines are determined based on the pose information of the QR code, and then the first region of interest (ROI) on the first beacon plane and the second ROI on the second beacon plane are determined based on the segmentation lines. The first ROI corresponds to the first checkerboard area, and the second ROI corresponds to the second checkerboard area. By determining the segmentation lines using the pose information of the QR code, the ROI in the captured image is confirmed. This allows for localization and detection using only one type of captured image, improving image resolution and the accuracy of the visual localization method.

[0088] It should be noted that, in the embodiments of this application, the first and second checkerboard areas can be replaced with non-checkerboard calibration patterns for calibration. Specifically, the first checkerboard area can be replaced with a first QR code array calibration board for positioning detection, and the second checkerboard area can be replaced with a second QR code array calibration board. When using the first and second QR code array calibration boards for positioning, steps S310 and S320 can be omitted, achieving visual positioning.

[0089] Reference Figure 8 , Figure 8 This is another visual positioning method provided in the embodiments of this application. After determining the first region of interest of the first beacon plane and the second region of interest of the second beacon plane based on the dividing line, the visual positioning method further includes steps S330 and S350, specifically,

[0090] S330: Perform rotational projection transformations using rectangles of the first and second preset sizes respectively to obtain the first oblique rectangle and the second oblique rectangle.

[0091] S340: Using the first oblique rectangle, identify the first checkerboard area on the first beacon plane in the captured image as the first region of interest;

[0092] S350: The second checkerboard region on the second beacon plane in the captured image is identified as the second region of interest using the second oblique rectangle.

[0093] The first checkerboard area on the first beacon plane in the captured image is identified as the first region of interest (ROI) using a first oblique rectangle. Similarly, the second checkerboard area on the second beacon plane in the captured image is identified as the second ROI using a second oblique rectangle, further defining the ROI in the captured image. The first and second ROIs are rotated according to the pose information of the QR code to ensure that the paths of the corner points in the ROIs remain consistent. This allows for calibration using only one image, improving the resolution of the captured image and further enhancing the accuracy of the visual positioning method.

[0094] It should be noted that the rectangle of the first preset size is larger than the first checkerboard area and can cover the first checkerboard area. By setting the rectangle of the first preset size to be larger than the first checkerboard area, it is ensured that the target corner points can be identified, thus improving the accuracy of the visual positioning method. In this application, the first preset size is set according to the size of the first checkerboard area. The embodiments of this application do not limit the size of the first size.

[0095] It should be noted that the rectangle of the second preset size is larger than the second checkerboard area and can cover the second checkerboard area. By setting the rectangle of the second preset size to be larger than the second checkerboard area, it is ensured that the target corner points can be identified, thus improving the accuracy of the visual positioning method. In this application, the second preset size is set according to the size of the second checkerboard area. The embodiments of this application do not limit the size of the second size.

[0096] Reference Figure 9 , Figure 9 yes Figure 5 The flowchart of steps S500 includes steps S510 and S520, specifically...

[0097] S510: Obtain the displacement marker of the i-th corner point of the first chessboard area relative to the center of the first beacon plane;

[0098] S520: Determine the first target corner point in the first beacon plane corresponding to the displacement mark.

[0099] The first target corner point is determined by the displacement mark of the i-th corner point of the first checkerboard area relative to the center of the first beacon plane, thereby realizing the calibration of the first target corner point on the first beacon plane. Each corner point in the first checkerboard area corresponds to a target corner point, which improves the accuracy of the positioning method.

[0100] Reference Figure 10 , Figure 10 yes Figure 7 The flowchart of step S310 includes steps S311 to S313, specifically...

[0101] S311: Determine the path from the first visual QR code to the midpoint coordinates that is equal to the distance from the second visual QR code to the midpoint coordinates;

[0102] S312: Determine the orientation information of the path as a QR code;

[0103] S313: Determine the dividing line based on the posture information of the QR code.

[0104] By combining the first visual QR code, the second visual QR code, and the midpoint coordinates, the path from the first visual QR code to the midpoint coordinates that is equal to the distance from the second visual QR code to the midpoint coordinates is determined as the QR code's pose information. Based on the QR code's pose information, a dividing line is determined, and then the region of interest (ROI) on the captured image is determined based on the dividing line. The first and second ROIs are rotated according to the QR code's pose information to ensure that the paths of the corner points on the ROIs remain consistent. This allows for calibration using only one image, improving the resolution of the captured image and further enhancing the accuracy of the visual positioning method.

[0105] It should be noted that the midpoint coordinates are the midpoint coordinates in the reference plane coordinate system.

[0106] Reference Figure 11 , Figure 11This is a schematic diagram of the structure of a controller 2000 provided in an embodiment of this application. The hardware structure of the controller 2000 includes a processor 2001, which can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement a visual positioning method provided in this embodiment of the application; and a memory 2002, which can be implemented using a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM), etc. The memory 1002 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 2002 and is called and executed by the processor 2001. The input / output interface 2003 is used to implement information input and output. The communication interface 2004 is used to realize communication interaction between this device and other devices. Communication can be realized through wired means (such as USB, network cable, etc.) or through wireless means (such as mobile network, WIFI, Bluetooth, etc.). The bus transmits information between various components of the device (such as the processor 2001, memory 2002, input / output interface 2003 and communication interface 2004). The processor 2001, memory 2002, input / output interface 2003 and communication interface 2004 are connected to each other within the device through the bus.

[0107] This application also provides a storage medium, which is a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements a flowchart of the aforementioned visual positioning method. As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0108] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. A visual positioning method, characterized in that, An application to a camera used to take pictures toward a visual positioning device; the visual positioning device includes: A first beacon plane, the first beacon plane including a first visual QR code and a first checkerboard area; The second beacon plane includes a second visual QR code and a second checkerboard area; A reference plane is provided, on which the first beacon plane and the second beacon plane are disposed. The first beacon plane and the second beacon plane are not on the same plane as the reference plane, nor are they on the same plane. The reference plane is oriented toward the camera to be calibrated to assist the camera in visual positioning based on the first beacon plane and the second beacon plane. The visual positioning method includes: The camera captures an image of the visual positioning device. On the captured image, a first visual QR code beacon is determined in the first beacon plane based on the first beacon identifier, and a second visual QR code beacon is determined in the second beacon plane based on the second beacon identifier; The pose of the reference plane relative to the camera is estimated based on the position and pose of the first visual QR code beacon and the position and pose of the second visual QR code beacon; Based on the posture information of the QR code, the first region of interest and the second region of interest on the captured image are rotated and transformed, wherein the first region of interest corresponds to the first checkerboard area and the second region of interest corresponds to the second checkerboard area. In the first region of interest, a first target corner point corresponding to the first beacon plane is identified; in the second region of interest, a second target corner point corresponding to the second beacon plane is identified. The pose of the first beacon plane to the camera is obtained based on the first target corner point, and the pose of the second beacon plane to the camera is obtained based on the second target corner point.

2. The visual positioning method according to claim 1, characterized in that, The step of determining the first visual QR code beacon in the first beacon plane based on the first beacon identifier includes: Obtain the pose from the camera coordinates to the first beacon plane coordinates and the rotation matrix from the first chessboard area to the camera coordinates; A coordinate system is created based on the camera coordinates to the first beacon plane coordinate pose and the rotation matrix from the first checkerboard area to the camera coordinates; Obtain the relative coordinates of the first checkerboard area to the reference plane under the first beacon plane coordinates; The coordinates from the first beacon marker to the camera are obtained based on the relative coordinates of the first checkerboard area to the reference plane under the first beacon plane coordinates; The corresponding first visual QR code beacon is determined based on the coordinates of the first beacon identifier to the camera.

3. The visual positioning method according to claim 1, characterized in that, After estimating the pose of the reference plane relative to the camera based on the position and pose of the first visual QR code beacon and the position and pose of the second visual QR code beacon, the method further includes: The dividing line is determined based on the posture information of the QR code; The first region of interest in the first beacon plane and the second region of interest in the second beacon plane are determined based on the dividing lines.

4. The visual positioning method according to claim 3, characterized in that, After determining the first region of interest of the first beacon plane and the second region of interest of the second beacon plane based on the dividing line, the method further includes: By performing rotational projection transformations on rectangles of the first and second preset sizes respectively, a first oblique rectangle and a second oblique rectangle are obtained. The first checkerboard region on the first beacon plane in the captured image is identified as the first region of interest using the first oblique rectangle. The second checkerboard region on the second beacon plane in the captured image is identified as the second region of interest using the second oblique rectangle.

5. The visual positioning method according to claim 1, characterized in that, The step of identifying the first target corner point corresponding to the first beacon plane in the first region of interest includes: Obtain the displacement marker of the i-th corner point of the first chessboard area relative to the center of the first beacon plane; Determine the first target corner point in the first beacon plane that corresponds to the displacement mark.

6. The visual positioning method according to claim 3, characterized in that, Determining the dividing line based on the posture information of the QR code includes: Determine the path whose distance from the first visual QR code to the midpoint coordinates is equal to the distance from the second visual QR code to the midpoint coordinates; The path is determined as the posture information of the QR code; The dividing line is determined based on the posture information of the QR code.

7. The visual positioning method according to claim 1, characterized in that, A visual clearance groove is provided between the first beacon plane and the second beacon plane. The visual clearance groove is used to prevent the first beacon plane from obscuring the second beacon plane and to prevent the second beacon plane from obscuring the first beacon plane when the camera is not perpendicular to the reference plane.

8. The visual positioning method according to claim 7, characterized in that, The visual positioning device further includes a positioning pin mounting seat for mounting a positioning pin, and the visual relief groove further includes a positioning pin mounting hole, wherein the positioning pin mounting seat and the positioning pin mounting hole cooperate with each other.

9. A computer-readable storage medium, characterized in that: The device stores computer-executable instructions for performing the visual positioning method as described in any one of claims 1 to 8.

Images