Precise positioning system

Abstract

The present invention discloses a precise positioning system which can achieve a positioning accuracy of less than several millimeters. The system is composed of a camera, a microcontroller, an imaging panel, an inertial measurement unit and an altimeter. The central area of ​​the imaging panel is made of an optical filtering material, and the remaining area is made of a diffusion material. The camera captures images of edgeless or projected markers and obtains their posture relative to a landmark coordinate system. If the edgeless markers and projected markers are fixed at certain positions, their global postures are known. When the relationship between the positioning system and the camera coordinate system, the marker coordinate system and the world coordinate system is known, the global posture of the positioning system relative to the world coordinate system can be obtained. The altimeter can be used to obtain height information. Using the height information and the global posture information in the XY plane, the global posture of the system in three-dimensional space can be determined.

Description

Technical Field

[0001] The invention discloses an imaging panel, a camera, an inertial measurement unit, an altimeter and an MCU forming a precise positioning system for obtaining its attitude relative to a passive or projected marker. Background Art

[0002] Precise positioning systems are essential for the navigation of mobile robots such as unmanned aerial vehicles (UAVs) or unmanned ground vehicles (UGVs). For certain applications that require mobile operation, positioning and localization accuracy are critical.

[0003] There are already positioning systems on the market that use ultrasonic sensors, infrared sensors, laser rangefinders, wireless beacons and vision.

[0004] The problem with the positioning system based on ultrasonic or infrared sensors is that the system can only provide position information but not orientation information. In order to facilitate navigation, mobile robots need additional sensors to obtain orientation information.

[0005] The problem with laser rangefinder-based positioning systems is that in certain dynamic environments, the accuracy of the calculated position may decrease. Sometimes, in certain situations, it is impossible to obtain your position at all.

[0006] There is a vision-based positioning system that uses a camera to obtain its pose from a marker on the ground. However, the marker is easily damaged, or it is not allowed to be placed on the ground in some places. In addition, this system cannot be used for positioning on UAVs.

[0007] The positional accuracy of vision-based systems using visual simultaneous localization and mapping (vSLAM) can be affected by changing lighting conditions and dynamic environments. Sometimes, it is simply impossible to determine one’s position in certain situations.

[0008] Wireless positioning systems suffer from the uncertainty of non-line-of-sight conditions and radio multipath problems, which can affect positioning accuracy.

[0009] As an example of a marker-based system, patent WO 2004 / 015369 A2 discloses a tracking, automatic calibration and map generation system with artificial markers on the ceiling as one of the positioning methods. Patent CN 102419178 A discloses a mobile robot positioning system and method based on capturing infrared markers on the ceiling. Patent CN 102135429 A discloses an indoor positioning system based on passive markers on the ceiling. Patent WO 2008 / 013355 A1 discloses a system and method for calculating position using a combination of odometers and illuminated infrared markers.

[0010] These systems use cameras to capture images of passive or illuminated artificial markers. The greater the distance between the camera and the marker, the lower the positioning accuracy. In addition, these markers do not contain sub-markers of different sizes and styles to facilitate image recognition from different distances. In addition, the effective image recognition distance of passive or illuminated artificial markers is shorter than that of projected markers. Summary of the invention

[0011] The object of the present invention is to provide a positioning system which can be used for UAVs and UGVs with a definite and precise positioning accuracy.

[0012] According to the present invention, a precise positioning system capable of achieving a positioning accuracy of less than several millimeters includes an imaging panel, a camera, an inertial measurement unit, an altimeter and an MCU.

[0013] According to a first aspect of the invention, a passive or projected tag consists of a sub-tag incorporating large, medium and small two-dimensional codes, and a set of solid and hollow circles and squares surrounding the two-dimensional codes.

[0014] According to the second aspect of the present invention, the imaging panel is located within the focal length of the camera. The central area of ​​the imaging panel is made of a light filtering material for filtering out unwanted spectra, and the camera can directly see the passive marker through the filter and obtain its posture relative to the marker. The remaining area of ​​the imaging panel is made of a diffusion material for capturing the marker image projected thereon.

[0015] According to the third aspect of the present invention, an imaging panel, a camera and an MCU may be installed on the mobile robot to obtain its global posture information.

[0016] According to a fourth aspect of the invention, an altimeter on the mobile robot may be used to obtain its altitude information, and an inertial measurement unit together with the mobile robot's odometer may be used to estimate its position when information cannot be obtained from markers or projected markers.

[0017] According to a fifth aspect of the invention, passive or projected markers may be placed on the pallet to facilitate precise alignment between the pallet and a mobile forklift for maneuvering.

[0018] According to the sixth aspect of the present invention, directional RFID tags can be placed on pallets or cabinets used for mobile forklifts to understand their general posture, and then precise manipulation tasks can be performed with the help of passive or projected markers thereon.

[0019] According to a seventh aspect of the invention, passive markers or projected markers may be placed on a person's coat to enable a mobile robot to follow.

[0020] According to an eighth aspect of the present invention, there is provided an unmanned aerial vehicle (UAV) equipped with an imaging panel, a camera, an inertial measurement unit, a light detection and ranging sensor, an infrared projector, an altimeter, and an MCU for navigating around a building.

[0021] According to a ninth aspect of the invention, building lighting fixtures can be used as reference positions for drone navigation. Passive markers and projected markers can be mounted next to the lighting fixtures for drone positioning. The projector projects the marker downwards for capture by the camera on the drone.

[0022] According to the tenth aspect of the present invention, an altimeter on a drone is used to obtain the altitude information of the drone, while an inertial measurement unit and a light detection and ranging sensor can be used to estimate its posture when position and direction information cannot be obtained from lighting devices, markers or projected markers.

[0023] According to the eleventh aspect of the present invention, the infrared projector on the UAV is used to project light onto the lighting device to trigger its motion sensor and determine the working status of the lighting device based on the change in brightness level. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Passive markers, marker projectors, imaging panels, and cameras are shown;

[0025] Figure 2 An imaging panel and a camera are shown;

[0026] Figure 3 shows the image processing process of a QR code image projected onto a panel;

[0027] Figure 4 The pose of the projected QR code relative to the camera coordinate system is shown;

[0028] Figure 5 shows the projection of a marker array onto the ground;

[0029] Figure 6 The structure of a drone equipped with a positioning system is shown;

[0030] Figure 7 A fleet of drones used for material delivery is shown;

[0031] Figure 8 A mobile forklift for material handling is shown;

[0032] Fig. 9 A marking configuration is shown;

[0033] Fig.10 A type of sub-tag construction with large and small QR codes is shown;

[0034] Fig.11 A type of sub-mark construction with a medium and small QR code is shown;

[0035] Fig.12 A type of sub-marker configuration with solid and hollow circles is shown;

[0036] Fig.13 A type of sub-marker construction with solid and hollow squares is shown;

[0037] Fig.14 shows the coordinate system for positioning components in the system;

[0038] Fig.15 The concept of QR code projection is shown;

[0039] Fig.16 shows the relationship between the ceiling and the projected marker in the X direction;

[0040] Fig.17 The relationship between the ceiling projected markers in the Y direction is shown;

[0041] Fig.18 The construction of a pallet with markings is shown;

[0042] Fig.19 shows pallet lifting using an autonomous mobile forklift;

[0043] Fig. 20 A pallet with directional RFID tags is shown;

[0044] Fig.21 A cabinet with a directional RFID tag is shown;

[0045] Fig. 22 A robot is shown following a person carrying a passive or projected marker;

[0046] Fig.23 The drone is shown navigating through a building using a marker as a reference;

[0047] Fig.24 The drone is shown navigating with projected markers;

[0048] Fig.25 The UAV is shown projecting infrared light to trigger the motion sensor of the lighting fixture. DETAILED DESCRIPTION

[0049] like Figure 1As shown, the projector 100 projects the marker onto an imaging panel 102 located within the focal length of the camera 101. The imaging panel 102 is composed of an optical filter material in its central area and a diffusion material in the rest of the area. The camera 101 can capture a passive marker image 103 through a filter 104, or a marker image projected onto the diffusion area of ​​the imaging panel 102.

[0050] The tag can be in the form of a QR code or other image pattern that can be recognized and processed by the MCU.

[0051] like Figure 2 As shown, one or more images of the marker may be projected onto panel 102 and the camera 101 captures the image and transmits it to the MCU for processing.

[0052] Figure 3 The MCU image processing process of a QR code (two-dimensional code) projected on an imaging panel is shown. The image captured on the panel is first converted to a black and white image, and then three squares at the edge of the image are identified. Based on the identified three squares, the local coordinate system O q Assigned to the QR code, and the coordinates of the four corners D1, D2, D3, and D4 can be obtained. The content of the QR code is unique and represents its relative to the marking coordinate system O m relative position.

[0053] like Figure 4 As shown, based on Figure 3 The obtained QR code coordinates can be obtained from the image relative to the camera coordinate system O c Once fixed, the global pose of all markers will be known. With this information, the global pose of the camera in the XY plane can be obtained.

[0054] Figure 5 Shown is the situation when an array of markers is projected onto the ground for positioning.

[0055] like Figure 6 As shown, the imaging panel 102 and the camera 101 can be installed on a drone. Figure 5 When the array of projected markers is shown, one or more marker images will be projected onto the panel. Figure 3 and 4 The process shown can be used to obtain the global attitude of the UAV for navigation purposes. The altimeter on the UAV can be used to obtain its height information relative to the ground. Using the height information and the global attitude in the XY plane, the global attitude of the UAV in 3D space can be determined. The inertial measurement unit and the mobile robot's odometry, which can be in the form of a visual odometry, can be used to estimate its attitude during periods when there is no information from the markers.

[0056] Based on the above positioning system, drone swarms can be deployed indoors to achieve fast point-to-point material delivery, such as Figure 7 shown.

[0057] like Figure 8 As shown, the imaging panel, camera, and MCU can also be placed on a UGV such as an autonomous forklift for positioning, navigation, and precise manipulation.

[0058] Fig. 9 A tag type is shown having large, medium and small QR codes, plus filled and hollow squares and circles.

[0059] Fig.10 Shows the formation Fig. 9 A sub-marker type of a part of the mark shown in . A small QR code is nested inside the large QR code, and four small QR codes are surrounded near the four corners of the large QR code. In the horizontal and vertical directions, solid or hollow circles and squares are placed between the four small QR codes.

[0060] Fig.11 Shows the formation Fig. 9 A sub-mark type of a portion of the mark shown. Four medium QR codes are arranged in an array of two rows and two columns, and are also surrounded by four small QR codes near the four corners of the array. Solid or hollow circles and squares are placed between the four small QR codes in the horizontal and vertical directions.

[0061] Fig.12 A sub-mark type having eight solid or hollow circles is shown, with solid circles representing "1" and hollow circles representing "0". The combination of solid and hollow circles represents the position of the sub-mark in the X direction of the mark coordinate system. For example, counting from left to right, seven hollow circles followed by one solid circle represent binary "00000001", indicating its unique position.

[0062] Fig.13 Another sub-mark type is shown with eight solid or hollow squares, with solid squares representing "1" and hollow squares representing "0". The combination of solid and hollow squares represents the position of the sub-mark in the Y direction of the mark coordinate system. For example, counting from bottom to top, six hollow squares followed by two solid squares represent binary "00000011", indicating its unique position.

[0063] Fig.14 The world, robot, camera, panel and marker coordinate systems are shown. w , O r , O c , O p , and O mAll coordinate systems are right-handed, with the z-axis oriented by aligning the index finger of the right hand along the positive x-axis and flexing the palm toward the positive y-axis.

[0064] The world coordinate system O w Fixed in one position, the robot coordinate system O r Fixed on the mobile robot, the camera coordinate system O c The origin of the panel coordinate system is placed at the center of its focusing lens and connected to the mobile robot. p Placed in the top center of the imaging panel and also connected to the mobile robot, mark the coordinate system O m It can be on the ceiling.

[0065] The object is relative to the reference frame O f The posture can be expressed by the homogeneous transformation matrix as

[0066]

[0067] where the 4x4 matrix T f,o The upper left 3x3 submatrix represents the object relative to the reference coordinate system O f The upper right 3x1 vector represents the position of the object relative to the same coordinate system.

[0068] The isomorphic transformation matrix of the marker coordinate system relative to the world coordinate system, each sub-marker posture relative to the marker coordinate system, the robot coordinate system relative to the world coordinate system, the camera coordinate system relative to the robot coordinate system, the panel coordinate system relative to the camera coordinate system, and each sub-marker posture relative to the panel coordinate system are represented as T w,m 、T m,q 、T w,r 、T r,c 、T c,p In this way, the following equation holds

[0069] T w,m T m,q =T w,r T r,c T c,p T p,q

[0070] The purpose of the positioning system is to obtain the mobile robot relative to the world coordinate system T w,r The posture is expressed as

[0071]

[0072] Once fixed to the ceiling, the pose of each marker relative to the world coordinate system and the pose of each sub-marker relative to the marker coordinate system are known and expressed as

[0073]

[0074]

[0075] The camera is installed on the mobile robot, and its posture relative to the robot coordinate system is known, expressed as

[0076]

[0077] Fig.15 The relationship between the QR code mark on the ceiling and its corresponding projected mark on the imaging panel is shown. The projected QR code image is captured by the camera and then processed by the MCU to obtain the coordinates of its four corners (in pixels). The four corners are represented as D1, D2, D3, and D4. For example, if the positions of D1 and D2 in the camera coordinate system are represented as [x c,D1 y c,D1 0] T and [x c,D2 y c,D2 0] T , then their positions on the imaging panel relative to the camera coordinate system can be obtained as

[0078] P c,D1 =[-x c,D1 d c,p / f c -y c,D1 d c,p / f c d c,p ] T

[0079] P c,D2 =[-x c,D2 d c , / f c -y c,D2 d c,p / f c d c,p ] T

[0080] The focal length of the camera is represented by f c , the distance between the top surface of the panel and the focusing lens is denoted as d c,p . It is assumed that the panel and robot coordinate systems have the same orientation.

[0081] At this point, the robot's position relative to the world coordinate system can be expressed as

[0082]

[0083] It can be obtained from the robot's roll, pitch and yaw angles, which are measured by the onboard gyroscope.

[0084] Assume that the camera coordinate system and the robot coordinate system have the same orientation, and the origin of the camera coordinate system is just above the origin of the robot coordinate system. p,q The QR code on the ceiling is shown in the figure below. First, place the QR code on the ceiling. w,r The reciprocal of P c,D1 By multiplying, the position vector of D1 can be converted to the coordinate system of the ceiling mark O m Coordinate systems with the same orientation O p′ , and expressed in the camera coordinate system as

[0085]

[0086] Fig.16 and Fig.17 The marking coordinate system O is shown m The ceiling QR code and its coordinate system O p′ The relationship between the projected QR codes in P c,D1(p / / m) , we can get the coordinate system O p′ The position of the ceiling QR code corner d1 is represented as follows:

[0087]

[0088] Likewise, we can get

[0089]

[0090] Where (x qd1 ,y qd1 ) and (x qd2 ,y qd2 ) are the coordinates of the ceiling QR code corners d1 and d2, (x ml ,y ml ) is the coordinate of the projector light source in the marker coordinate system, (x qD1 ,y qD1 ) is in the coordinate system O p′ The coordinates of the ceiling QR code indicated in d ql is the vertical distance between the light source and the marking plane, d pl is the marking coordinate system O m and O p′ The vertical distance between.

[0091] Based on P p′,D1 and P p′,D2 , we can get the ceiling QR code relative to the coordinate system O p′ The angle θ of the X axis (d1,d2), so that the ceiling QR code is relative to the coordinate system O p′ The homogeneous transformation matrix can be expressed as

[0092]

[0093] The ceiling QR code is relative to the coordinate system O p The homogeneous transformation matrix can be expressed as

[0094]

[0095] Following a similar derivation sequence, we can obtain Figure 10 to Figure 13 The sub-marks shown are for coordinate system O p posture.

[0096] When one or more sub-marker poses are obtained, the pose of the robot relative to the world coordinate system can be obtained as follows:

[0097]

[0098] If you have eight solid or hollow circles Fig.12 The sub-marker shown in is projected onto the imaging panel, and its posture T can also be obtained according to the above similar derivation process. p,q , and the robot can be obtained relative to the world coordinate system T w,r If only a part of the eight solid or hollow circles of the sub-marker is projected onto the imaging panel, although the sub-marker T p,q Although the robot has a precise pose, its orientation can still be obtained, so the robot's orientation relative to the world coordinate system can be obtained.

[0099] Similarly, for Fig.13 The sub-tags shown in , can obtain the robot relative to the world coordinate system T w,r attitude or orientation.

[0100] Fig.18 The construction of a tray is shown with passive or projected markers placed on the legs of the tray. Fig.19 As shown, the postures T of the left, middle and right marks on the pallet relative to the mobile forklift coordinate system can be obtained. r,ql 、T r,qm and T r,qr With the distance between the left edge of the moving forklift and the left mark d1 and the distance between the right edge of the moving forklift and the right mark d2, the forklift can use these parameters to correctly align the pallet and perform precise pallet lifting tasks.

[0101] Fig. 20A pallet is shown with directional RFID tags installed, where one RFID tag restricts the tag read area only on the horizontal plane, and three RFID tags restrict the tag read area only on the left, middle, and right vertical planes.

[0102] An automatic forklift equipped with an RFID reader will understand the approximate posture of the pallet based on the reading of the RFID tag on the pallet. Combined with the passive or projected marker on the pallet, the forklift will be able to first identify the approximate posture of the pallet and then perform precise pallet lifting tasks.

[0103] Fig.21 A cabinet with directional RFID tags installed is shown, where one RFID tag restricts the tag read area only on the horizontal plane, and three RFID tags restrict the tag read area only on the left, middle, and right vertical planes. An automated forklift equipped with an RFID reader will understand the approximate posture of the cabinet based on the readings of the RFID tags on the cabinet. When combined with passive or projected markers on the cabinet or the pallet below, the forklift will be able to first identify the approximate posture of the cabinet and then perform precise cabinet lifting tasks.

[0104] Fig. 22 The concept is shown when a person wears a passive marker or a projected marker, and a robot with a positioning system calculates the pose relative to the marker and follows the person in front of it.

[0105] Fig.23 A drone is shown navigating in a building with lighting fixtures, passive markers, and marker projectors as reference positions.

[0106] Fig.24 It shows the situation when the UAV acquires the pose relative to the projector marker based on the projected marker on the UAV imaging panel and navigates in the building to perform inspection and surveillance tasks.

[0107] Fig.25 The figure shows a situation when a drone carrying an infrared projector shines infrared light to trigger the motion sensor of a lighting device. The onboard camera can be used to detect whether the brightness level of the lighting device changes after the trigger. If the brightness level is adjusted up, it means that the lighting device is working well, otherwise, the faulty lighting device needs to be replaced. The drone can detect and record the working condition to facilitate building lighting maintenance.

[0108] Using lighting fixtures, passive markers, and projected markers as position references, drones can also perform building surveillance in conjunction with onboard inertial measurement unit light detection and ranging sensors and altimeters.

Claims

1. A precise positioning system, comprising: An imaging panel, the central area of ​​which is made of optical filter material for removing unwanted spectra, and the remaining area is made of diffusion material for projecting the formation of the artificial marker image; A camera that captures either a passive artificial mark image through a filter of an imaging panel or a projected artificial mark image formed on an imaging panel; the artificial mark image includes: A sub-mark that combines large, medium, and small QR codes with solid, hollow circles and squares surrounding the QR codes; The small QR code is nested inside or outside the large QR code; In the horizontal and vertical directions, groups of solid or hollow circles and squares are placed around the QR code; The solid or hollow circles are arranged in groups, where the solid circle represents "1" and the hollow circle represents "0", and the combination of the solid and hollow circles in the group represents the position of the sub-mark in the horizontal direction. Counting from left to right, seven hollow circles followed by a solid circle represent binary "00000001", indicating the horizontal position of the sub-mark; The solid or hollow squares are arranged in groups, the solid square represents "1", the hollow square represents "0", and the combination of the solid and hollow squares in the group represents the position of the sub-mark in the vertical direction. From bottom to top, six hollow squares followed by two solid squares represent binary "00000011", indicating the vertical position of the sub-mark; An inertial measurement unit, which is used to measure the attitude of the system; an altimeter, which is used to measure the system altitude; and The MCU obtains the global attitude of the system relative to the world coordinate system.

2. The system according to claim 1, characterized in that: If the camera captures a QR code type sub-tag of a fixed position passive tag through the imaging panel's filter, then: The posture of the sub-marker relative to the camera coordinate system can be obtained, and the global posture of the positioning system relative to the world coordinate system can be obtained by using the known relationships between the positioning system and the camera coordinate system, the sub-marker and the marker coordinate system, and the marker and the world coordinate system; Knowing the sub-marker size, the obtained sub-marker image size in pixels, and the focal length of the camera, the distance between the positioning system and the passive marker will be obtained.

3. The system according to claim 1, characterized in that: If the camera captures a circular sub-marker of a passive marker at a fixed position through the imaging panel's filter, then: The posture of the sub-marker relative to the camera coordinate system can be obtained, and the global posture of the positioning system relative to the world coordinate system can be obtained by using the known relationship between the positioning system and the camera coordinate system, the sub-marker and the marker coordinate system, and the marker and the world coordinate system; The combination of solid and hollow circles in the group represents the position of the sub-marker in the X direction of the mark coordinate system. Counting from left to right, seven hollow circles followed by a solid circle in the group represent binary "00000001", indicating its unique position; Given the sub-marker size, the obtained sub-marker image size in pixels, and the focal length of the camera, the distance between the positioning system and the passive marker will be obtained; If only a portion of the solid or hollow circle of a sub-marker in a group is captured, although the precise pose of the sub-marker cannot be obtained, its orientation can still be obtained, and thus the global orientation of the positioning system relative to the world coordinate system can be obtained.

4. The system according to claim 1, characterized in that: If the camera captures a square sub-marker of a fixed-position passive marker through the imaging panel's filter, then: The posture of the sub-marker relative to the camera coordinate system can be obtained, and the global posture of the positioning system relative to the world coordinate system can be obtained by using the known relationships between the positioning system and the camera coordinate system, the sub-marker and the marker coordinate system, and the marker and the world coordinate system; The combination of solid and hollow squares in the group represents the position of the sub-mark in the Y direction of the mark coordinate system. Counting from bottom to top, six hollow squares followed by two solid squares represent binary "00000011", indicating its unique position; Given the sub-marker size, the obtained sub-marker image size in pixels, and the focal length of the camera, the distance between the positioning system and the passive marker will be obtained; If only a portion of the solid or hollow square of a sub-marker in the group is captured, although the exact pose of the sub-marker cannot be obtained, its orientation can still be obtained, and therefore the global orientation of the positioning system relative to the world coordinate system can be obtained.

5. The system according to claim 1, characterized in that: If a sub-mark of the QR code type is projected onto the diffuse area of ​​the imaging panel and captured by the camera, then: The projected posture of the sub-marker relative to the camera coordinate system can be obtained, and the global posture of the positioning system relative to the world coordinate system can be obtained by using the known relationship between the positioning system and the camera coordinate system, the projected sub-marker and the marker coordinate system, and the marker and the world coordinate system; Knowing the sub-marker size, the obtained sub-marker image size in pixels, and the focal length of the camera, the distance between the positioning system and the passive marker will be obtained.

6. The system according to claim 1, characterized in that: If a circular sub-marker is projected onto the diffuse area of ​​the imaging panel and captured by the camera, then: The projected posture of the sub-marker relative to the camera coordinate system can be obtained, and the global posture of the positioning system relative to the world coordinate system can be obtained by using the known relationship between the positioning system and the camera coordinate system, the projected sub-marker and the marker coordinate system, and the marker and the world coordinate system; The combination of solid and hollow circles in the group represents the position of the sub-marker in the X direction of the mark coordinate system. Counting from left to right, seven hollow circles followed by a solid circle represent binary "00000001", indicating its unique position; Given the sub-marker size, the obtained sub-marker image size in pixels, and the focal length of the camera, the distance between the positioning system and the passive marker will be obtained; If only a portion of the solid or hollow circle of a sub-marker in the group is captured, although the precise pose of the sub-marker cannot be obtained, its orientation can still be obtained, and thus the global orientation of the positioning system relative to the world coordinate system can be obtained.

7. The system according to claim 1, characterized in that: If a square sub-marker is projected onto the diffuse area of ​​the imaging panel and captured by the camera, then: The projected posture of the sub-marker relative to the camera coordinate system can be obtained, and the global posture of the positioning system relative to the world coordinate system can be obtained by using the known relationship between the positioning system and the camera coordinate system, the projected sub-marker and the marker coordinate system, and the marker and the world coordinate system; The combination of solid and hollow squares in the group represents the position of the sub-mark in the Y direction of the mark coordinate system. Counting from bottom to top, six hollow squares followed by two solid squares represent binary "00000011", indicating its unique position; Given the sub-marker size, the obtained sub-marker image size in pixels, and the focal length of the camera, the distance between the positioning system and the passive marker will be obtained; If only a portion of the solid or hollow square of a sub-marker in the group is captured, although the exact pose of the sub-marker cannot be obtained, its orientation can still be obtained, and therefore the global orientation of the positioning system relative to the world coordinate system can be obtained.

8. The system according to claim 1, characterized in that: Combining sub-markers with large, medium, and small QR codes and groups of solid and hollow circles and squares will increase the chances of getting a marking gesture in a variety of situations; At a certain distance, only a part of the large QR code is captured and cannot be processed by the MCU, but at this time, the complete image of the medium or small QR code can still be captured to obtain its posture; At another distance, the small QR code image is not clear, but the large or medium QR code image is clear enough to be used to obtain its pose; In some cases, no QR code image is clear enough to obtain its pose, but the solid or hollow circle or square image is still clear enough to obtain its pose, or if only a portion of the solid or hollow image is captured, at least its orientation can be obtained; Given the sub-marker size, the obtained sub-marker image size in pixels, and the focal length of the camera, the distance between the positioning system and the passive marker will be obtained; If one or more sub-marker poses are obtained simultaneously, the combination of pose information will obtain more reliable pose information relative to the world coordinate system.

9. The system according to any one of claims 1 to 8, characterized in that: The altimeter on the mobile robot can be used to obtain its altitude information, and when no information can be obtained from markers or projected markers, the inertial measurement unit and the odometer of the mobile robot can estimate the position of the system through dead reckoning or other algorithms.

10. The system according to any one of claims 1 to 8, characterized in that: Passive markers or projected markers are placed on a pallet or cabinet, and the positioning system on the autonomous forklift calculates its own posture and distance relative to the pallet or cabinet based on the passive marker or projected marker image captured on the imaging panel, and then performs precise pallet or cabinet lifting tasks.

11. The system according to any one of claims 1 to 8, characterized in that: Place directional RFID tags, passive tags, or projected tags on pallets or cabinets: One RFID tag limits the tag reading area only to the horizontal plane, while three RFID tags limit the tag reading area only to the left, middle, and right vertical planes, and an automatic forklift equipped with an RFID reader / writer will obtain the approximate posture and distance of the pallet or cabinet based on the reading of the RFID tag on the pallet; Based on directional RFID tags, once the forklift knows its approximate posture and distance relative to the pallet or cabinet, it will calculate its more precise posture and distance relative to the pallet or cabinet based on the captured passive marker image or the projected marker image on the imaging panel, and then perform precise pallet or cabinet lifting tasks.

12. The system according to any one of claims 1 to 8, characterized in that: By mounting passive markers or projected markers on the top of the rack, the mobile robot obtains its pose and distance relative to the rack and then performs precise material loading and unloading tasks.

13. The system according to any one of claims 1 to 8, characterized in that: Passive markers or projected markers are installed on people or mobile devices, and the mobile robot behind obtains its posture and distance relative to the person or mobile device based on the captured passive or projected marker images, and then performs subsequent tasks.

14. The system according to any one of claims 1 to 8, characterized in that: Passive markers or projected markers are mounted on the ceiling of a building, and a drone with a positioning system obtains its pose with respect to the world coordinate system to navigate in the building to perform inspection and surveillance tasks.

15. The system according to claim 14, characterized in that: An altimeter on the drone is used to obtain its altitude information, while inertial measurement units, visual odometry, and light detection and ranging sensors can be used to estimate its attitude when reference position and orientation information from lighting fixtures, markers, or projected markers is not available.

16. The system according to claim 14, characterized in that: A drone equipped with an infrared projector irradiates infrared light to trigger the motion sensor of the lighting device, and the onboard camera can be used to detect whether the brightness level of the lighting device changes after being triggered. If the brightness level is increased, it indicates that the lighting device is working; if it is not working, the faulty lighting device needs to be replaced, which helps with building lighting maintenance work.

17. The system according to claim 1, characterized in that: Passive tags and directional RFID tags are placed on the ground at known fixed locations. The RFID reader on the mobile robot reads the RFID tags to obtain their approximate global pose. Then the camera on the mobile robot obtains a more accurate global pose based on the captured passive tag images, thereby navigating from one place to another.

Images