A dot matrix positioning system for load-bearing robots based on vision technology

Abstract

The present application relates to the field of robot positioning technology, and discloses a load robot dot matrix positioning system based on visual technology, including a feature acquisition module, an image processing module, a similarity matching module, and an information output module. After the similarity matching module, a similarity determination module is embedded. If the similarity S is less than the rated value, the error correction module is called to correct the coordinate error and then output the coordinate information; otherwise, the coordinate information is output. By embedding the error correction module after the similarity matching module, the real-time true height value is obtained after the robot moves forward for a debugging distance, and the last calculated height is compared to accurately obtain the real-time error amount, and then error correction is performed, the accumulated error can be effectively and accurately corrected at the algorithm level, saving the cost of debugging equipment and improving the stability of the robot's long-term operation.

Description

Technical Field

[0001] The present application relates to the field of robot positioning technology, and in particular to a load robot dot matrix positioning system based on vision technology. Background Art

[0002] Load-carrying robots are often used for handling tasks in factory workshops or logistics transfer stations. The most important technology of these robots is how to achieve self-positioning and posture adjustment in unmanned workshops, so that they can automatically identify the route and complete the handling.

[0003] At present, the main methods of robot positioning include beacon-based positioning, environmental map model matching positioning, vision-based monocular or binocular positioning, environmental feature point image matching positioning, etc. The robots produced and designed by various manufacturers have both single-function positioning and joint positioning.

[0004] Among them, beacon positioning is a feature point beacon that contains location information. The robot obtains the location information of the beacon to determine its own location information. For example, a common beacon positioning is to use QR codes (such as QR codes) on the ground, and the QR codes and coordinates are bound to establish a dot map. The robot obtains location information by scanning the code, and then obtains its own location information through algorithms to achieve positioning.

[0005] Monocular vision positioning uses a single camera to take pictures, and matches the characteristic photos taken at the specified point with the pre-established location information map to confirm the robot's coordinates.

[0006] In the above positioning method, QR code beacon positioning requires long-term maintenance of the QR code posted on the ground. However, due to the lack of depth information (height) of the captured two-dimensional plane coordinates, monocular vision positioning makes it difficult for the robot to accurately locate the three-dimensional space coordinates, and due to the similarity error of image matching, cumulative errors will be generated, and the robot needs to be corrected regularly, otherwise it will deviate too much from the original path. Therefore, the present invention aims to realize how to combine monocular vision positioning with beacon positioning to further improve the positioning accuracy of the robot and avoid long-term maintenance. Summary of the invention

[0007] The present application proposes a load robot dot matrix positioning system based on vision technology, which has the advantages of saving maintenance workload at feature points and correcting accumulated errors based on the algorithm level, so as to solve the technical problems raised by the above-mentioned background technology.

[0008] To achieve the above-mentioned purpose, the present application adopts the following technical solution: a load-bearing robot based on vision technology, including a robot body and a camera and a reflective tape fixedly installed on the bottom edge of the side of the robot body. The reflective tape is laid on the indoor ceiling. The camera is used to capture the ceiling reflective tape laying information when the robot body is located. The reflective tape feature image corresponding to each fixed point corresponds to the position coordinate information of the robot.

[0009] The camera rotates 90 degrees on the spot to take pictures of the ceiling reflective stickers in four specific directions, and the ceiling reflective sticker pictures in each direction correspond to the position and running direction information of the robot body.

[0010] A load robot dot matrix positioning system based on vision technology includes a feature acquisition module, an image processing module, a similarity matching module and an information output module.

[0011] The feature collection module is used to collect feature images.

[0012] The image processing module is used to grayscale the acquired feature image, extract feature points whose brightness is obviously higher than the surrounding ones, and combine the extracted feature points into a feature image.

[0013] The similarity matching module is used to match the feature image with the image stored at the specified point, complete the matching after translation and rotation, and obtain the corresponding coordinate information of the match.

[0014] The information output module is used to output the acquired coordinate information.

[0015] Furthermore, after the similarity matching module, a similarity determination module is embedded. If the similarity S is less than the rated value, the error correction module is called to correct the coordinate error and then output the coordinate information; otherwise, the coordinate information is output.

[0016] Furthermore, the error correction module corrects the coordinate error including the following steps:

[0017] S1, obtain the feature point image coordinate u, which is the horizontal coordinate of the feature point randomly obtained in the feature image in the similarity matching module;

[0018] S2, obtain the world coordinate x, which is the world coordinate corresponding to the image coordinate u;

[0019] S3, locking the focal length f of the camera used to take the last feature image;

[0020] S4, driving the robot forward a debugging distance Δx, and taking correction pictures;

[0021] S5, obtaining the pixel coordinates du of the same feature point in the correction image and the feature image;

[0022] S6, calculating the deviation rate η between the actual height H and the calculated height h;

[0023] S7, determine the deviation rate η < rated value, if yes, execute step S8; if no, assign the real height H to the calculated height h, and recalibrate the corresponding relationship value between the world coordinate system and the image coordinate system according to h / f;

[0024] S8. Output coordinates.

[0025] Furthermore, the camera is arranged at the bottom edge of the robot body to shoot upward, and one side of the camera imaging is always bounded by the top edge of the robot body.

[0026] Furthermore, the sizes of the several reflective stickers on the roof are different.

[0027] The present invention has the following beneficial effects:

[0028] The present application provides a load robot dot matrix positioning system based on vision technology, which replaces the traditional QR code laid on the ground as a feature point with reflective stickers laid on the indoor ceiling, and uses a picture of a combination of reflective stickers to form the feature coordinates of the specified position, so as to identify the position coordinate information, thereby effectively reducing the maintenance workload of the feature points.

[0029] The present application uses similarity matching to call up error correction to correct the accumulated errors of the robot, and obtains the real-time true height value after the robot moves forward a debugging distance, and compares the last calculated height to accurately obtain the real-time error amount, and then performs error correction. The accumulated errors can be corrected effectively and accurately at the algorithm level, saving the cost of debugging equipment and improving the long-term stability of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings, which constitute a part of the specification, illustrate the embodiments disclosed in the present application and, together with the description, serve to explain the principles disclosed in the present application.

[0031] The present application can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0032] Figure 1 This is a schematic diagram of the installation position of the robot camera of the present invention;

[0033] Figure 2 is a system block diagram of the present invention;

[0034] Figure 3 This is a schematic diagram of debugging imaging in Embodiment 2 of the present invention;

[0035] Figure 4 This is a block diagram of the error correction module in the second embodiment of the present invention.

[0036] In the figure: 1. Robot body; 2. Camera. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0038] Embodiment 1

[0039] See also Figure 1 A load-bearing robot based on vision technology includes a robot body 1, a camera 2 fixedly installed on the bottom edge of the side of the robot body 1, and a reflective sticker. The reflective sticker is laid on the indoor ceiling and is randomly and irregularly pasted. The camera 2 is used to shoot the laying information of the ceiling reflective sticker when the robot body 1 is located. The characteristic picture of the reflective sticker corresponding to each fixed point corresponds to the position information and action instruction information of the corresponding robot.

[0040] The reflective coefficient of the reflective tape is much greater than that of ordinary materials. Under the illumination of the robot's own light source, the brightness of the reflective tape in the pattern is much greater than that of other materials on the roof. The pattern is processed with a threshold value that is greater than the background and less than the light intensity of the reflective tape, and the resulting pattern is the characteristic image of the reflective tape. The robot takes pictures at known coordinate positions, and uses a table in the robot storage unit to record the binding of the position coordinates and the characteristic image of the reflective tape. The specific image represents the specific position of the robot. The robot continuously takes pictures of the roof while moving, and the continuous and different specific images formed are recorded in the robot storage unit, which can be used as a comparison pattern of the walking route. After the camera 2 takes these pictures at different positions, it can determine its own position. The camera 2 moves to form a series of pictures, which also determines the path of the robot's walking.

[0041] The traditional QR code dot matrix feature recording method laid on the ground will be cancelled, and replaced by reflective stickers laid on the ceiling of the house. The captured picture contains multiple reflective stickers. After image feature extraction and processing, a feature picture is formed. The feature picture of each fixed position represents a position coordinate. Compared with the traditional visual positioning of the robot, this method retains the same map information system, recognition system and scheduling drive system, and only converts the feature recognition method. The coordinate information obtained by scanning the code at a specific location is converted into image feature matching and then the coordinate information is obtained by looking up the table. In this way, the maintenance workload of the QR code can be reduced. At the same time, the feature picture taken at the specified location is matched with the reference in the storage unit after a small amount of translation and rotation. According to the amount of translation and rotation, the robot's own posture can be adjusted to complete the adaptation, thereby improving the positioning accuracy of the robot.

[0042] In order to further improve the recognition accuracy of robot positioning and expand the information contained in the feature image, camera 2 rotates 90 degrees on the spot to shoot ceiling reflective sticker images in four specific directions. The ceiling reflective sticker image in each direction corresponds to the plane position coordinate information and running direction information of the robot body 1. In this way, when the robot moves to any one of the four directions of any specific position, the reflective sticker image obtained by taking a photo is compared with the feature image of the same direction at that position, and the position coordinates and vehicle direction bound to the reference image can be found.

[0043] A load robot dot matrix positioning system based on vision technology includes a feature acquisition module, an image processing module, a similarity matching module and an information output module.

[0044] The feature acquisition module is used to collect feature images, such as RFID radio frequency identification coordinate information, camera scanning and recognition of QR code coordinate information, camera acquisition of images containing feature points, etc.

[0045] The image processing module is used to grayscale the acquired feature image, extract feature points whose brightness is significantly higher than the surrounding ones (such as using the nearest neighbor matching algorithm to obtain the edges of feature points), and combine the extracted feature points into a feature image.

[0046] The similarity matching module is used to match the feature image with the image stored at the specified point, complete the matching after translation and rotation, and obtain the coordinate information corresponding to the match. The functions of this module can be realized by any algorithm that can complete image matching, such as grayscale matching to calculate the square sum of errors or normalized product to determine similarity.

[0047] The information output module is used to output the acquired coordinate information so that the robot can issue action instructions.

[0048] The feature acquisition module sends the collected reflective sticker highlight feature image to the image processing module. After the image is grayed out by the image processing module, it is sent to the similarity matching module for library image matching. After obtaining the coordinate information, it is sent to the information output module.

[0049] Because similarity matching uses the threshold method to terminate the matching cycle, multiple accumulated similarity matches will cause the robot's position and path errors to become larger and larger. Therefore, it is necessary to further add error correction to the robot's positioning system.

[0050] After the similarity matching module, a similarity judgment is embedded. If the similarity S is less than the rated value, the error correction module is called to correct the coordinate error and then output the coordinate information; otherwise, the coordinate information is output.

[0051] Embodiment 2

[0052] See also Figure 3 and 4 To facilitate understanding, we first explain the conversion of coordinate systems. Monocular camera calibration is actually the conversion of coordinate systems. The user-defined three-dimensional world coordinate system is expressed as a description of the position of the target object in the real world. The camera coordinate system is a coordinate system established on the camera. The world coordinate system can be converted to the image coordinate system using the camera's intrinsic and extrinsic parameters. The image coordinate system is the projection relationship from the camera coordinate system to the image coordinate system. The pixel coordinate system is the coordinate of the image point on the digital image after the object is imaged, and the unit is the number of pixels.

[0053] The error correction module corrects the coordinate error by the following steps:

[0054] S1, obtain the feature point image coordinate u, which is the horizontal coordinate of the feature point randomly obtained in the feature image in the similarity matching module;

[0055] S2, obtain the world coordinate x, which is the world coordinate corresponding to the image coordinate u;

[0056] S3, locking the focal length f of the camera used to take the last feature image;

[0057] S4, driving the robot forward a debugging distance Δx, and taking correction pictures;

[0058] S5, obtaining the pixel coordinates du of the same feature point in the correction image and the feature image;

[0059] S6, calculating the deviation rate η between the actual height H and the calculated height h;

[0060] The deviation rate η is calculated using the following formula:

[0061]

[0062]

[0063]

[0064] In the formula, H represents the real height in the world coordinate system, h represents the calculated height in the world coordinate system, f represents the focal length of the camera, Δx represents the debugging distance the robot moves forward in the x direction, du represents the pixel coordinates after the robot moves forward Δx, x represents the world coordinates of the feature point obtained by taking the feature picture for the last time, and u represents the image coordinates obtained by taking the feature picture for the last time. The above calculated height is the built-in world coordinate height value after the robot accumulates errors, and the real height is the real-time world coordinate height value after the robot moves forward the debugging distance.

[0065] S7, determine the deviation rate η < rated value, if yes, execute step S8; if no, assign the real height H to the calculated height h, and recalibrate the corresponding relationship value between the world coordinate system and the image coordinate system according to h / f;

[0066] S8. Output coordinates.

[0067] By moving the robot forward along the specified horizontal coordinate direction for a debugging distance Δx, the real-time theoretical height of the feature point is obtained according to the correspondence between the pixel coordinate change value of the feature point and the real translation distance, and then the deviation rate is calculated by comparing the built-in calculated height of the robot at the last coordinate conversion with the real-time theoretical height to obtain the cumulative error of the robot. At the same time, the real theoretical value can be used as the correction standard to recalibrate the corresponding relationship between the world coordinate system and the image coordinate system, thereby achieving correction. The correction of cumulative error is completed at the algorithm level without the need to add redundant parameter detection equipment.

[0068] In order to further improve the correction accuracy, the camera 2 is set at the bottom edge of the robot body 1 to shoot upward, and one side of the camera 2 image is always bounded by the top edge of the robot body 1. Each feature image taken by the camera 2 has the top edge of the robot body 1 as the pixel reference edge, so that the pixel coordinates calibrated in this way are more accurate.

[0069] In order to further improve the accuracy of feature point acquisition and thus improve the accuracy of correction, the reflective stickers on the roof have different sizes, which makes it easier for the image processing module to extract feature points and helps to lock the same feature point in step S5, improves the locking accuracy, and simplifies the difficulty of the locking algorithm.

Claims

1. A load robot dot matrix positioning system based on vision technology, characterized in that: It includes feature acquisition module, image processing module, similarity matching module and information output module; Feature collection module, used to collect feature images; An image processing module is used to grayscale the acquired feature image, extract feature points whose brightness is obviously higher than that of the surrounding points, and combine the extracted feature points into a feature image; The similarity matching module is used to match the feature image with the image stored at the specified point, complete the matching after translation and rotation, and obtain the coordinate information corresponding to the match; An information output module, used to output the acquired coordinate information; After the similarity matching module, a similarity determination module is embedded. If the similarity S is less than the rated value, the error correction module is called to correct the coordinate error and then output the coordinate information; Otherwise, output coordinate information; The error correction module corrects the coordinate error and comprises the following steps: S1, obtain the feature point image coordinate u, which is the horizontal coordinate of the feature point randomly obtained in the feature image in the similarity matching module; S2, obtain the world coordinate x, which is the world coordinate corresponding to the image coordinate u; S3, locking the focal length f of the camera used to take the last feature image; S4, driving the robot forward a debugging distance Δx, and taking correction pictures; S5, obtaining the pixel coordinates du of the same feature point in the correction image and the feature image; S6, calculating and obtaining the deviation rate η between the actual height H and the calculated height h; S7, determine the deviation rate η < rated value, if yes, execute step S8; if no, assign the real height H to the calculated height h, and recalibrate the corresponding relationship value between the world coordinate system and the image coordinate system according to h / f; S8. Output coordinates.

2. A load robot using the load robot dot matrix positioning system based on vision technology as claimed in claim 1, characterized in that: The invention comprises a robot body (1), a camera (2) fixedly mounted on the side bottom edge of the robot body (1), and a reflective tape, wherein the reflective tape is laid on the indoor ceiling, and the camera (2) is used to photograph the reflective tape laying information of the ceiling when the robot body (1) is located, and the reflective tape feature image corresponding to each fixed point is the position coordinate information of the corresponding robot.

3. The load-carrying robot according to claim 2, characterized in that: The camera (2) rotates 90 degrees in situ to take pictures of the ceiling reflective sticker in four specific directions, and the picture of the ceiling reflective sticker in each direction corresponds to the position and running direction information of the robot body (1).

4. The load-carrying robot according to claim 2, characterized in that: The camera (2) is arranged at the bottom edge of the robot body (1) to shoot images upward, and one side of the camera (2) imaging is always bounded by the top edge of the robot body (1).

5. The load-carrying robot according to claim 3, characterized in that: The several reflective stickers on the roof are of different sizes.

Images