Visual positioning control method based on label pattern

Abstract

The application discloses a visual positioning control method based on a label pattern, which comprises the following steps: step A, a robot pre-processes an image collected by a camera thereof, and searches for graphical attributes and vertices of a label pattern in the pre-processed image; step B, based on the graphical attributes and the vertices of the label pattern, the monocular ranging principle is used to calculate the distance between the label pattern and the camera and the deflection angle of the label pattern relative to the camera; step C, whether the distance between the label pattern and the camera and the deflection angle of the label pattern relative to the camera both satisfy a preset positioning condition is judged, if yes, it is determined that the visual positioning of a device to be positioned is completed, otherwise, step D is executed; and step D, according to the distance between the label pattern and the camera and the deflection angle of the label pattern relative to the camera, the robot moves to a predicted position point, then an image of the label pattern is collected by using the camera, and then steps A to C are executed.

Description

Technical Field

[0001] This application relates to the technical field of monocular visual pose measurement, and in particular to a visual positioning control method based on label patterns. Background Technology

[0002] Visual ranging is a crucial technology in robotics, with wide applications in visual localization, target tracking, and visual obstacle avoidance. Common visual ranging methods include monocular ranging, which boasts a simple structure and fast computation speed, thus offering promising application prospects.

[0003] Existing monocular ranging technologies generally simplify the monocular vision system into a camera projection model, establish a ranging model through geometric derivation, obtain the transformation relationship between image coordinates and the world coordinate system, and finally calculate the obstacle distance through geometric calculations (including the pinhole imaging model and the geometric proportions of similar triangles). However, this approach requires manual alignment of the camera's optical axis, and since the focal length of the monocular camera is fixed, image blurring is prone to occur when the camera captures object images. This necessitates time-consuming visual positioning corrections based on various environmental factors, affecting the robot's normal navigation operations, including returning to a specific charging location for charging. It also limits the improvement of the camera's ranging accuracy for charging docks or other components requiring visual positioning. Summary of the Invention

[0004] This application proposes a visual positioning control method based on label patterns, and the specific technical solution is as follows:

[0005] A visual positioning control method based on label patterns is proposed, comprising: Step A, where the robot preprocesses the image captured by its camera, and then searches for the vertices and graphic attributes of the label pattern within the preprocessed image, wherein the label pattern is set on the surface of the device to be positioned; Step B, based on the graphic attributes and vertices of the label pattern, calculating the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera using the monocular ranging principle; Step C, determining whether the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera, both meet preset positioning conditions. If yes, the visual positioning of the device to be positioned is completed, enabling the robot to align with or contact the corresponding label pattern in the device to be positioned; otherwise, proceeding to Step D; Step D, based on the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera, the robot moves to the predicted position point, then uses the camera to capture an image of the label pattern, and then executes steps A to C.

[0006] Compared to existing technologies, this method uses the vertices of identified label patterns to calculate the distance and angle information of the corresponding labels relative to the same camera. This allows the robot to predict the relative positional relationship between various label patterns and the same camera (which may be located in different positions), providing accurate positioning information for the robot to navigate back to the positioning device and make contact. Furthermore, it eliminates the need to spend excessive time collecting multiple frames of images at the same location. Instead, within the same frame, it uses monocular ranging to calculate the distance and angle information of multiple identified label patterns. By introducing preset positioning conditions and continuously calculating and updating the distance between the label pattern and the camera, as well as the deflection angle of the label pattern relative to the camera, the robot's movement direction is corrected to align with the orientation matched by the label pattern. This improves the visual positioning accuracy of the camera as the robot approaches the positioning device, suppresses the influence of camera ranging errors, and increases navigation accuracy while reducing the amount of real-time positioning calculations.

[0007] Further, in step D, if all the label patterns involved in the judgment in step C are positioning graphic labels, and if the distance between the positioning graphic label and the camera, or the deflection angle of the positioning graphic label relative to the camera, does not meet the preset positioning conditions, then the robot sets the predicted position point and moves to the predicted position point based on the distance between the positioning graphic label and the camera, and the deflection angle of the positioning graphic label relative to the camera. The robot updates the preset position point each time it executes step D. If, in step C, the distance between the positioning graphic label and the camera meets the preliminary positioning conditions, after executing step D, the robot begins to identify the target graphic label at the newly set predicted position point, and then configures the target graphic label as the label pattern from steps B to D. Determining that the visual positioning of the device to be positioned is completed includes completing the visual positioning of the corresponding label pattern in the device to be positioned. The label pattern is a target graphic label or a positioning graphic label. The corresponding label pattern in the device to be positioned is a target graphic label, which represents the assembly port in the device to be positioned specifically for robot contact.

[0008] Therefore, the initial positioning of the device to be positioned or its assembly port is completed, because the target graphic label represents the assembly port in the device to be positioned for the robot to contact, thereby initially suppressing the influence of ranging error caused by the camera acquiring patterns from multiple angles during the robot's movement, so as to delineate the location characteristics of the area where the target graphic label is located.

[0009] Furthermore, before identifying the target graphic label, during steps A to D, the robot sequentially traverses each identified positioning graphic label to guide the robot from the identified positioning graphic labels on both sides to the unidentified area in the middle. At the last determined predicted position point, the robot identifies the rectangular label at the center of the target graphic label and determines that the distance between the rectangular label and the camera meets the preset positioning conditions, thus confirming the existence of the target graphic label in the unidentified area in the middle, and determining that multiple positioning graphic labels are distributed on both sides of the target graphic label, which is composed of multiple rectangular labels. Moreover, the robot adjusts its pose to align with the rectangular label and determines that the deflection angle of the rectangular label relative to the camera meets the preset positioning conditions, making the last determined predicted position point the position point of the charging interface on the robot's walking plane. Here, the device to be positioned is a charging base, the target graphic label is used to represent the label of the charging interface of the charging base, and the vertical direction of its plane represents the docking direction of the charging interface. The assembly port in the device to be positioned for robot contact is the charging interface of the charging base. This technical solution guides the robot's movement towards the charging interface by using positioning graphic labels distributed on both sides of the target graphic label and multiple rectangular labels that make up the target graphic label, thereby reducing the distance between the robot and the charging interface of the charging dock.

[0010] Furthermore, the size of a positioning graphic tag on the charging dock surface is larger than the size of any rectangular tag included in the target graphic tag on the charging dock surface. This ensures that: when the robot identifies the positioning graphic tag but fails to identify the rectangular tag, and consequently fails to identify the target graphic tag, the robot is at a first predicted position point; and when the robot identifies the rectangular tag but fails to identify the positioning graphic tag, the robot is at a second predicted position point. The distance between the first predicted position point and the charging port is greater than the distance between the second predicted position point and the charging port. The positioning graphic tag does not include rectangular tags. This design corrects the pose of the target graphic tag, reduces the impact of ranging errors, and improves the accuracy of charging port position recognition. It also accelerates the robot's movement from the identified positioning graphic tags on both sides towards the unidentified image area in the center.

[0011] Furthermore, in step A, the robot identifies the positioning graphic label and / or target graphic label from multiple label patterns at once based on the graphic attributes of the label patterns, thereby obtaining the vertices and graphic attributes of the identified label patterns. Whenever multiple positioning graphic labels are identified, in step B, based on the graphic attributes and vertices of each positioning graphic label, the robot calculates the distance between each positioning graphic label and the camera, as well as the deflection angle of each positioning graphic label relative to the camera, using the monocular ranging principle. Then, the robot sequentially iterates through the distances between each identified positioning graphic label and the camera. When, in step C, the robot determines that the distances between the camera and two positioning graphic labels located on opposite sides of the central position with different placement patterns are not the two smallest values ​​among the identified distances between the positioning graphic labels and the camera, it determines that the distances between the currently identified label patterns and the camera do not meet the preset positioning conditions. This accurately eliminates unsuitable positioning locations and guides the subsequently set predicted position points to the area between the two positioning graphic labels located on opposite sides of the central position with different placement patterns.

[0012] Further, the robot executes step D to set a predicted position point in front of the two positioning graphic labels with different placement patterns that are closest to each other on both sides of the middle position. Then, the robot adjusts its pose and moves to the currently set predicted position point based on the deflection angle of the predicted position point relative to the camera, so as to reduce the distance between the robot and the target graphic label. Then, the robot repeats steps A to D until it is determined in step C that the distance between the two positioning graphic labels with different placement patterns that are closest to each other on both sides of the middle position and the camera is the two smallest distances among the distances between the identified positioning graphic labels and the camera. At this point, it is determined that the distance between the positioning graphic label and the camera meets the preliminary positioning condition, but the distance between the currently identified label patterns and the camera does not meet the preset positioning condition. After executing step D, the robot begins to identify the target graphic label in step A. Before the robot identifies the target graphic label in step A, it is not allowed that the distance between the currently identified label patterns and the camera meets the preset positioning condition. The robot continues to set new predicted position points to move towards the narrowed unknown area. At this point, the robot has not yet recognized the target graphic label or the rectangular labels contained within it, but it is closer to the target graphic label. Therefore, multiple positioning graphic labels play a role in correcting the robot's movement direction.

[0013] Further, after identifying the target graphic label in step A, the robot then identifies the individual rectangular labels that make up the target graphic label; then in step B, based on the graphic attributes and vertices of each rectangular label, it calculates the distance between each rectangular label and the camera, as well as the deflection angle of each rectangular label relative to the camera, using the monocular ranging principle; then it sequentially iterates through the distances between each identified rectangular label and the camera; when the robot determines in step C that the distance between the rectangular label at the center of the target graphic label and the camera is not the smallest among the distances between the identified rectangular labels and the camera, it determines that the distance between the currently identified target graphic label and the camera does not meet the preset positioning conditions; then in step D... In the process, the robot moves towards the rectangular label at the center of the target graphic label. The distance between the rectangular label at the center of the target graphic label and the camera is the smallest among the distances between the identified rectangular labels and the camera. The distance between the currently identified target graphic label and the camera is determined to meet the preset positioning conditions. The currently moved position is the latest predicted position. Then, at the latest predicted position, the robot's movement direction is adjusted to be parallel to the vertical direction of the rectangular label at the center of the target graphic label. The distance between the currently identified target graphic label and the camera, as well as the deflection angle of the currently identified target graphic label relative to the camera, are determined to meet the preset positioning conditions.

[0014] In summary, the robot first determines the location of the target graphic tag based on the identified positioning graphic tag, which achieves a coarse positioning effect on the charging interface. Then, by using the rectangular tags built into the target graphic tag, the robot continuously changes the position it needs to navigate to, constantly reducing the distance between the robot and the charging interface, until the latest calculated distance and angle meet the preset positioning conditions, thus completing the precise positioning of the charging interface. This improves the accuracy of the camera's positioning of the charging interface, and in turn improves the accuracy of the robot returning to the charging dock for charging.

[0015] Furthermore, the robot identifies a positioning graphic tag as consisting of a triangular tag and determines the side length of the triangular tag by its vertex. The arrangement of the positioning graphic tags on one side of the target graphic tag differs from that on the other side. Each positioning graphic tag on each side of the target graphic tag is evenly spaced along a straight line on the charging base surface. The robot identifies a target graphic tag as consisting of multiple identical rectangular tags and determines the side length of each rectangular tag by its vertex. A target graphic tag is a regular polygonal shape arranged around a rectangular tag, with different numbers of rectangular tags distributed in the neighboring areas on either side of the central position within the target graphic tag to distinguish the two sides. Highlighting the specificity of the target graphic tag allows for guiding a robot located to the left of the target graphic tag to move to the right, or to the right of the target graphic tag to move to the left, continuously bringing the robot closer to the target graphic tag. This improves the positioning accuracy of the target graphic tag.

[0016] Furthermore, the target graphic label includes three rows of rectangular labels. The row passing through the center contains three rectangular labels: one at the center of the target graphic label, parallel to the central axis of the charging dock, and the other two on either side of the center. The row passing through the center is designated the middle row. Above the middle row are two rectangular labels, arranged in the same column as the central label and the label on one side of it. Below the middle row are two rectangular labels, also arranged in the same column as the central label and the label on the other side of it. These indicate the directions directly above, below, to the left, and to the right of the target graphic label. Therefore, the number of rectangular labels distributed in the neighborhood on either side of the center of the target graphic label differs, allowing the robot to accurately distinguish the left and right sides of the target graphic label during traversal of the label pattern.

[0017] Furthermore, the method for identifying a location graphic label and / or a target graphic label from multiple label patterns at once based on the graphic attributes of the label pattern includes: when the robot searches for the graphic attributes of the label pattern and the vertices of the label pattern in the preprocessed image, the robot detects the number of edges forming a closed shape; wherein, the graphic attributes of the label pattern are the edge line features of the closed shape, and the edge line features of the closed shape include the number of edges forming the closed shape and the number of vertices of the closed shape; in the closed shape, the robot identifies the line connecting two adjacent vertices as an edge forming the closed shape; when the robot detects that the number of edges forming the closed shape is 3, the currently detected closed shape is identified as a triangle label, and the location graphic label is identified; when the robot detects that the number of edges forming the closed shape is 4, the currently detected closed shape is identified as a rectangle label; if the number of rectangle labels detected by the robot in the same frame is the total number of all rectangle labels required to form a target graphic label, and the cumulatively detected rectangle labels are symmetrically set with one of the rectangle labels as the center position, and the number of rectangle labels distributed in the neighborhood on both sides of the center position is different, then a target graphic label is identified. By combining graphic attributes to analyze the shape of the label pattern and the distribution characteristics of its internal pixels, different shapes of labels can be distinguished. Then, by combining the number of different shapes of label patterns and their distribution positions, the target graphic label and the location graphic label can be identified.

[0018] Further, in step B, the method for calculating the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera using the monocular ranging principle, includes: the robot sets the target straight line direction as the extension direction of the plane of the object to be measured on the horizontal plane, and sets the target straight line direction as parallel to the robot's walking plane; the robot sets two vertices distributed along the target straight line direction in a label pattern as two adjacent target detection points in the plane of the object to be measured; wherein, the distance Ux between two adjacent target detection points in the plane of the object to be measured is obtained in advance; the plane of the object to be measured represents the charging base surface where the label pattern is located, perpendicular to the horizontal ground or the robot's walking plane; among the two adjacent target detection points, the robot records one of the target detection points as the first detection point, and calculates the distance dx1 from the side of the first detection point to the camera using the pinhole imaging model; the robot records the other target detection point as the second detection point, and calculates the distance dx2 from the side of the second detection point to the camera using the pinhole imaging model; based on the side-angle relationship of the triangle, the deflection angle of the label pattern relative to the camera is obtained by the following formula:

[0019]

[0020] dx2*sin(bx21)=Ux*sin(ax)+dx1*sin(bx11);

[0021] Ux*cos(ax)=dx2*cos(bx21)+dx1*cos(bx11);

[0022] bx12 = 90 - bx11;

[0023] bx22 = 90 - bx21;

[0024] Wherein, the angle between the perpendicular segment from the camera to the side to be tested at the first detection point and the pinhole plane of the camera in the opposite direction of the target straight line is denoted as bx11, and the angle between the perpendicular segment from the camera to the side to be tested at the second detection point and the pinhole plane of the camera in the target straight line is denoted as bx21; the robot sets the angle between the plane of the object to be tested and the pinhole plane of the camera in the target straight line as the tilt angle of the plane of the object to be tested, where ax is the tilt angle of the plane of the object to be tested; the tilt angle of the plane where the label pattern is located is the tilt angle of the plane of the object to be tested; the deflection angle of the label pattern relative to the camera includes the angle between the perpendicular segment from the camera to the side to be tested at the first detection point and the optical axis, bx12, and the angle between the perpendicular segment from the camera to the side to be tested at the second detection point and the optical axis, bx22; the distance between the label pattern and the camera mentioned in step B includes the distance from the side to be tested at the first detection point to the camera and the distance from the side to be tested at the second detection point to the camera; the monocular ranging principle mentioned in step B includes the pinhole imaging model.

[0025] When calculating the deflection angle of the label pattern relative to the camera, this technical solution will construct Formula 1 from both vertical and horizontal perspectives for the corresponding target detection points and their sides, and then perform trigonometric calculations to ensure the comprehensiveness and representativeness of the angle calculation. Moreover, based on the tilt angle of the plane of the object under test, the influence of various label patterns on the ranging error generated by the camera can be calculated.

[0026] Further, in the label pattern, the edge that forms a certain angle with the target straight line direction is denoted as the edge to be measured of the label pattern, wherein the edge to be measured is not the edge connecting the first detection point and the second detection point; the method for calculating the distance between the label pattern and the camera using the pinhole imaging model includes: obtaining in advance the lens focal length f of the camera, the side length w of the edge to be measured, and the pixel width p formed by the edge to be measured in the imaging plane of the camera; the distance between the edge to be measured and the camera is calculated using the following formula:

[0027]

[0028] If one endpoint of the edge to be tested is the first detection point, then the edge to be tested is the edge to be tested where the first detection point is located, and then d is set to be equal to the distance dx1 from the camera to the edge to be tested where the first detection point is located; if one endpoint of the edge to be tested is the second detection point, then the edge to be tested is the edge to be tested where the second detection point is located, and then d is set to be equal to the distance dx2 from the camera to the edge to be tested where the second detection point is located; wherein, when the plane of the object to be tested is not parallel to the pinhole plane of the camera, dx1 is not equal to dx2, and the intersection line of the plane of the object to be tested and the pinhole plane of the camera is set perpendicular to the direction of the target straight line. In summary, it can be considered as constructing a pinhole imaging model between the plane of the object under test, the pinhole plane of the camera, and the imaging plane of the camera from the perspective of the side view (in fact, it is to use the geometric relationship of similar triangles from the side view of each plane (preferably the direction perpendicular to the horizontal ground). The length of the perpendicular line segment from the camera to the side of the object under test where the first detection point or the second detection point is located is calculated. Under the premise of ignoring the distortion caused by the vertical or horizontal direction, the plane of the object under test, the pinhole plane of the camera, and the imaging plane of the camera remain parallel to each other from the side view, saving the amount of calculation of the distance between the label pattern and the camera.

[0029] Furthermore, when the label pattern is represented as a triangular label, the two adjacent target detection points are the two vertices of the base of the triangular label. Each triangular label's corresponding target detection points are distributed along the target straight line in the plane of the object to be measured. The line connecting the two vertices of a triangular label that form a first angle with the target straight line is denoted as the side to be measured where the first detection point is located. The line connecting the two vertices of the same triangular label that form a second angle with the target straight line is denoted as the side to be measured where the second detection point is located. The sum of the second and first angles is equal to 180 degrees. The lengths of the sides to be measured where the first and second detection points are located are both preset to be less than a preset distortion error value. Thus, based on the two adjacent target detection points already identified by the robot, the sides to be measured where the first and second detection points are located are extracted within the same triangular label.

[0030] When the label pattern includes a rectangular label, the two adjacent target detection points are two vertices of the side of the rectangular label parallel to the target line direction. The two sides of the rectangular label perpendicular to the target line direction are the test sides where the first detection point and the second detection point are located, respectively. Both the test sides where the first and second detection points are located are parallel to the pinhole plane of the camera. Thus, effective test sides are extracted within the same rectangular label based on the two adjacent target detection points already identified by the robot.

[0031] Further, in step D, the robot selects the region formed by the angles formed by the perpendicular segments of the perpendicular lines from the identified triangular labels with different placement patterns to the unidentified area on both sides of the camera. Then, the robot sets the predicted position point within the region formed by the selected angle with the smallest angle. This allows the robot to determine, through step C, that the distances between the two triangular labels with different placement patterns located on either side of the middle position and the camera are the two sets of distances with the smallest values ​​among all the identified triangular labels and the camera. Each time step D is executed, the angle with the smallest angle selected by the robot is updated, thereby updating the predicted position point and guiding the robot to move from the identified triangular labels on both sides towards the unidentified area in the middle.

[0032] Furthermore, after the robot identifies the rectangular label in the target graphic label, during step D, as the robot moves towards the rectangular label at the center of the target graphic label, if the robot is detected to be to the left of the rectangular label at the center of the target graphic label based on the deflection angle of the currently identified rectangular label relative to the camera, the robot moves to the right to the new predicted position point; or, if the robot is detected to be to the right of the rectangular label at the center of the target graphic label based on the deflection angle of the currently identified rectangular label relative to the camera, the robot moves to the left to the new predicted position point; until at the latest determined predicted position point, it is determined that the distance between the rectangular label at the center of the target graphic label and the camera is the smallest among the distances between all identified rectangular labels and the camera, and the distance between the currently identified target graphic label and the camera satisfies the preset positioning conditions. This allows the robot to overcome the ranging error interference caused by image distortion, while aligning the robot's movement direction with the charging interface, correcting the robot's recharge direction, reducing errors in the recharge process, and improving recharge efficiency.

[0033] Furthermore, after the robot determines that the distance between the currently identified target graphic label and the camera meets the preset positioning conditions, the robot adjusts the optical axis of the camera to be perpendicular to the plane of the object under test by rotating. This changes the robot's movement direction to be parallel to the perpendicular direction of the rectangular label at the center of the target graphic label, thereby determining that the deflection angle of the currently identified target graphic label relative to the camera meets the preset positioning conditions. Making the plane of the object under test (understood as the plane where the charging port or the rectangular label is located) perpendicular to the optical axis of the camera ensures the accuracy of subsequent imaging and reduces or even eliminates errors.

[0034] Furthermore, if the robot detects that one edge of the label pattern is not parallel to the pinhole plane of the camera, it confirms that this non-parallel edge causes distortion in the camera, thus determining that the label pattern is not parallel to the pinhole plane and that the label pattern is distorted in the camera. Conversely, if the robot detects that one edge of the label pattern is parallel to the pinhole plane of the camera, it confirms that this parallel edge does not cause distortion in the camera. This achieves the detection of whether the label pattern in the plane of the object being tested is distorted in the camera.

[0035] Furthermore, when the robot identifies two adjacent target detection points from the plane of the object under test, it calculates the product of the distance Ux between the two adjacent target detection points in the plane of the object under test and the sine of the tilt angle of the plane of the object under test, Ux*sin(ax). Then, Ux*sin(ax) is set as the ranging error caused by the distortion of the line connecting the two adjacent target detection points in the camera. When the distance between the currently identified target graphic label and the camera, and the deflection angle of the currently identified target graphic label relative to the camera both meet the preset positioning conditions, the ranging error caused by the distortion of the line connecting the two adjacent target detection points in the camera falls within the preset target error range, where the preset target error range includes the value 0.

[0036] By utilizing the size difference between the edge lines of two adjacent target detection points of the label pattern before and after distortion, the timing for the robot to align with the rectangular label at the center of the target graphic label is determined. The ranging error caused by distortion is linked to the robot's preset positioning conditions, thereby improving the accuracy of the robot in contacting and aligning with the rectangular label at the center of the target graphic label.

[0037] Furthermore, when the plane of the object to be measured is not parallel to the pinhole plane of the camera, if the robot detects that the side of the rectangular label perpendicular to the straight line direction of the target is parallel to the pinhole plane of the camera, the robot determines that the ranging error caused by the distortion of the side of the rectangular label parallel to the horizontal direction in the camera is not equal to a value of 0, thus confirming that the side parallel to the horizontal direction is distorted in the camera. The ranging error caused by the distortion of the side to be measured in the rectangular label in the camera is equal to a value of 0, thus confirming that the side to be measured in the rectangular label is distorted in the camera. The side to be measured in the rectangular label includes the side where the first detection point is located and the side where the second detection point is located. Therefore, by calculating the ranging error generated by each side of the rectangular label, it is determined whether distortion has occurred, saving computational load and ensuring ranging accuracy.

[0038] Furthermore, when the distance between two adjacent target detection points obtained by the robot is less than 0.5*(dx1+dx2), the robot sets the ranging error caused by the distortion of the line connecting the two adjacent target detection points in the camera to be equal to 0. In summary, before executing the visual positioning control method, the label pattern on the plane of the object to be measured can be set to a relatively small size, thereby minimizing the ranging error caused by distortion when the plane of the object to be measured is not parallel to the imaging plane of the camera, i.e., suppressing the distance error caused by angle changes.

[0039] Furthermore, if the two vertices of the edge to be tested where the first detection point is located are updated to the two adjacent target detection points, then the edges originally distributed along the target line direction in the same triangle label are set as the edge to be tested where one of the updated target detection points is located, and the edge to be tested where the original second detection point is located is set as the edge to be tested where the updated other target detection point is located, so that the target line direction is updated to be parallel to the edge to be tested where the original first detection point is located; then the distance from the camera to the edge to be tested where one of the updated target detection points is located is set to be equal to dx1, and the distance from the camera to the edge to be tested where the updated other target detection point is located is set to be equal to dx2; when the distance between the two adjacent target detection points after the update is less than 0.5*(dx1+dx2), the robot sets the ranging error caused by the distortion of the line connecting the two adjacent target detection points in the camera to be equal to the value 0, and determines that the preset distortion error value is equal to 0.5*(dx1+dx2). In summary, during the execution of the aforementioned visual positioning control method, the label patterns containing two adjacent target detection points with a distance of less than 0.5*(dx1+dx2) are specifically selected for distance and angle calculation, thereby improving the accuracy of planar positioning calculation of the object under test.

[0040] Further, in step A, the robot preprocesses the images captured by its camera by: first, converting the captured images to grayscale to obtain a binarized image; then, performing Gaussian smoothing on the binarized image to remove image noise, resulting in a smoothed image; then, performing edge detection on the smoothed image to obtain a target edge map; and finally, performing dilation on the target edge map to obtain the preprocessed image, which the robot can use to identify the vertices and graphic attributes of the label pattern from the preprocessed image. This overcomes the problems of image blurring and noise interference within a specific light field of view. Therefore, a single monocular camera with a fixed focal length can achieve clear imaging, acquiring all vertices of the label pattern at once, thereby improving the accuracy of calculating the length of the edges connecting the vertices.

[0041] Further, in step A, the method for searching for the vertices of the label pattern within the preprocessed image includes: extracting a closed shape within the preprocessed image so that the closed shape represents the label pattern the robot needs to search for; wherein the closed shape is generated during edge detection; then extracting the vertices and graphic attributes of the label pattern based on the closed shape; then filtering the extracted vertices and graphic attributes of the label pattern to suppress noise in the label pattern; then setting the filtered vertices as the vertices of the label pattern searched by the robot within the preprocessed image, and setting the filtered graphic attributes as the graphic attributes of the label pattern searched by the robot within the preprocessed image to distinguish label patterns of different shapes. This allows the robot to obtain accurate graphic attributes and vertices. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating the visual positioning control method based on label patterns disclosed in an embodiment of this application.

[0043] Figure 2 This is a schematic diagram showing the distribution of triangular and rectangular labels disclosed in an embodiment of this application.

[0044] Figure 3 This is a schematic diagram of the image obtained after preprocessing the label pattern disclosed in the embodiments of this application.

[0045] Figure 4 This is a schematic diagram illustrating the principle of calculating the distance between the camera and the triangular label from a side view and calculating the deflection angle of the triangular label relative to the camera from a top view, as disclosed in this application embodiment.

[0046] Figure 5 This is a schematic diagram illustrating the principle of calculating the distance between the camera and the rectangular label from a side view and calculating the deflection angle of the rectangular label relative to the camera from a top view, as disclosed in this application embodiment.

[0047] Figure 6 This is a schematic diagram illustrating the principle of locating the charging interface using multiple triangular tags and multiple rectangular tags, as disclosed in an embodiment of this application. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described below are only for explaining the present invention and are not intended to limit the present invention.

[0049] In the description of the invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", and "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention.

[0050] This application discloses a visual positioning control method based on label patterns. The main body executing the visual positioning control method is a robot equipped with a camera. The robot identifies and positions the device with the label pattern by executing the visual positioning control method. It can be specifically used to locate the charging dock and its charging interface, or to locate other signal base stations or machine recycling base stations, so as to guide the robot to gradually approach or even align with the device with the label pattern by using the image information of the collected label pattern, and to contact the device in the correct posture.

[0051] To improve the visual positioning accuracy of a robot for a device to be positioned with a label pattern, this embodiment discloses a visual positioning control method based on the label pattern, such as... Figure 1 As shown, it includes:

[0052] Step A: The robot preprocesses the images captured by its camera, then searches for the graphic attributes and vertices of the label patterns within the preprocessed images to determine the position information of the label patterns on the surface of the device to be positioned and to identify their shape features; then, Step B is executed. The label patterns are placed on the surface of the device to be positioned so that the robot's camera can capture image information of the label patterns, including edge contour information. Specifically, a set of label patterns can be set along the horizontal direction on the surface of the device to be positioned. This set of label patterns includes multiple label patterns of the same or different shapes and sizes, and can be symmetrically arranged on the surface of the device to be positioned. The robot first controls the camera to capture images at a defined location. This camera is a monocular camera, and the camera's field of view covers some or all of the label patterns set on the device to be positioned. When the device to be positioned is a charging dock for the robot to replenish its power, the robot's camera can capture images of the label patterns set on the surface of the charging dock, but may not be able to distinguish the information of various shaped label patterns, including the direction corresponding to the charging interface.

[0053] Specifically, the label pattern may include labels of a specific shape and their vertex positions. The graphic attributes of the label pattern include information about the line segments connecting the labels of a specific shape. These line segments include the end-to-end line segments, which can be identified by the robot from the image, and the number of identified line segments can be counted. Instead of being placed on a surface in the positioning device that has a contact and docking structure with the robot's body parts, the label pattern is placed on a surface that can be detected by a camera at the front of the robot (i.e., fall within the camera's detection field of view). For example, the surface of the charging base for a robotic vacuum cleaner, specifically the charging interface installed on the surface of the base, consists of charging electrodes and other mechanical structures that cooperate with the charging electrodes docking with the robot, including but not limited to magnetic adsorption and elastic adsorption components. The charging interface is preferably mounted directly below the label pattern, and the orientation of the charging interface is parallel to the perpendicular line of the plane containing the label pattern. Multiple label patterns can be placed at equal intervals near the charging interface, and the charging interface itself also has a corresponding special pattern, forming a directional label to guide the robot back to its charging position.

[0054] Preferably, the robot performs preprocessing on the image captured by the camera in step A to remove noise interference, obtain higher precision graphic attributes and vertices of the label pattern, improve the recognition accuracy of the label pattern, and solve the problem of image blurring that easily occurs when the monocular camera equipped on the robot captures images with a fixed focal length.

[0055] Step B: Based on the graphic attributes and vertices of the label pattern, the distance between the label pattern and the camera, as well as the deflection angle of the label pattern relative to the camera, are calculated using the monocular ranging principle. This further determines the ranging error caused by label pattern distortion. Then, step C is executed. When the label pattern is placed on the surface of the charging dock and associated with the charging interface, the robot first identifies several larger label patterns at once (limited by the distance between the camera and the charging dock). Then, the positions of the pre-identified label patterns on the surface of the charging dock are stored for distance and angle calculations. This allows the positional characteristics of each vertex that makes up the label pattern on the surface of the charging dock to be determined, such as equal spacing or central symmetry about the charging interface. Specifically, based on the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera, the robot can determine the relative positional relationship between the robot and the label pattern. In addition to determining the robot's current position information (coordinates and angle), the positional information of the label pattern and the plane it occupies can also be determined.

[0056] In this embodiment, the deflection angle of the label pattern relative to the camera can be represented by the angle between the incident light at the corner of the label pattern extracted from the surface of the device to be positioned and the optical axis of the camera. The tilt angle of the plane where the label pattern is located can be the angle between the plane where the label pattern is located and the pinhole plane of the camera. The distance between the label pattern and the camera can be represented by the distance between the edge of the label pattern perpendicular to the horizontal ground and the pinhole plane of the camera (which can be reduced to the center position of the camera lens) to suppress the distortion effect of the label pattern in the camera (lens).

[0057] Step C: Determine whether the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera, both meet the preset positioning conditions. If yes, the visual positioning of the device to be positioned is considered complete, allowing the robot to align with the corresponding label pattern in the device. This may even allow the robot to move linearly in its current pose until it contacts the middle position (or the label pattern on the central axis) of the device. By setting the preset positioning conditions as a criterion, the influence of ranging errors caused by the robot acquiring data from multiple perspectives is overcome, improving visual positioning accuracy. Otherwise, proceed to step D. Determining the completion of visual positioning of the device includes completing the visual positioning of the corresponding label pattern in the device and the overall positioning of the device. When representing the overall positioning of the device, the positioning results of the local position corresponding to the label pattern or the position of the component most favorable to the docking robot can be used.

[0058] When the robot determines that it has completed the visual localization of the device to be located, if the device is a charging dock, the robot's movement direction is towards the charging dock's charging port, and the final predicted position is located at the charging port. Then, the newly calculated distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera, are used as the visual localization result, and the robot stops executing steps A to D. When the robot's movement direction is determined to be towards the charging dock's charging port, or to be parallel to the orientation of the charging port, it can be considered that the robot accurately completes the recharge docking operation by moving along the current movement direction. That is, when the label pattern corresponding to the charging port is recognized, the robot completes the docking charging of its charging electrodes with the charging port while moving in a straight line along the orientation of the charging port.

[0059] Preferably, determining whether the distance between the label pattern and the camera meets preset positioning conditions includes determining whether the distance between the label pattern and the camera falls within the distance error range specified by the preset positioning conditions, or determining whether the distance between the currently identified label pattern and the camera is the label pattern with the smallest distance to the camera among all identified label patterns, to determine that the robot has gradually explored the label area with the most positioning value from the middle of the identified label patterns; at the same time, determining whether the deflection angle of the label pattern relative to the camera meets preset positioning conditions includes determining whether the deflection angle of the currently identified label pattern relative to the camera is the location of a specific label pattern (e.g., the aforementioned label area with the most positioning value, generally set as the middle position of a group of label patterns). The angle corresponding to the opposite direction of the perpendicular direction of the plane (generally perpendicular to the outer surface of the device to be positioned), but the robot's forward direction or the optical axis of the camera is generally pointing towards the device to be positioned, is required for the camera to capture an image of the label pattern. Whether the deflection angle of the label pattern relative to the camera meets the preset positioning conditions also includes determining whether the deflection angle of the label pattern relative to the optical axis of the camera falls within the angle error range specified by the preset positioning conditions. This angle error range can represent the minimum allowable angle range between the optical axis of the camera and the perpendicular direction of the plane containing the specific label pattern (e.g., the label area with the most positioning value mentioned above, generally set as the middle position of a group of label patterns) (generally perpendicular to the outer surface of the device to be positioned). Thus, based on the preset positioning conditions, the robot is adjusted and controlled to gradually approach the device to be positioned by relying on the guidance direction formed by the specific arrangement order of the label patterns.

[0060] Step D: Based on the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera, the robot moves to the predicted position point, then uses the camera to capture an image of the label pattern, and then returns to step A to execute steps A to C. That is, if the distance between the label pattern and the camera and / or the deflection angle of the label pattern relative to the camera do not meet the preset positioning conditions, the robot moves to a new position and repeats steps A to C until the distance between the label pattern and the camera and the deflection angle of the label pattern relative to the camera both meet the preset positioning conditions. Only then is the visual positioning of the device to be positioned completed, and the execution of steps A to D stops.

[0061] If the device to be located is a charging dock, based on the aforementioned steps A to D, the predicted position point determined by the last executed step D can be a navigation position point in the direction corresponding to the tag pattern or charging interface (the specific representation of the charging position in the charging dock) that is relatively close to the charging position. Specifically, in step D, the robot uses the angle and distance calculated in real time in step B to convert into coordinate information in the corresponding polar coordinate system. This not only determines the robot's current position point but also initially determines the relative positional relationship of multiple identified tags in the charging dock relative to the unidentified area, thereby initially establishing the relative positional relationship between the charging position of the charging dock and the robot's current position point. Based on this relative positional relationship, the robot adjusts its pose and moves to a predicted position point. The predicted position point can be adjusted according to the actual focal length of the camera. By correcting the robot's movement direction, the tag pattern corresponding to the charging interface can be identified, and the robot's charging electrodes can be controlled to move closer to the charging position of the charging dock. By repeatedly executing steps A to D to correct the robot's position (updating the predicted position point), the influence of ranging errors caused by image distortion acquired by the camera can be overcome.

[0062] Compared to existing technologies, this method uses the vertices of identified label patterns to calculate the distance and angle information of the corresponding labels relative to the same camera. This allows the robot to predict the relative positional relationship between various label patterns and the same camera (which may be located in different positions), providing accurate positioning information for the robot to navigate back to the positioning device and make contact. Furthermore, it eliminates the need to spend excessive time collecting multiple frames of images at the same location. Instead, within the same frame, it uses monocular ranging to calculate the distance and angle information of multiple identified label patterns. By introducing preset positioning conditions and continuously calculating and updating the distance between the label pattern and the camera, as well as the deflection angle of the label pattern relative to the camera, the robot's movement direction is corrected to align with the orientation matched by the label pattern. This improves the visual positioning accuracy of the camera as the robot approaches the positioning device, suppresses the influence of camera ranging errors, and increases navigation accuracy while reducing the amount of real-time positioning calculations.

[0063] As one embodiment, step D specifically includes: when the label patterns involved in the judgment of step C are all positioning graphic labels, if it is determined that the distance between the positioning graphic label and the camera, or the deflection angle of the positioning graphic label relative to the camera, does not meet the preset positioning conditions, then according to the distance between the positioning graphic label and the camera, and the deflection angle of the positioning graphic label relative to the camera, the robot sets the predicted position point and moves to the predicted position point, so that the robot can identify the target graphic label at the predicted position point. It should be noted that the preset position point is updated every time the robot executes step D. If, in step C, the distance between the positioning graphic label and the camera meets the preliminary positioning conditions, then after the robot completes step D, it begins to identify the target graphic label at the newly set predicted position point. At this time, the distance between the robot's current position point (the newly set predicted position point) and the device to be positioned allows the camera to identify the target graphic label. Therefore, the preliminary positioning of the device to be positioned or its assembly port is completed, because the target graphic label represents the assembly port in the device to be positioned for the robot to contact. This initially suppresses the influence of ranging error caused by the camera acquiring patterns from multiple angles during the robot's movement, so as to delineate the location characteristics of the area where the target graphic label is located.

[0064] Then, the target graphic label is configured as the label pattern in steps B to D. Therefore, based on this, if all the label patterns involved in step C are target graphic labels, and if the distance between the target graphic label and the camera, or the deflection angle of the target graphic label relative to the camera, does not meet the preset positioning conditions, the robot updates the predicted position point and moves to the updated predicted position point according to the distance between the target graphic label and the camera, and the deflection angle of the target graphic label relative to the camera. Then, steps A to C are re-executed until step C determines that the distance between the target graphic label and the camera meets the preset positioning conditions. At this point, the robot completes the visual positioning of the assembly port of the device to be positioned.

[0065] It should be noted that the label pattern is a target graphic label or a positioning graphic label; the corresponding label pattern in the device to be positioned is a target graphic label. In step A, the robot identifies the positioning graphic label based on the graphic attributes of the label pattern, so that the currently identified positioning graphic label represents the label pattern and is used to execute steps B to D. That is, if the robot does not identify the target graphic label, the label pattern in steps B and D executed by the robot is the positioning graphic label. The robot identifies the target graphic label at the last set predicted position point, so that the currently identified target graphic label represents the label pattern and is used to execute step B. Then, the corresponding distances and angles calculated by the robot in step B, as disclosed in the aforementioned embodiments, all meet the preset positioning conditions, and the robot stops executing steps A to D. During this process, the label pattern can be composed of rectangular labels from the target graphic label, with a specific arrangement of rectangular labels guiding the robot closer to the assembly port of the device to be positioned. This achieves the robot's positioning of the assembly port at a specific location within the device to be positioned.

[0066] In one embodiment, the device to be positioned is a charging dock, and the target graphic label is used to represent the label of the charging dock's charging interface, with its vertical direction representing the docking direction of the charging interface. The assembly port in the device to be positioned, specifically for robot contact, is the charging interface of the charging dock. Before the robot identifies the target graphic label, during steps A to D, the robot sequentially traverses each identified positioning graphic label to guide the robot from the identified positioning graphic labels on both sides to the unidentified area in the middle, thus bringing the robot closer to the charging interface. Specifically, the robot sequentially identifies multiple positioning graphic labels as evenly distributed on both sides of the target graphic label, and sets each positioning graphic label as a label of the same type in the charging dock. Each time the robot sequentially traverses each positioning graphic label in a single direction, it confirms that the robot is approaching or moving away from the charging interface, which can correct the robot's movement direction.

[0067] At the last determined predicted position point, the robot identifies a rectangular label at the center of the target graphic label and determines that the distance between this rectangular label and the camera meets preset positioning conditions, ensuring the presence of the target graphic label within the previously unidentified area. It also determines that multiple positioning graphic labels are distributed on either side of the target graphic label, which is composed of multiple rectangular labels, preferably arranged in multiple rows and columns. This allows for vertical and horizontal pose correction, improving final positioning accuracy. Furthermore, the robot adjusts its pose to align with the rectangular label, ensuring that the deflection angle of the rectangular label relative to the camera meets preset positioning conditions, making the last determined predicted position point the location of the charging interface on the robot's walking plane. Specifically, at the last determined predicted position point, the robot's movement direction aligns with the charging interface, confirming the completion of visual positioning of the charging dock or charging interface. It's worth noting that the unidentified label area in the middle is the area where the target graphic label is positioned on the charging dock surface. However, during the robot's identification and positioning of the graphic label—that is, before the distance between the positioning graphic label and the camera meets the initial positioning conditions—the distance between the camera and the charging dock surface is relatively far, resulting in a blurred image of the graphic label in the camera's imaging plane. Therefore, the positioning graphic labels distributed on both sides of the target graphic label, along with multiple rectangular labels that make up the target graphic label, need to work together to guide the robot's movement direction towards the charging interface, thereby reducing the distance between the robot and the charging dock's charging interface.

[0068] Preferably, during step A, the robot first sequentially traverses the various positioning graphic labels distributed to the left of the target graphic label; then, it identifies the target graphic label. The robot traverses the positioning graphic labels from left to right at certain intervals, identifying the vertices, edges, shape, or area covered by the graphic itself of each label pattern. Alternatively, during step A, the robot starts from the rightmost positioning graphic label and sequentially traverses the various positioning graphic labels from right to left.

[0069] Preferably, the positioning graphic tags of the same shape type are the same in shape and size, but the positions of each positioning graphic tag are different. Specifically, they can be arranged along the same straight line at the corresponding positions on the surface of the charging base. It can be understood that each positioning graphic tag is set in the tag area, so that it can be identified by the robot before the robot identifies the target graphic tag, so as to find the location of the target graphic tag. Of course, the target graphic tag is also set in the tag area.

[0070] In the aforementioned embodiment, the size of a positioning graphic tag on the charging dock surface is larger than the size of any rectangular tag included in the target graphic tag on the charging dock surface. This ensures that: when the robot identifies the positioning graphic tag, it does not identify the rectangular tag, and consequently, it does not identify the target graphic tag; at this point, the robot is at a first predicted position. Furthermore, when the robot identifies the rectangular tag, it does not identify the positioning graphic tag; at this point, the robot is at a second predicted position. The distance between the first predicted position and the charging interface is greater than the distance between the second predicted position and the charging interface. The positioning graphic tag does not include a rectangular tag to create a difference relative to the target graphic tag, thus providing pose correction for identifying the target graphic tag and improving the accuracy of charging interface position recognition.

[0071] Therefore, within the surface of the charging dock where the label pattern is located, if the coverage area of ​​a positioning graphic label on the charging dock surface is greater than the coverage area of ​​a rectangular label on the charging dock surface, then within the same focal length environment, when the robot's camera is at a position far away from the charging dock, such as a position where the distance from any of the label patterns is much greater than the focal length, the robot will first identify the graphic attributes of the positioning graphic label (the complete edge information of the label forming a triangle, specifically including the number of sides) and can determine that a positioning graphic label has been identified. However, due to the greater distance and the smaller size of the rectangular label, the camera does not have the conditions for imaging. Therefore, the robot at the same position will not identify the rectangular label, and thus will not identify the target graphic label.

[0072] During the repeated execution of steps A to C, the robot sequentially identifies each positioning graphic label before starting to identify the target graphic label. When approaching a rectangular label, for example, at a position where the distance to one of the rectangular labels is less than or equal to the focal length, all rectangular labels can be identified, thus confirming the identification of a target graphic label. This can also be understood as the robot confirming the identification of a target graphic label after consecutively identifying multiple rectangular labels. This guides the robot from the identified positioning graphic labels on both sides towards the unidentified image area in the middle, reducing the impact of ranging errors.

[0073] As an embodiment of locating graphic labels, regarding steps A to C mentioned above, in step A, the robot identifies the location graphic label and / or target graphic label from multiple label patterns at once based on the graphic attributes of the label pattern, thereby obtaining the vertices and graphic attributes of the identified label pattern. Whenever the robot acquires the graphic attributes and vertices of a label pattern, it already possesses the parameter basis for recognizing the currently acquired label pattern. The graphic attributes and vertices of a label pattern constitute the basic judgment elements for the robot to recognize a label pattern of a specific shape. Whenever the robot identifies multiple positioning graphic tags, in step B, based on the graphic attributes and vertices of each positioning graphic tag, it calculates the distance between each positioning graphic tag and the camera, as well as the deflection angle of each positioning graphic tag relative to the camera, using the monocular ranging principle. Then, it sequentially traverses the distances between each identified positioning graphic tag and the camera, which can be done along a straight line to systematically reflect the changing patterns of the distances between each positioning graphic tag and the camera. When the robot determines in step C that the distances between the two positioning graphic tags with different placement patterns located on either side of the middle position and the camera are not the two smallest distances among the identified positioning graphic tags and the camera, it determines that the distances between the currently identified tag patterns and the camera do not meet the preset positioning conditions, that is, the robot is far away from the two closest positioning graphic tags on either side of the middle position.

[0074] To distinguish the orientation of the positioning graphic labels relative to the target graphic label, the positioning graphic labels located on both sides of the target graphic label are set to different placements. For example, one positioning graphic label has a vertex facing upwards and a bottom edge parallel to the horizontal direction, with the vertex located above the bottom edge; the other positioning graphic label has a vertex facing downwards and a bottom edge parallel to the horizontal direction, with the vertex located below the bottom edge. Generally, two positioning graphic labels with different placements located on either side of the middle position and closest to each other can be considered as the two positioning graphic labels located on either side of the middle position and closest to each other. Since robots typically move on horizontal surfaces, the positioning graphic tags used to correct the robot's direction of movement can be arranged in a horizontal row. The middle position here refers to the center of the area where multiple identified positioning graphic tags are located, specifically the center of the area set by a row of positioning graphic tags. The two closest positioning graphic tags on either side of the middle position are the positioning graphic tag adjacent to the left and the positioning graphic tag adjacent to the right of the middle position in the same row. The distance between each positioning graphic tag and the camera represents the distance between each positioning graphic tag and the robot. Since each positioning graphic tag has at least two sides in the vertical or horizontal direction, the distance between a positioning graphic tag and the robot may have at least two distance values ​​from the same camera to at least two sides. Therefore, the at least two distance values ​​corresponding to a positioning graphic tag are recorded as a set of distances. The more distances, the higher the positioning accuracy. The comparison of distance values ​​across multiple groups is performed by comparing the distances corresponding to edges set in the same direction within each group. If one group has all its edges set in the same direction corresponding to the smallest distance, it is classified as the group with the smallest distance. Similarly, if two groups have all their edges set in the same direction corresponding to the smallest distance, they are classified as the two groups with the smallest distance. Therefore, this embodiment accurately excludes unsuitable locations and guides the subsequently set predicted location points to the area between the two closest positioning graphic labels located on either side of the central position.

[0075] Based on the above embodiments, regarding steps A to D, the robot executes step D to set a predicted position point in front of the two positioning graphic tags, which are located on opposite sides of the middle position and have different placement patterns and are closest to each other. Then, the robot adjusts its pose (including the robot's direction of movement) according to the deflection angle of the currently set predicted position point relative to the camera and moves to the currently set predicted position point to shorten the distance between the robot and the target graphic tag, or to shorten the distance between the robot and the charging port of the charging dock, so that the final set predicted position point is the charging port of the charging dock.

[0076] Then the robot repeats steps A to D until step C determines that the distance between the two closest positioning graphic tags on both sides of the middle position and the camera is the two smallest distances among the identified positioning graphic tags and the camera. At this point, the distance between the positioning graphic tags and the camera satisfies the preliminary positioning condition. However, the distance between the currently identified tag patterns and the camera does not meet the preset positioning condition. New predicted position points need to be set to move towards the narrowed unknown area. At this time, the robot has not yet identified the target graphic tag or the rectangular tags contained within it, but the distance relative to the target graphic tag is closer.

[0077] Therefore, after performing step D, the robot begins to identify the target graphic label in step A; if the graphic label is located using... Figure 2 The triangle label indicates that, in Figure 2 The combination of rectangular labels in the middle position represents the target graphic label. The three vertices of the triangular label to the left of the target graphic label are arranged with two vertices at the top and the remaining vertex at the bottom. The three vertices of the triangular label to the right of the target graphic label are arranged with one vertex at the top and the remaining two vertices at the bottom, thus forming two groups of triangular labels with different arrangements. Each group contains three triangular labels. Figure 2 and Figure 6 It is known that the target graphic label is located between the triangle label with vertex T3 and the triangle label with vertex T4. The center of the target graphic label contains a rectangle label with vertices C2 and D2. At present, it is only determined that the target graphic label is located between the triangle label with vertex T3 and the triangle label with vertex T4. Moreover, the triangle label with vertex T3 and the triangle label with vertex T4 have been located. Therefore, the area where the located target graphic label is located can be delineated, and then handed over to the subsequent steps B to D for visual positioning.

[0078] It is worth noting that before the robot identifies the target graphic label in step A, the distance between each currently identified label pattern and the camera is not allowed to meet the preset positioning conditions. That is, it is constrained by the predicted position points set in each step D, forming a route that tends towards the target graphic label but not towards the positioning graphic label. Therefore, in this embodiment, the robot will sequentially traverse multiple positioning graphic labels before successfully identifying the target graphic label located in the middle. Since the robot's movement direction can represent the optical axis direction of the camera, representing the lens orientation, when the robot successfully identifies a target graphic label, its movement direction is aligned with that target graphic label. Thus, multiple positioning graphic labels serve to correct the robot's movement direction.

[0079] As an embodiment of locating a target graphic label, regarding the aforementioned steps A to D, after the robot identifies the target graphic label in step A, it further identifies the individual rectangular labels that make up the target graphic label. Preferably, after the robot sets the predicted position point to a distance from the plane where the label pattern is located to a preset correction distance, the robot cannot identify the positioning graphic label based on the graphic attributes of the label pattern in the newly executed step A. Instead, it identifies the target graphic label so that the currently identified target graphic label represents the label pattern. The preset correction distance is determined by the focal length of the camera and is positively correlated with the size of the imaging area of ​​the label pattern or the actual coverage area on the surface of the charging dock, reducing the difficulty for the robot to identify the target graphic label and the number of times the predicted position point needs to be set.

[0080] Then, in step B, based on the graphic attributes and vertices of each rectangular label, the distance between each rectangular label and the camera, as well as the deflection angle of each rectangular label relative to the camera, are calculated using the monocular ranging principle. Then, the distances between each identified rectangular label and the camera are traversed sequentially. This can be done by traversing each rectangular label along a pre-set straight line. If there are vertical and horizontal directions, the distances between each rectangular label and the camera need to be traversed row by row and column by column.

[0081] When the robot determines in step C that the distance between the rectangular label at the center of the target graphic label and the camera is not the smallest among the distances between the identified rectangular labels and the camera, it determines that the distance between the currently identified target graphic label and the camera does not meet the preset positioning conditions. It should be noted that the center position here can be the center position of the area set by the multi-row, multi-column target graphic labels, such as the center of symmetry. The set of distances between each rectangular label and the camera represents the distance between the target graphic label and the robot. Since a rectangular label has two sides in either the vertical or horizontal direction, the distance between a rectangular label and the robot may have two values: the distance from the same camera to both sides. Therefore, the two distance values ​​corresponding to a rectangular label are recorded as a group of distances. The more rectangular labels there are, the more distance groups corresponding to the same target graphic label, and the higher the accuracy of positioning using the target graphic label. The comparison of the values ​​of multiple distance groups is done by comparing the distances corresponding to the sides set in the same direction within each group. If there is a group where the distances corresponding to the sides set in all directions are classified as the smallest, then the robot is considered to have the smallest distance.

[0082] Then, in step D, the robot moves towards the rectangular label at the center of the target graphic label. The distance between the rectangular label at the center of the target graphic label and the camera is the smallest among the distances between the identified rectangular labels and the camera. This ensures that the distance between the currently identified target graphic label and the camera meets the preset positioning conditions. The currently moved position is the latest predicted position, which can be denoted as the location of the charging port. Then, at the latest predicted position, the robot's movement direction is adjusted to be parallel to the perpendicular direction of the rectangular label at the center of the target graphic label, specifically pointing towards the interior of the device to be positioned where the target graphic label is located. Therefore, it can be determined that the distance between the currently identified target graphic label and the camera, and the deflection angle of the currently identified target graphic label relative to the camera, both meet the preset positioning conditions. This completes the visual positioning of the charging dock, including the visual positioning of the charging port, which is essentially precise positioning after overcoming distortion errors caused by various viewing angles. Figure 2 and Figure 6 It is known that the target graphic label is located between the triangle label with vertex T3 and the triangle label with vertex T4. At the center of the target graphic label, there is a rectangle label with vertices C2 and D2. Currently, not only can the specific position information of each rectangle label of the target graphic label be determined (including the angle and distance relative to the camera), but also the rectangle label with vertices C2 and D2 can be accurately located. That is, the distance between the rectangle label with vertices C2 and D2 and the camera is the smallest value among the distances between the identified rectangle labels and the camera. This allows the robot to walk in a straight line towards the rectangle label with vertices C2 and D2 until they make contact.

[0083] In summary, this embodiment first determines the position of the target graphic tag based on the identified positioning graphic tag, which achieves a coarse positioning effect for the charging interface. Then, the rectangular tag inside the target graphic tag continuously changes the position that the robot needs to navigate to, and continuously reduces the distance between the robot and the charging interface until the latest calculated distance and angle meet the preset positioning conditions. Only then is the precise positioning of the charging interface completed, which improves the accuracy of the camera's positioning of the charging interface and thus improves the accuracy of the robot returning to the charging dock for charging.

[0084] As one embodiment, the robot identifies a positioning graphic tag as consisting of a triangular tag, and determines the side length of the triangular tag by its vertices. Specifically, within the imaging plane of the camera, the length of the line connecting adjacent vertices is calculated from the shape and vertices of the positioning graphic tag, which serves as the side length of the triangular tag, thus determining the sides that enclose the triangular tag. The vertices of the tag pattern include the three vertices of each triangular pattern, that is, the common endpoints of the two sides of the triangular tag. Figure 2 As shown by the triangular pattern on the left and the inverted triangular pattern on the right, the three positioning graphic labels arranged to the left of the rectangular label are all triangles in shape and are evenly spaced, represented as follows: Figure 2 The triangle containing vertex T1, the triangle containing vertex T2, and the triangle containing vertex T3; the three positioning graphic labels arranged to the right of the rectangle label are all inverted triangles, and are equally spaced, represented as follows: Figure 2 The triangle containing vertex T4, the triangle containing vertex T5, ​​and the triangle containing vertex T6. This embodiment will... Figure 2 The triangles and inverted triangles shown are both considered as triangle labels. Moreover, the triangle and the inverted triangle are triangle labels with different placement patterns. For example, in the triangle label to the left of the rectangular label, one vertex of the triangle label is located above the base of the triangle label; in the triangle label to the right of the rectangular label, one vertex of the triangle label is located below the base of the triangle label, forming triangle labels with different placement patterns on both sides of the middle position.

[0085] The placement of the positioning graphic tags on one side of the target graphic tag differs from that on the other side. Each positioning graphic tag on each side of the target graphic tag is evenly spaced on the charging base surface. Furthermore, among the positioning graphic tags on the same side of the target graphic tag, any one positioning graphic tag is designed to be obtained by translating another positioning graphic tag along the extension direction of the charging base surface. This can be understood as a translation from left to right or from right to left on the charging base surface. The robot then systematically identifies each positioning graphic tag along a straight line to distinguish those with the same placement, thereby determining the position of each positioning graphic tag on the charging base surface.

[0086] The robot identifies a target graphic label as composed of multiple identical rectangular labels arranged together, and determines the side length of each rectangular label by its vertices. A target graphic label is a regular polygonal shape arranged around a rectangular label as its center. Within this target graphic label, a portion of the neighborhood of the center position has no rectangular labels distributed, ensuring that all rectangular labels distributed around the center position do not exceed a complete circle. That is, the outer rectangular labels do not encircle the rectangular label at the center of symmetry. This can be represented by the different numbers of rectangular labels distributed in the neighborhoods on either side of the center position, thus distinguishing the two sides of the target graphic label. For example, reserving space... Figure 2 The top left and bottom right corners of the target graphic label are not filled with rectangular labels, which helps to distinguish the orientation of the target graphic label. Preferably, the charging interface is located directly below the target graphic label. When the robot identifies and locates the target graphic label, the orientation of the charging interface relative to the target graphic label can also be determined, enabling the robot to locate or even point to the charging interface. In this way, the height of the target graphic label and the charging interface on the charging base surface can be adapted to the assembly height of the robot's charging electrodes. The robot only identifies a target graphic label after searching four or more corner positions or four or more edges, highlighting the specificity of the target graphic label and improving the positioning accuracy of the target graphic label.

[0087] In the aforementioned embodiments, during the execution of step D or the repeated execution of steps A to C, the robot sequentially identifies each positioning graphic label along a straight line. Generally, it identifies each positioning graphic label from left to right or from right to left on the plane containing the label pattern, distinguishing the positioning graphic labels with the same arrangement among those placed on the same side of the target graphic label. In some embodiments, the robot's identification direction within the plane containing the label pattern is parallel to the robot's walking plane and can be parallel to the base of each triangular label, so that the triangular label on one side of the target graphic label is obtained by rotating the triangular label on the other side of the target graphic label 180 degrees around the center of symmetry of the target graphic label. Therefore, corresponding to... Figure 2 In the diagram, the three triangular labels to the left of the rectangular label at the center are obtained by rotating the three triangular labels to the right of the rectangular label at the center by 180 degrees around the center. During the traversal of the label pattern, these labels are used to distinguish the left and right sides of the target graphic label, so that the positioning graphic labels distributed on the left and right sides of the target graphic label correspond to labels with different placement patterns. This facilitates guiding the robot located on the left side of the target graphic label to move to the right, or guiding the robot located on the right side of the target graphic label to move to the left, so that the robot's movement direction continuously approaches the target graphic label.

[0088] As one example, see Figure 2 It can be seen that the target graphic label contains three rows of rectangular labels. The row passing through the center contains three rectangular labels, preferably square labels. A rectangular label is placed at the center of the target graphic label and is parallel to the central axis of the charging dock. The other two rectangular labels are located on either side of the center. The center position is filled with a rectangular label as shown below. Figure 2 The rectangular labels for vertices C2 and D2 are shown in the diagram. The row passing through the center is designated as the middle row. Two rectangular labels are placed in the row above the middle row, aligned with the central label and the label on one side of it, respectively. Two rectangular labels are also placed in the row below the middle row, aligned with the central label and the label on the other side of it, respectively, to indicate the direction directly above, below, to the left, and to the right of the target graphic label. Specifically, combined with... Figure 2 It can be seen that the records in the same row are arranged from left to right, and the records in the same column are arranged from top to bottom. The leftmost end of the first row of the target graphic labels does not have a rectangular label to indicate the top of the target graphic label; the rightmost end of the last row of the target graphic labels does not have a rectangular label to indicate the bottom of the target graphic label; the top of the first column of the target graphic labels does not have a rectangular label to indicate the left side of the target graphic label; and the bottom of the last column of the target graphic labels does not have a rectangular label to indicate the right side of the target graphic label. Figure 2 The diagram shows an octagon composed of seven rectangular labels. Therefore, the number of these rectangular labels distributed in the neighborhoods on either side of the center position of the target graphic label differs, allowing the robot to accurately distinguish the left and right sides of the target graphic label during the traversal of the label pattern.

[0089] Preferably, the charging interface is located directly below the rectangular label at the center of symmetry of the target graphic label. The specific assembly height is adapted to the assembly height of the robot's docking electrode within the robot body to improve the accuracy of robot docking and charging. When the robot's movement direction is parallel to the vertical direction of the rectangular label at the center of symmetry, and the robot's movement direction is towards the rectangular label at the center of symmetry, the robot's movement direction is aligned with the charging interface. This can be understood as the robot's movement direction being opposite to the orientation of the charging interface.

[0090] Preferably, in order to facilitate the robot to identify each positioning graphic label through the imaging plane of the camera, a specific traversal direction is specified on the imaging plane of the camera, such as parallel to the horizontal coordinate axis in the image coordinate system of the camera, so as to traverse the corresponding image pixels of each label pattern in the imaging plane in turn through the image coordinate system, thereby realizing the sequential identification of each positioning graphic label.

[0091] In the foregoing embodiments, the method for identifying a location graphic label and / or a target graphic label from multiple label patterns at once based on the graphic attributes of the label pattern includes: when the robot searches for the graphic attributes of the label pattern and the vertices of the label pattern in the preprocessed image, the robot detects the number of edges that form a closed shape; wherein, the graphic attributes of the label pattern are the edge line features of the closed shape, and the edge line features of the closed shape include the number of edges that form the closed shape and the number of vertices of the closed shape; in the closed shape, the robot identifies the line connecting two adjacent vertices as an edge that forms a closed shape, which in this embodiment refers to the edge line and excludes the diagonal of the closed shape; preferably, the closed shape is formed by multiple line segments connected end to end, or by multiple pixels connected sequentially, wherein the pixels at the corner positions of the closed shape are identified as vertices of the closed shape. Therefore, during the process of detecting the number of sides of a closed shape in the acquired image, when the robot detects 3 sides forming a closed shape, it identifies the currently detected closed shape as a triangle label and determines the location graphic label; when the robot detects 4 sides forming a closed shape, it identifies the currently detected closed shape as a rectangle label; if the number of rectangle labels detected by the robot in the same frame is the total number of rectangle labels required to form a target graphic label, and the detected rectangle labels are symmetrically set with one rectangle label as the center position, and the number of rectangle labels distributed in the two neighboring areas on both sides of the center position is different, then a target graphic label is determined to be identified. Thus, by combining graphic attributes to analyze the shape of the label pattern and the distribution characteristics of the internal pixels, different shapes of labels can be distinguished. Furthermore, by combining the number and distribution position of different shaped label patterns, the target graphic label and the location graphic label can be identified.

[0092] As one embodiment, in step B, the method for calculating the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera using the monocular ranging principle includes: the robot sets the target straight line direction as the extension direction of the plane of the object to be measured on the horizontal plane, and sets the target straight line direction as parallel to the robot's walking plane. The robot sets the direction of the target straight line direction as parallel to the extension direction of the horizontal coordinate axis in the imaging plane of the camera on the horizontal plane, and sets the horizontal coordinate axis in the imaging plane of the camera as parallel to the robot's walking plane, so as to facilitate the identification of the label pattern in the imaging plane of the camera. Moreover, the vertical plane perpendicular to the target straight line direction is perpendicular to the robot's walking plane, or it can be parallel to the pinhole plane of the camera to suppress the influence of distortion error. The pinhole plane of the camera is parallel to the imaging plane of the camera. For example, the vertices of the identified label pattern are traversed along the positive direction of the horizontal coordinate axis in the imaging plane. Here, the identified label pattern is the label pattern that has been clearly imaged in the imaging plane of the camera, and its graphic attributes and vertices are determined by the robot.

[0093] Then, the robot sets two vertices of a label pattern distributed along the target straight line as two adjacent target detection points in the plane of the object under test, serving as the two endpoints of an edge of the label pattern. The distance Ux between two adjacent target detection points in the plane of the object under test is obtained in advance, representing the actual width of a label pattern. This distance can be derived through mechanical design, measured with a simple ruler, or directly obtained by querying lens factory parameters (including radius of curvature, distance, refractive index, etc.). The distance Ux between two adjacent target detection points can be calculated when identifying the graphic attributes and vertices of the label pattern. The plane of the object under test refers to the surface of the charging base where the label pattern is located, perpendicular to the horizontal ground or the robot's walking plane.

[0094] In this embodiment, the monocular ranging principle includes a pinhole imaging model and the geometric proportions of similar triangles. Among two adjacent target detection points, the robot designates one as the first detection point, typically the left-hand target detection point, and uses the pinhole imaging model to calculate the distance dx1 from the edge to be measured at the first detection point to the camera, which is correlated with the side length in the label pattern perpendicular to the target line direction. The robot designates the other target detection point as the second detection point, typically the right-hand target detection point, and uses the pinhole imaging model to calculate the distance dx2 from the edge to be measured at the second detection point to the camera. Correspondingly, the camera lens has a pinhole plane passing through the lens center (optical center of the camera), and an imaging plane located behind the lens, which are respectively... Figures 4 to 6In the diagram, planes Lf (pinhole plane) and Lp (imaging plane) are represented. These planes are shown from both a top-down view (perpendicular to the robot's walking plane) and a side-view view (parallel to the robot's walking plane), and are both displayed as line segments. The plane of the object under test represents the charging dock plane where the label patterns are located. Since all label patterns are placed on the same plane, the plane of the object under test can also represent the plane where the charging interface is located. The pinhole plane, in the calculation scenario of the pinhole imaging model, is the plane where the optical center of the camera is located; the optical center of the camera is the center of the lens.

[0095] Based on the side-angle relationships of a triangle, the deflection angle of the label pattern relative to the camera is calculated using the following formula:

[0096]

[0097] dx2*sin(bx21)=Ux*sin(ax)+dx1*sin(bx11);

[0098] Ux*cos(ax)=dx2*cos(bx21)+dx1*cos(bx11);

[0099] bx12 = 90 - bx11;

[0100] bx22 = 90 - bx21;

[0101] The deflection angle of the label pattern relative to the camera includes the angle bx12 between the perpendicular segment from the camera to the side to be tested where the first detection point is located and the optical axis, and the angle bx22 between the perpendicular segment from the camera to the side to be tested where the second detection point is located and the optical axis. The angle bx11 is the angle formed by the perpendicular segment from the camera to the side to be tested where the first detection point is located and the pinhole plane of the camera in the opposite direction to the target straight line. The perpendicular segment from the camera to the side to be tested where the first detection point is located can be understood as the perpendicular segment passing through the optical center of the camera and perpendicular to the edge of the label pattern passing through the first detection point. This is applicable to the calculation of the distance between the camera and the side to be tested where the first detection point is located in the pinhole imaging model. The deflection angle of the label pattern relative to the camera includes the angle bx12 between the perpendicular segment from the camera to the side to be tested where the first detection point is located and the optical axis, which is equal to the difference between 90 degrees and bx11. This can be understood by relating it to the correspondence between top-down and side-down views. That is, the distance dx1 from the edge to be tested where the first detection point is located to the camera can be regarded as constructing a pinhole imaging model between the plane of the object to be tested, the pinhole plane of the camera, and the imaging plane of the camera from the side-down view (in fact, it is to use the geometric relationship of similar triangles from the side-down view of each plane (preferably the direction perpendicular to the horizontal ground). The length of the perpendicular segment from the camera to the edge to be tested where the first detection point is located is calculated, and then the distance dx1 from the edge to be tested where the first detection point is located to the camera is obtained from the top-down view. Based on this, the length of the perpendicular segment from the camera to the edge to be tested where the first detection point is located is equivalent to the distance dx1 from the edge to be tested where the first detection point is located to the camera.

[0102] The angle between the perpendicular segment from the camera to the side to be tested where the second detection point is located and the pinhole plane of the camera in the direction of the target straight line is denoted as bx21. The perpendicular segment from the camera to the side to be tested where the second detection point is located can be understood as the perpendicular segment passing through the optical center of the camera and perpendicular to the edge of the label pattern passing through the second detection point. It is applicable to the calculation of the distance between the camera and the side to be tested where the second detection point is located in the pinhole imaging model. The deflection angle of the label pattern relative to the camera includes the angle between the perpendicular segment from the camera to the side to be tested where the second detection point is located and the optical axis, bx22, which is equal to the difference between 90 degrees and bx21. This can be understood by relating it to the correspondence between the top-down and side-down perspectives. That is, the distance dx2 from the edge to be tested where the second detection point is located to the camera can be regarded as constructing a pinhole imaging model between the plane of the object to be tested, the pinhole plane of the camera, and the imaging plane of the camera from the side-down perspective (in fact, it is to use the geometric relationship of similar triangles from the side-down perspectives of each plane (preferably the direction perpendicular to the horizontal ground). The length of the perpendicular segment from the camera to the edge to be tested where the second detection point is located is calculated, and then the distance dx2 from the edge to be tested where the second detection point is located to the camera is obtained from the top-down perspective. Based on this, the length of the perpendicular segment from the camera to the edge to be tested where the second detection point is located is equivalent to the distance dx2 from the edge to be tested where the second detection point is located to the camera.

[0103] It should be noted that the distance between the label pattern and the camera mentioned in step B includes the distance from the edge to be measured where the first detection point is located to the camera, and the distance from the edge to be measured where the second detection point is located to the camera; the monocular ranging principle mentioned in step B includes the pinhole imaging model.

[0104] When the label pattern is a target graphic label, the distance between the target graphic label and the camera includes the distance from the side to be tested where the first detection point of each rectangular label that makes up the target graphic label is located to the camera, and the distance from the side to be tested where the second detection point of the same rectangular label is located to the camera. Similarly, the deflection angle of the target graphic label relative to the camera includes the deflection angle of each rectangular label that makes up the target graphic label relative to the camera.

[0105] The robot sets the angle between the plane of the object under test and the pinhole plane of the camera as the tilt angle of the object under test, where ax is the tilt angle of the object under test. The tilt angle of the plane containing the label pattern is also the tilt angle of the object under test. In Formula 1, distances dx1, dx2, and Ux are known quantities, while angles bx11, bx21, bx12, bx22, and ax are unknown quantities. The equation constructed from Formula 1 can be used to obtain angles bx11, bx21, bx12, bx22, and ax. Thus, by constructing a function relationship between the cosine theorem and the sides and angles of a triangle corresponding to the aforementioned formula using the pinhole imaging model, the tilt angle of the object under test (the plane where the charging interface is located) and the azimuth information of the label pattern at the first and second detection points relative to the camera can be calculated. Therefore, a monocular camera with a fixed focal length can be used to determine the specific angle information of the carrier plane containing a label pattern. In summary, distances dx1 and dx2 are calculated from the side view, while distances Ux, angles bx11, bx21, bx12, bx22, and ax can be obtained from the top view. The side and top views can be considered as two perpendicular directions, with one preferably vertical and the other horizontal. Therefore, in this embodiment, when calculating the deflection angle of the label pattern relative to the camera, Formula 1 is constructed sequentially from both vertical and horizontal perspectives for the corresponding target detection points and their edges. This is used for trigonometric calculations to ensure the comprehensiveness and representativeness of the angle calculations. Furthermore, based on the tilt angle of the object's plane, the influence of various label patterns on the ranging error generated by the camera can be calculated. For a detailed analysis, please refer to the aforementioned embodiments corresponding to triangular and rectangular labels.

[0106] As an implementation method for calculating the position information of the triangular label relative to the camera, see [link to relevant documentation]. Figure 4 It can be seen that the label pattern recognized by the robot is Figure 4 In the implementation scenario of triangle TC1D1, step B involves calculating the distance between the triangular tag and the camera, and the deflection angle of the triangular tag relative to the camera, using the monocular ranging principle. This includes combining... Figure 4It can be seen that the robot sets two vertices C1 and D1, which are distributed along the target line direction in a triangle TC1D1, as two target detection points in the plane Lr1 of the object to be measured. The distance between target detection point C1 and target detection point D1 is denoted as U1, which is the length of the base of the triangle. U1 is equal to Ux in the aforementioned formula 1. The robot designates the target detection point C1, which is biased to the left, as the first detection point C1, and the distance from the side to be measured where the first detection point C1 is located to the location O of the camera (denoted as the optical center of the camera) is denoted as d1. d1 is equal to dx1 in the aforementioned formula 1 and is obtained in advance using the pinhole imaging model. The robot designates the target detection point D1, which is biased to the right, as the second detection point D1, and the distance from the side to be measured where the second detection point D1 is located to the location O of the camera (denoted as the optical center of the camera) is denoted as d2. d2 is equal to dx2 in the aforementioned formula 1 and is obtained in advance using the pinhole imaging model.

[0107] The line connecting two vertices of a triangular label that form a first angle with the target line is denoted as the first side to be measured of the triangular label, corresponding to... Figure 4 The side C1T of the triangle is taken as the side to be tested where the first detection point C1 is located; similarly, the line connecting two vertices of the same triangle label that are distributed along the second angle with the target line is denoted as the second side to be tested of the triangle label, corresponding to... Figure 4 The side D1T of the triangle is taken as the side to be measured where the second detection point D1 is located; wherein the sum of the angles of the second included angle and the first included angle is equal to 180 degrees. It should be noted that the incident light from the first detection point C1 and the incident light from the second detection point D1 intersect at the optical center O of the camera, the optical center O of the camera is located on the pinhole plane Lf of the camera, and the distance from the optical center O of the camera to the imaging plane Lp of the camera is equal to the focal length f of the camera lens. Preferably, in order to reduce the influence of camera distortion, especially when the plane of the object to be measured where the triangular label is located is not parallel to the pinhole plane of the camera, the lengths of the side to be measured where the first detection point is located and the lengths of the side to be measured where the second detection point is located are both preset to be less than the preset distortion error value; or even much less than the preset distortion error value, then d1 and d2 can be directly calculated in advance using the pinhole imaging model.

[0108] Based on the relationship between the sides and angles of a triangle, the deflection angle of the triangle label TC1D1 relative to the camera is calculated using the following formula:

[0109]

[0110] d2*sin(b21)=U1*sin(a1)+d1*sin(b11);

[0111] U1*cos(a1)=d2*cos(b21)+d1*cos(b11);

[0112] b12 = 90 - b11;

[0113] b22 = 90 - b21;

[0114] exist Figure 4 In the diagram, the plane Lv1 passing through the first detection point C1 (actually a virtual plane passing through the first side to be tested C1T and parallel to the pinhole plane Lf of the camera), the plane Lv2 passing through the second detection point D1 (actually a virtual plane passing through the second side to be tested D1T and parallel to the pinhole plane Lf of the camera), the pinhole plane Lf, and the imaging plane Lp are parallel to each other.

[0115] The robot sets the angle between the plane Lr1 of the object to be tested, where the target detection points C1 and D1 are located, and the pinhole plane Lf (or plane Lv1, or imaging plane Lp) of the camera as the tilt angle a1 of the plane of the object to be tested. a1 is equivalent to ax disclosed in the aforementioned formula and is an angle value that urgently needs to be calculated and obtained.

[0116] The angle between the perpendicular segment from the camera to the side to be tested at the first detection point C1 and the pinhole plane Lf of the camera in the opposite direction of the target straight line is denoted as b11, which is equivalent to the angle between the perpendicular segment from the camera to the side to be tested at the first detection point C1 and the plane Lv1 in the target straight line direction. The angle between the perpendicular segment from the camera to the side to be tested at the second detection point D1 and the pinhole plane Lf of the camera in the target straight line direction is denoted as b21, which is equivalent to the angle between the perpendicular segment from the camera to the side to be tested at the second detection point D1 and the plane Lv1 in the opposite direction of the target straight line direction. The angle between the perpendicular line segment passing through the optical center O of the camera and perpendicular to the first side C1T to be measured and the plane Lv1 is denoted as b11, which is equivalent to the angle bx11 disclosed in Formula 1 above; the angle between the perpendicular line segment passing through the optical center O of the camera and perpendicular to the second side D1T to be measured and the plane Lv1 is denoted as b21, which is equivalent to the angle bx21 disclosed in Formula 1 above; the robot sets the angle between the plane Lr1 of the object to be measured and the imaging plane Lp (or the pinhole plane Lf) of the camera as the tilt angle of the plane of the object to be measured, where a1 is the tilt angle of the plane of the object to be measured, which is equivalent to the angle ax disclosed in Formula 1 above.

[0117] The deflection angle of the triangular tag relative to the camera includes the angle b12 between the perpendicular segment from the camera to the side to be measured where the first detection point is located and the optical axis, and the angle b22 between the perpendicular segment from the camera to the side to be measured where the second detection point is located and the optical axis. Specifically, the angle b12, which includes the angle between 90 degrees and b11, is equivalent to the angle bx12 disclosed in Formula 1. Similarly, the angle b22, which includes the angle between 90 degrees and b21, is equivalent to the angle bx22 disclosed in Formula 1.

[0118] exist Figure 4 In Formula 2 disclosed in the illustrated embodiment, distances d1, d2, and U1 are known quantities, while angles b11, b21, b12, b22, and a1 are unknown quantities. The equations constructed from Formula 2 can be used to obtain angles b11, b21, b12, b22, and a1. Specifically, the distance d1 between the first measured side C1T and the optical center O of the camera can be considered as constructing a pinhole imaging model from a side-view perspective between the measured object plane Lr1, the camera's pinhole plane Lf, and the camera's imaging plane Lp (actually, it is calculated using the geometric relationships of similar triangles from the side-view perspectives of each plane; see details in [reference needed]). Figure 4 Using the similar triangle relationship constructed on the right, the length of the perpendicular segment from the camera to the side to be tested, where the first detection point C1 is located, is calculated. Then, the distance d1 from the first side to be tested to the camera is obtained from the top view. Similarly, the distance d2 between the second side to be tested D1T and the optical center O of the camera can be considered as constructing a pinhole imaging model from the side view between the plane Lr1 of the object to be tested, the pinhole plane Lf of the camera, and the imaging plane Lp of the camera (actually, it is calculated from the side view of each plane using the geometric relationship of similar triangles; see details). Figure 4 (Construct similar triangles on the right), calculate the length of the perpendicular segment from the camera to the side to be tested where the second detection point D1 is located, and then obtain the distance d2 from the second side to be tested to the camera from the top view.

[0119] As an implementation method for calculating the position information of the rectangular label relative to the camera, see [link to relevant documentation]. Figure 5 It can be seen that the label pattern recognized by the robot is Figure 5 The rectangle C2D2FE is located on a plane Lr2 that is not parallel to the camera's imaging plane Lp. In step B, the method for calculating the distance between the rectangular tag and the camera, and the deflection angle of the rectangular tag relative to the camera, using the monocular ranging principle, includes: combining... Figure 5It can be seen that the robot sets two vertices C2 and D2, distributed along the target line direction in a rectangle C2D2FE, as two target detection points in the plane Lr2 of the object to be measured. The distance between target detection point C2 and target detection point D2 is denoted as U2, which is the length of the side C2D2 of the rectangle. U2 is equal to Ux disclosed in Formula 1 above. The robot designates the target detection point C2 on the left as the first detection point C2, and the distance from the side to be measured where the first detection point C2 is located to the location O of the camera (denoted as the optical center of the camera) is denoted as d3. d3 is equal to dx1 disclosed in Formula 1 above, which is obtained in advance using the pinhole imaging model. The robot designates the target detection point D2 on the right as the second detection point D2, and the distance from the side to be measured where the second detection point D2 is located to the location O of the camera (denoted as the optical center of the camera) is denoted as d4. d4 is equal to dx2 disclosed in Formula 1 above, which is obtained in advance using the pinhole imaging model.

[0120] The line connecting two vertices of a rectangular label, perpendicular to the target straight line, is denoted as the first side to be measured of the rectangular label, corresponding to... Figure 5 The side C2E of the rectangle is taken as the side to be tested where the first detection point C2 is located. From a side view, it is preferably parallel to the pinhole plane Lf of the camera. Similarly, the line connecting two vertices of the same rectangular label distributed perpendicular to the target line direction is denoted as the second side to be tested of that rectangular label, corresponding to... Figure 5 The side D2F of the rectangle is used as the side to be measured where the second detection point D2 is located. From the side view, it is preferably parallel to the pinhole plane Lf of the camera. It should be noted that the incident light from the first detection point C2 and the incident light from the second detection point D2 intersect at the optical center O of the camera. The optical center O of the camera is located in the pinhole plane Lf of the camera, and the distance from the optical center O of the camera to the imaging plane Lp of the camera is equal to the focal length f of the camera lens.

[0121] Based on the side-angle relationships of a triangle (obtained using the theorem of similar triangles), the deflection angle of the rectangular label C2D2FE relative to the camera is calculated using the following formula:

[0122]

[0123] d4*sin(b41)=U2*sin(a2)+d3*sin(b31);

[0124] U2*cos(a2)=d4*cos(b41)+d3*cos(b31);

[0125] b32 = 90 - b31;

[0126] b42 = 90 - b41;

[0127] exist Figure 5 In the diagram, the plane Lv3 passing through the first detection point C2 (actually a virtual plane passing through the first side to be tested C2E and parallel to the pinhole plane Lf of the camera), the plane Lv4 passing through the second detection point D2 (actually a virtual plane passing through the second side to be tested D2F and parallel to the pinhole plane Lf of the camera), the pinhole plane Lf, and the imaging plane Lp are parallel to each other. The focal length of the camera remains unchanged relative to any of the aforementioned embodiments. The robot sets the angle between the plane Lr2 of the object to be tested, where the target detection points C2 and D2 are located, and the pinhole plane Lf (or imaging plane Lp) of the camera as the tilt angle a2 of the plane of the object to be tested. a2 is equivalent to ax disclosed in the aforementioned formula and is an angle value that urgently needs to be calculated and obtained.

[0128] The angle between the perpendicular line segment passing through the optical center O of the camera and perpendicular to the first side to be measured C2E and the plane Lv3 is denoted as b31, which is equivalent to the angle bx11 disclosed in Formula 1 above; the angle between the perpendicular line segment passing through the optical center O of the camera and perpendicular to the second side to be measured D2F and the plane Lv3 is denoted as b41, which is equivalent to the angle bx21 disclosed in Formula 1 above; the robot sets the angle between the plane Lr2 of the object to be measured and the imaging plane Lp (or the pinhole plane Lf) of the camera as the tilt angle of the plane of the object to be measured, where a2 is the tilt angle of the plane of the object to be measured, which is equivalent to the angle ax disclosed in Formula 1 above.

[0129] The deflection angle of the rectangular label relative to the camera includes the angle b32 between the perpendicular segment of the camera to the side to be measured (located at the first detection point C2) and the optical axis, and the angle b42 between the perpendicular segment of the camera to the side to be measured (located at the second detection point D2) and the optical axis. Specifically, the angle b32, which includes the angle between 90 degrees and b31, is equivalent to the angle bx12 disclosed in Formula 1. Similarly, the angle b42, which includes the angle between 90 degrees and b41, is equivalent to the angle bx22 disclosed in Formula 1.

[0130] exist Figure 5In Formula 3 disclosed in the illustrated embodiment, distances d3, d4, and U2 are known quantities, while angles b31, b41, b32, b42, and a2 are unknown quantities. The equations constructed from Formula 3 can be used to obtain angles b31, b41, b32, b42, and a2. Specifically, the distance d3 between the first measured side C2E and the optical center O of the camera can be considered as constructing a pinhole imaging model from a side-view perspective between the measured object plane Lr2, the camera's pinhole plane Lf, and the camera's imaging plane Lp (actually, it is calculated using the geometric relationships of similar triangles from the side-view perspectives of each plane; see details in [reference needed]). Figure 5 Using the similar triangle relationship constructed on the right, the length of the perpendicular segment from the camera to the side to be tested, where the first detection point C2 is located, is calculated. Then, the distance d3 from the first side to be tested, C2E, to the camera is obtained from the top view. Similarly, the distance d3 between the second side to be tested, D2F, and the optical center O of the camera can be considered as constructing a pinhole imaging model from the side view between the plane Lr2 of the object to be tested, the pinhole plane Lf of the camera, and the imaging plane Lp of the camera (actually, it is calculated from the side view of each plane using the geometric relationship of similar triangles; see details). Figure 5 Using the similar triangle relationship constructed on the right, the length of the perpendicular segment from the camera to the side to be tested where the second detection point D2 is located is calculated. Then, the distance d4 from the second side to be tested to the camera is obtained from the top view. Thus, using two mutually perpendicular viewpoints (top view and side view), Formula 3 disclosed in this embodiment is constructed for the target detection point and the side it is located in the rectangular label. Trigonometric calculations are performed in this way to obtain the angle while avoiding the influence of camera distortion, thereby improving the accuracy of the deflection angle of the rectangular label relative to the camera. Furthermore, when the deflection angles of all the rectangular labels required to form a target graphic label relative to the camera are calculated in sequence, the deflection angle of the target graphic label relative to the camera can be obtained. Since all the rectangular labels required to form a target graphic label are parallel to each other or on the same plane, and the graphic attributes, shape and size of each rectangular label are the same, it is only necessary to calculate the angle between one of the rectangular labels and the imaging plane Lp (or pinhole plane Lf) of the camera, which can be used as the angle between the target graphic label and the imaging plane Lp (or pinhole plane Lf) of the camera.

[0131] As one embodiment, in the label pattern, the robot records the edge that forms a certain angle with the target straight line as the edge to be measured in the label pattern. Specifically, it can be divided into the edge to be measured where the first detection point is located and the edge to be measured where the second detection point is located. The angle between the edge to be measured and the target straight line is not 0, and the edge to be measured is not the edge connecting the first detection point and the second detection point. Formulas one to three in the aforementioned embodiments all require the distance from the edge to be measured where the first detection point is located to the camera, and the distance from the edge to be measured where the second detection point is located to the camera. These all belong to the distance between the label pattern and the camera mentioned in step B. It can be understood that the distance between the label pattern and the camera mentioned in step B includes the distance from the edge to be measured where the first detection point is located to the camera and the distance from the edge to be measured where the second detection point is located to the camera. The monocular ranging principle mentioned in step B includes the pinhole imaging model. Therefore, without considering distortion errors, the method by which the robot can calculate the distance between the label pattern and the camera using a pinhole imaging model includes: obtaining in advance the lens focal length f of the camera (a pre-set fixed lens parameter), the side length w of the side to be measured (as the object height of the pinhole imaging model), and the pixel width p formed by the side to be measured in the imaging plane of the camera (the actual image height that changes with the side length w of the side to be measured); if the intersection line of the plane of the object to be measured and the pinhole plane of the camera is set perpendicular to the target straight line direction, the side to be measured perpendicular to the target straight line direction will not be distorted in the camera, wherein the plane of the object to be measured and the pinhole plane of the camera are not parallel, and the side of the label pattern that is parallel to the target straight line direction will be distorted in the camera.

[0132] Then, the distance between the relevant edge to be measured and the camera is calculated using the following formula:

[0133]

[0134] Based on the aforementioned embodiments, if one endpoint of the edge to be tested is the first detection point, then the edge to be tested is the edge to be tested where the first detection point is located. Then, d is set to be equal to the distance dx1 from the edge to be tested where the first detection point is located to the camera. The first detection point belongs to the target detection point. Specifically, the case where one endpoint of the edge to be tested is the first detection point is expressed as follows: if the intersection of the line connecting the two adjacent target detection points and the edge to be tested is the first detection point, then the edge to be tested is the first edge to be tested. w represents the side length of the first edge to be tested. Substituting into Formula 4, the distance dx1 from the edge to be tested where the first detection point is located to the camera can be obtained. Similarly, if one endpoint of the edge to be tested is the second detection point, then the edge to be tested is the edge to be tested where the second detection point is located. Then, d is set to be equal to the distance dx2 from the edge to be tested where the second detection point is located to the camera. The second detection point is also a target detection point. Specifically, the case where one endpoint of the edge to be tested is the second detection point is expressed as follows: if the intersection of the line connecting the two adjacent target detection points mentioned above and the edge to be tested is the second detection point, then the edge to be tested is the second edge to be tested. w represents the side length of the second edge to be tested. Substituting into Formula 4, the distance dx2 from the edge to be tested where the second detection point is located to the camera can be obtained.

[0135] It should be noted that the lens focal length f and the side length w of the side to be measured can be obtained through mechanical design, or by simple ruler measurement, or by querying the lens's factory parameters (including radius of curvature, distance, refractive index, etc.). The pixel width p formed by the side to be measured in the imaging plane of the camera can be obtained in the coordinate system of the imaging plane of the camera.

[0136] It is worth noting that when the plane of the object under test is not parallel to the pinhole plane of the camera, dx1 is not equal to dx2, and it is even less equal to the distance from the camera to the plane of the object under test when the optical axis of the camera is perpendicular to the plane of the object under test (which is directly calculated from the pinhole imaging model). Therefore, the tilt angle between the plane of the object under test and the pinhole plane of the camera will produce a ranging error. It is necessary to perform the aforementioned steps A to D to deal with the impact of the ranging error on the robot's positioning and alignment with the charging interface until it is determined that the distance between the currently identified target graphic label and the camera, and the deflection angle of the currently identified target graphic label relative to the camera, meet the preset positioning conditions.

[0137] As an implementation method for calculating the distance between the measured side of a triangular label and the camera, when the label pattern is represented as a triangular label, the two adjacent target detection points are the two vertices of the base of the triangular label. Each triangular label's corresponding target detection points are distributed along the target straight line in the plane of the object being measured. The line connecting two vertices of a triangular label that form a first angle with the target straight line is denoted as the measured side containing the first detection point, and is called the first measured side. The line connecting two vertices of the same triangular label that form a second angle with the target straight line is denoted as the measured side containing the second detection point, and is called the second measured side. The sum of the second and first angles is equal to 180 degrees. Thus, based on the two adjacent target detection points already identified by the robot, the measured sides containing the first and second detection points are extracted within the same triangular label. The lengths of the side to be tested where the first detection point is located and the lengths of the side to be tested where the second detection point is located are both preset to be less than the preset distortion error value. Preferably, since the distance between two adjacent target detection points obtained by the robot is so small that it can be ignored, it can also be regarded as the ranging error caused by the distortion of the line connecting the two adjacent target detection points falling within the preset target error range. In fact, the distance error is caused by the change in the angle between the line connecting the two adjacent target detection points and the pinhole plane of the camera. Therefore, in this embodiment, the distance between two adjacent target detection points set on the surface of the object to be tested (specifically the surface of the charging base) by reducing the label pattern is initially reduced to reduce the ranging error caused by distortion, thereby achieving the effect of initial correction of angle distortion.

[0138] See Figure 4 It can be seen that the label pattern recognized by the robot is Figure 4 When the robot selects triangle TC1D1, it sets two vertices C1 and D1, distributed along the target line, as two target detection points in the plane Lr1 of the object to be measured. The robot designates the left-leaning target detection point C1 as the first detection point C1. Ignoring distortion of the side to be measured containing the first detection point C1 in the camera, the robot designates the distance from the side to be measured containing the first detection point C1 to the camera's location O (denoted as the optical center of the camera) as d1, where d1 can be replaced by d in the aforementioned formula four. Ignoring distortion of the side to be measured containing the second detection point D1 in the camera, the robot designates the right-leaning target detection point D1 as the second detection point D1, and the distance from the side to be measured containing the second detection point D1 to the camera's location O (denoted as the optical center of the camera) as d2, where d2 can be replaced by d in the aforementioned formula four. The line connecting two vertices of a triangular label distributed along a first angle with the target line is designated as the first side to be measured of the triangular label, corresponding to... Figure 4Let side C1T of the triangle be the side to be tested, where the first detection point C1 is located. Figure 4 In the formula, the side length of the first side C1T to be tested is represented by w11, which is equivalent to w in Formula 4; the pixel width formed by the first side C1T in the imaging plane of the camera is represented by p11, which is equivalent to p in Formula 4. Similarly, the line connecting two vertices of the same triangular label that are distributed along the second angle with the direction of the target line is denoted as the second side to be tested of the triangular label, corresponding to... Figure 4 The side D1T of the triangle is used as the side to be tested where the second detection point D1 is located. The side length of the second side to be tested D1T is represented by w12, which is equivalent to w in Formula 4. The pixel width formed by the second side to be tested D1T in the imaging plane of the camera is represented by p12, which is equivalent to p in Formula 4.

[0139] exist Figure 4 In the schematic diagram from the side view on the right, the optical center O of the camera is located at the pinhole plane Lf of the camera, and the distance from the optical center O of the camera to the imaging plane Lp of the camera is equal to the fixed lens focal length f.

[0140] Based on the similar triangle theorem, the distance from the side to be tested where the first detection point C1 is located to the location O of the camera (denoted as the optical center of the camera) can be obtained using the following formula:

[0141]

[0142] Therefore, ignoring the distortion of the first and second sides in the camera, the distance d1 between the first side C1T and the optical center O of the camera can be considered as constructing a pinhole imaging model from the side view perspective between the object plane Lr1, the pinhole plane Lf, and the imaging plane Lp of the camera (actually, it is calculated from the side view perspective of each plane using the geometric relationship of similar triangles; see details). Figure 4 Using the similar triangle relationship constructed on the right, the length of the perpendicular segment from the camera to the side to be tested where the first detection point C1 is located is calculated, and then the distance d1 from the first side to be tested to the camera is obtained from the top view. Similarly, the distance d2 between the second side to be tested D1T and the optical center O of the camera can be regarded as constructing a pinhole imaging model between the plane Lr1 of the object to be tested, the pinhole plane Lf of the camera, and the imaging plane Lp of the camera from the side view (actually, it is calculated from the side view of each plane using the geometric relationship of similar triangles), the length of the perpendicular segment from the camera to the side to be tested where the second detection point D1 is located is calculated, and then the distance d2 from the second side to be tested to the camera is obtained from the top view.

[0143] It is worth noting that when the plane of the object under test and the pinhole plane of the camera are considered parallel, the distance from the camera to the first measured side of the label pattern is equal to the distance from the camera to the second measured side of the same label pattern. In this case, the label pattern does not produce distortion in the camera; in fact, combined with Figure 4 It can be seen that the plane Lr1 of the object to be measured is not parallel to the pinhole plane Lf of the camera. Among the two adjacent target detection points C1 and D1 distributed in a triangular label, the distance d1 from the camera to the first side C1T to be measured is not equal to the distance d2 from the camera to the second side D1T to be measured, thus generating a distance difference, which corresponds to a distance measurement error in one direction caused by the distortion of the label pattern.

[0144] As an implementation method for calculating the distance between the measured side of a rectangular label and the camera, when the label pattern is represented as a rectangular label, the two adjacent target detection points are two vertices of the side of the rectangular label parallel to the target line direction. The two sides of the rectangular label perpendicular to the target line direction are the measured sides where the first detection point and the second detection point are located, respectively. The measured side where the first detection point is located is marked as the first measured side, and the measured side where the second detection point is located is marked as the second measured side. Both the measured sides where the first and second detection points are located are parallel to the pinhole plane of the camera, thus avoiding distortion in the camera. Therefore, effective measured sides are extracted within the same rectangular label based on the two adjacent target detection points already identified by the robot.

[0145] See Figure 5 It can be seen that the label pattern recognized by the robot is Figure 5 When the rectangle C2D2FE is defined, the robot sets two vertices C2 and D2, distributed along the target line direction, as two target detection points in the plane Lr2 of the object to be measured. The robot designates the left-leaning target detection point C2 as the first detection point C2, and the distance from the edge to be measured containing the first detection point C2 to the camera's location O (denoted as the optical center of the camera) as d3, where d3 can be replaced by d in the aforementioned formula four. The robot designates the right-leaning target detection point D2 as the second detection point D2, and the distance from the edge to be measured containing the second detection point D2 to the camera's location O (denoted as the optical center of the camera) as d4, where d4 can be replaced by d in the aforementioned formula four. The line connecting two vertices of a rectangular label distributed along a first angle with the target line direction is designated as the first edge to be measured of the rectangular label, corresponding to... Figure 5 The side C2E of the rectangle is taken as the side to be tested where the first detection point C2 is located. Figure 5In the formula, the side length of the first side C2E to be tested is represented by w21, which is equivalent to w in Formula 4; the pixel width formed by the first side C2E to be tested in the imaging plane of the camera is represented by p21, which is equivalent to p in Formula 4. Similarly, the line connecting two vertices of the same triangular label that are distributed along the second angle with the direction of the target line is denoted as the second side to be tested of the triangular label, corresponding to... Figure 5 The side D2F of the triangle is used as the side to be tested where the second detection point D2 is located. The side length of the second side to be tested D2F is represented by w22, which is equivalent to w in Formula 4. The pixel width formed by the second side to be tested D2F in the imaging plane of the camera is represented by p22, which is equivalent to p in Formula 4.

[0146] exist Figure 5 In the schematic diagram from the side view on the right, the optical center O of the camera is located at the pinhole plane Lf of the camera, and the distance from the optical center O of the camera to the imaging plane Lp of the camera is equal to the fixed lens focal length f.

[0147] Based on the similar triangle theorem, the distance from the side to be tested where the first detection point C2 is located to the location O of the camera (denoted as the optical center of the camera) can be obtained using the following formula:

[0148]

[0149] Therefore, the distance d3 between the first side C2E to be measured and the optical center O of the camera can be considered as constructing a pinhole imaging model from the side view perspective between the plane Lr2 of the object to be measured, the pinhole plane Lf of the camera, and the imaging plane Lp of the camera (actually, it is calculated from the side view perspective of each plane using the geometric relationship of similar triangles; see details). Figure 5 Using the similar triangle relationship constructed on the right, the length of the perpendicular segment from the camera to the side to be tested where the first detection point C2 is located is calculated, and then the distance d3 from the first side to be tested to the camera is obtained from the top view. Similarly, the distance d4 between the second side to be tested D2F and the optical center O of the camera can be considered as constructing a pinhole imaging model between the plane Lr2 of the object to be tested, the pinhole plane Lf of the camera, and the imaging plane Lp of the camera from the side view (actually, the calculation is performed using the geometric relationship of similar triangles from the side view of each plane), calculating the length of the perpendicular segment from the camera to the side to be tested where the second detection point D2 is located, and then obtaining the distance d4 from the second side to be tested to the camera from the top view. It is worth noting that when the plane of the object to be tested is parallel to the pinhole plane of the camera, the distance from the camera to the first side to be tested of the label pattern is equal to the distance from the camera to the second side to be tested of the same label pattern. In this case, the label pattern does not produce distortion in the camera; in fact, combined with Figure 5It can be seen that the plane Lr2 of the object to be measured is not parallel to the pinhole plane Lf of the camera. Among the two adjacent target detection points C2 and D2 distributed in a rectangular label, the distance d3 from the camera to the first side to be measured C2E is not equal to the distance d4 from the camera to the second side to be measured D2F, thus generating a distance difference, which corresponds to the distance measurement error caused by the distortion of the label pattern.

[0150] In summary, this embodiment can be considered as constructing a pinhole imaging model between the plane of the object under test, the pinhole plane of the camera, and the imaging plane of the camera from the perspective of a side view (in fact, it uses the geometric relationship of similar triangles from the side view of each plane (preferably the direction perpendicular to the horizontal ground)). The length of the perpendicular line segment from the camera to the side of the object under test where the first detection point or the second detection point is located is calculated. Under the premise of ignoring the distortion caused by the vertical or horizontal direction, and assuming that the plane of the object under test, the pinhole plane of the camera, and the imaging plane of the camera are parallel to each other from the side view, the calculation of the distance between the label pattern and the camera is saved.

[0151] In the scenario where the robot performs steps A through D, such as... Figure 6 It can be seen that in the plane Lr3 of the object under test (the line segment representing the surface of the charging dock's setting label pattern in the top view), the rectangular label with vertices C2 and D2 is set between the two triangular labels, where, Figure 6 The two vertices C2 and D2, distributed along the target straight line, represent two adjacent target detection points of the rectangular label. This rectangular label can be located at the center of the target graphic label, serving as a position point for the robot's final positioning and alignment / docking. It should be noted that the vertices T1, T2, and T3 of the triangle label are located sequentially to the left of the rectangular label, and the vertices T4, T5, and T6 of the triangle label are located sequentially to the right of the rectangular label. These are all considered as the intersection points of the first and second sides to be tested of each triangle label. Figure 6 A triangle label is represented by one vertex, and vertices T1, T2, T3, C2, D2, T4, T5, and T6 are distributed sequentially along the target straight line in the plane Lr3 of the object to be measured.

[0152] As one embodiment, in step D, the robot selects the region formed by the angles formed by the perpendicular segments of the perpendicular lines from the camera to the sides of the identified triangular labels with different placement patterns on both sides of the unidentified image area (obtained by combining the top and side views of the same plane). Then, the robot sets the predicted position point within the region formed by the selected angle with the smallest angle, so that the robot can determine in step C at the predicted position point that the distance between the two triangular labels with different placement patterns that are located on both sides of the middle position and are closest to the camera is the two sets of distances with the smallest values ​​among the distances between the identified triangular labels and the camera. The distance between the camera and the two triangular tags, which are located on opposite sides of the center position and have different placement patterns, includes: the distance from the camera to the side to be measured of the first detection point of the triangular tag on the side of the center position along the target straight line, and the distance from the camera to the side to be measured of the second detection point of the triangular tag on the opposite side of the target straight line; wherein, the distance between a triangular tag and the camera includes the distance from the camera to the side to be measured of the first detection point of the triangular tag and the distance from the camera to the side to be measured of the second detection point of the triangular tag, which constitute a set of distances for a triangular tag.

[0153] In essence, this can be understood as distinguishing the triangular labels located on either side of the target graphic label by their placement. This involves selecting the test sides of the identified triangular labels from both sides of an unrecognized image area (occupied by the target graphic label), then obtaining the angles formed by the perpendicular segments from the optical center of the same camera to the test sides of the selected triangular labels on both sides. The region with the smallest angle is then selected as the candidate region for the predicted location point. The robot can recognize triangular labels with a larger coverage area but cannot recognize rectangular labels with a smaller coverage area when it is far from the charging dock or charging port. Therefore, the robot recognizes the triangular label first when performing step A for the first time. The positioning graphic label was described, but no target graphic label composed of multiple rectangular labels was identified. Therefore, the rectangular label required for the area formed by the smallest included angle could not be identified when the robot first executed step A. However, during subsequent repeated executions of steps A to D, the robot would explore and approach the position of this rectangular label, preferably located at the center of all identified triangular labels (which can also be understood as the axis of symmetry), specifically at the center of the target graphic label. Thus, the area formed by the perpendicular lines from the camera's optical center to the perpendicular lines from the closest triangular labels of different arrangements on both sides of the rectangular label is the area formed by the aforementioned smallest included angle. Therefore, each time step D is executed, the robot updates the included angle with the smallest angle selected, thereby updating the predicted position point and guiding the robot to move from the identified triangular labels on both sides towards the unidentified area in the middle.

[0154] Because in Figure 6 A triangle label is represented by one of the vertices, so in Figure 6 In this diagram, the line connecting the vertex to the optical center of the camera represents the perpendicular line segment from the camera to the edge to be measured where that vertex is located. Specifically, the perpendicular line segments from the optical center O of the camera to the edges to be measured of the nearest triangle labels on either side of the rectangular label are line segments OT3 and OT4, respectively, corresponding to... Figure 2 The leftmost triangle label (with one vertex at the top and two vertices at the bottom) is the closest to the mid-distance rectangular label. Figure 2 The rightmost inverted triangle label (one vertex at the bottom, two vertices at the top) is the closest to the mid-distance rectangular label. Having sequentially identified the triangle labels represented by vertices T1, T2, T3, T4, T5, and T6, the area formed by the angle with the smallest included angle can be set as... Figure 6In the angle region formed by the midline segments OT3 and OT4, the robot determines through step C that the distance between the triangle label corresponding to T3 and the camera and the distance between the triangle label corresponding to T4 and the camera are the two sets of distances with the smallest values ​​among the distances between vertices T1, T2, T3, T4, T5, and T6 and the camera. The distance between the triangle label corresponding to each vertex and the camera is recorded as the corresponding set of distances.

[0155] The perpendicular bisector of line segment C2D2, or the perpendicular line of the rectangle label it contains, is the axis of symmetry between the left vertices (including T1, T2, T3) and the right vertices (T4, T5, T6) of line segment C2D2. The robot then sets the predicted position point within the area formed by the smallest angle selected so far, corresponding to... Figure 6 In this process, the direction in which the predicted location point is set can be from the vertices of the triangular labels representing the identified triangles distributed on both sides of line segment C2D2 to one endpoint of line segment C2D2. This can be done when the distance between the positioning graphic label and the camera satisfies the preliminary positioning conditions disclosed in the aforementioned embodiments. Figure 6 Within the angle region formed by line segments OT3 and OT4; specifically, from a top-down perspective, the robot successively zooms in on the predicted position point from the angle region formed by line segment OT1 and other line segments, the angle region formed by line segment OT2 and other line segments, and the angle region formed by line segment OT3 and other line segments, until... Figure 6 Within the angle region formed by line segments OT3 and OT4, or by successively narrowing the predicted position point from the angle region formed by line segment OT6 and other line segments, the angle region formed by line segment OT5 and other line segments, and the angle region formed by line segment OT4 and other line segments, to... Figure 6 Within the angle region formed by line segments OT3 and OT4, during the repeated execution of steps A to D, the robot is guided to move from the identified positioning graphic tags on both sides towards the unidentified image region in the middle, so that the predicted position point is set at the center of symmetry between all the identified triangular tags, thereby reducing the distance between the robot and the perpendicular line of the plane containing the rectangular tag C2D2. Here, the predicted position point is only limited to the region formed by the smallest angle currently selected, without limiting the specific position point. It can be located on the side of the angle or in other angle regions with smaller angles, continuously providing the robot with flexible image acquisition positions.

[0156] Preferably, when the robot does not recognize the rectangular label, the robot calculates that the distance from the camera to the side to be measured of the rectangular label is within the range defined by the distance between the camera and the two triangular labels placed at opposite sides of the central position with different arrangements. Specifically, among the two triangular labels, the sides to be measured of the two triangular labels are the second side to be measured of the triangular label closest to the left of the rectangular label and the first side to be measured of the triangular label closest to the right of the rectangular label. The vertical line segments from the optical center O of the camera to the sides to be measured of the two triangular labels closest to the rectangular label are line segments OT3 and OT4, respectively. From a top-down perspective, the lower limit and upper limit of the distance range defined by the distance from the camera to the sides to be measured of the two triangular labels are represented by the lengths of line segments OT3 and OT4, respectively. Therefore, the distance from the camera to the sides to be measured of the two triangular labels can be set as the upper and lower limits of the distance from the camera to the sides to be measured of the rectangular label. Furthermore, once the robot has identified the rectangular label, it has also identified the two adjacent target detection points C2 and D2 of the rectangular label, as well as the test edges located at the vertices C2 and D2 of the rectangular label from the optical center O of the camera. Therefore, the lengths of the perpendicular segments from the optical center O of the camera to the test edges on both sides of the rectangular label can be calculated. These lengths fall within the distance range defined by the distance from the camera to the test edges of the two triangular labels, further limiting the distance from the camera to the test edges of the rectangular label. In particular, this is refined to the distance information of the rectangular label representing the charging interface relative to the camera, correspondingly forming the distance range of the charging interface represented by the target graphic label relative to the camera.

[0157] In one embodiment, after the robot identifies a rectangular label in the target graphic label, during step D, as the robot moves towards the rectangular label at the center of the target graphic label, if the robot is detected to be to the left of the rectangular label at the center of the target graphic label based on the deflection angle of the currently identified rectangular label relative to the camera, the robot moves to the right to a new predicted position point; or, if the robot is detected to be to the right of the rectangular label at the center of the target graphic label based on the deflection angle of the currently identified rectangular label relative to the camera, the robot moves to the left to a new predicted position point. This continues until, at the latest determined predicted position point, it is determined that the distance between the rectangular label at the center of the target graphic label and the camera is the smallest among the distances between all identified rectangular labels and the camera, and the distance between the currently identified target graphic label and the camera satisfies the preset positioning conditions. This allows the robot to overcome the ranging error interference caused by image distortion, while aligning its movement direction with the charging interface, correcting the robot's recharge direction, reducing errors in the recharge process, and improving recharge efficiency.

[0158] It should be noted that the predicted position point set in each execution of step D can be closer to the charging interface than the previously set predicted position point, that is, the distance between the predicted position point and the charging interface directly below the rectangular label located at the center of the target graphic label (which can be the center of symmetry disclosed in the aforementioned embodiments) is shortened. This continues until the distance between the rectangular label located at the center of the target graphic label and the camera is determined to be the smallest among the distances between all the identified rectangular labels and the camera at the latest determined predicted position point, and it is determined that the distance between the currently identified target graphic label and the camera meets the preset positioning conditions. During this distance shortening process, the tilt angle of the plane where the label pattern (which can also be regarded as the rectangular label) is located can be adjusted to reduce the ranging error caused by the edge distortion of the label pattern newly determined by the robot, or even approach a value of 0.

[0159] The distance between a rectangular label and the camera includes the distance from the camera to the side to be measured where the first detection point of the rectangular label is located, and the distance from the camera to the side to be measured where the second detection point of the rectangular label is located, forming a set of distances for a rectangular label; the distance between the target graphic label and the camera includes the distances between all the rectangular labels and the camera required to form the target graphic label.

[0160] Based on the above embodiments, after the robot determines that the distance between the currently identified target graphic label and the camera meets the preset positioning conditions, the robot adjusts the optical axis of the camera to be perpendicular to the plane of the object to be measured by rotating, so that the robot's movement direction becomes parallel to the vertical direction of the rectangular label at the center position of the target graphic label, thereby determining that the deflection angle of the currently identified target graphic label relative to the camera meets the preset positioning conditions.

[0161] Preferably, during the execution of step D, after the robot moves to the predicted position point determined in the last executed step D, the robot will identify the rectangular label as existing in the unrecognized image area in the subsequent step A. At this time, the difference between the distance between the newly determined label pattern and the camera and the ideal distance (the standard distance between the center position of the target graphic label and the camera) is equal to the preset distance error value, so that the setting of the predicted position point has a corrective effect.

[0162] Preferably, after the robot moves to the predicted position point determined in the last executed step D, the robot can further adjust the optical axis of the camera to be perpendicular to the plane of the object to be measured by rotation. The robot can then set the predicted position point near the perpendicular line of the rectangular label located at the center of the target graphic label, corresponding to... Figure 6 The perpendicular line between the rectangle label containing vertices C2 and D2 is the perpendicular bisector of line segment C2D2 from a top-down view. Moreover, the predicted position point set in each execution step D will be closer to the charging interface than the predicted position point set in the previous execution, along the perpendicular line of the rectangle label located at the center of the target graphic label.

[0163] In summary, this embodiment ensures that the plane of the object under test (understood as the plane where the charging interface is located or the plane where the rectangular label is located) is perpendicular to the optical axis of the camera, so as to ensure the accuracy of subsequent imaging and reduce or even eliminate errors.

[0164] It should be noted that if the robot detects that one edge of the label pattern is not parallel to the pinhole plane of the camera, it confirms that this edge, which is not parallel to the pinhole plane of the camera, causes distortion in the camera. Therefore, when the robot detects that the tilt angle of the plane of the object under test is not equal to 0 degrees and is not an integer multiple of 180 degrees, the robot determines that the label pattern in the plane of the object under test is distorted in the camera. If the robot detects that one edge of the label pattern is parallel to the pinhole plane of the camera, it confirms that this edge, which is parallel to the pinhole plane of the camera, does not cause distortion in the camera, but the label pattern in the plane of the object under test may still cause distortion in the camera. This achieves the detection of whether the label pattern in the plane of the object under test is distorted in the camera. Therefore, in the aforementioned embodiment, when the target straight line direction is the horizontal direction passing through the target graphic label, the edge of the label pattern parallel to the horizontal direction is distorted in the camera, and the edge of the label pattern perpendicular to the target straight line direction is not distorted in the camera, corresponding to... Figure 5 The first side to be measured, C2E, and the second side to be measured, D2F, are used to directly calculate the distance between the first and second sides to be measured and the camera without using distortion parameters or distortion formulas commonly used in existing technologies, thus improving calculation efficiency and accuracy.

[0165] Regarding the distortion phenomenon of the label pattern in the camera, it should be noted that, since the height of the intersection point between the principal ray of the label pattern in the plane of the object under test at different tilt angles and the imaging plane of the camera after passing through a monocular camera with a fixed focal length is inconsistent and is not equal to the ideal image height, distortion will occur. Specifically, among the sides of the rectangular label and the triangular label of the object under test at an angle to the pinhole plane, the side that is relatively closer to the optical center of the camera will have a larger imaging size (e.g., imaging width or imaging height) in the imaging plane, and will be taller than the ideal object due to distortion; the side that is relatively farther from the optical center of the camera will have a smaller imaging size (e.g., imaging width or imaging height) in the same imaging plane, and will be shorter than the ideal object due to distortion.

[0166] As an example of calculating ranging error, when the robot identifies two adjacent target detection points on the plane of the object under test, it calculates the product of the distance Ux between the two adjacent target detection points on the plane of the object under test and the sine of the tilt angle of the plane of the object under test, Ux*sin(ax). Then, Ux*sin(ax) is set as the ranging error caused by the distortion of the line connecting the two adjacent target detection points in the camera. Furthermore, when the distance between the currently identified target graphic label and the camera, and the deflection angle of the currently identified target graphic label relative to the camera, both meet preset positioning conditions, the ranging error caused by the distortion of the line connecting the two adjacent target detection points in the camera falls within a preset target error range, where the preset target error range includes the value 0. The two adjacent target detection points can originate from previously identified positioning graphic labels (triangle labels) or previously identified target graphic labels (rectangular labels). When the robot detects that the tilt angle of the plane of the object under test is not equal to 0 degrees, and the tilt angle of the plane of the object under test is not an integer multiple of 180 degrees, the robot determines that the plane of the object under test is not parallel to the pinhole plane of the camera, and determines that the label pattern in the plane of the object under test is distorted in the camera. Therefore, when the charging base plane where the label pattern is located is not parallel to the imaging plane of the camera, the edge of the label pattern is distorted after being captured by the camera. In this embodiment, the size difference (e.g., the difference between the actual image height and the ideal image height) of the edge line where two adjacent target detection points of the label pattern are located before and after distortion is obtained by multiplying the distance Ux between two adjacent target detection points in the plane of the object under test by the sine of the tilt angle of the plane of the object under test. As described in the foregoing embodiments, after the robot determines that the distance between the currently identified target graphic label and the camera meets the preset positioning conditions, the robot adjusts the optical axis of the camera to be perpendicular to the plane of the object under test by rotation. This changes the robot's movement direction to be parallel to the perpendicular direction of the rectangular label at the center of the target graphic label, thus determining that the deflection angle of the currently identified target graphic label relative to the camera meets the preset positioning conditions. Therefore, when both the distance between the currently identified target graphic label and the camera, and the deflection angle of the currently identified target graphic label relative to the camera, meet the preset positioning conditions, the ranging error caused by the distortion of the line connecting two adjacent target detection points in the camera is equal to 0. This indicates that the robot is aligned with the rectangular label at the center of the target graphic label, signifying that the robot has connected to the charging interface. By utilizing the size difference between the edge lines of the adjacent target detection points of the label pattern before and after distortion, the timing of the robot aligning with the rectangular label at the center of the target graphic label is determined. This links the ranging error caused by distortion with the robot's preset positioning conditions, improving the accuracy of the robot contacting and aligning with the rectangular label at the center of the target graphic label.

[0167] Based on the aforementioned embodiment for calculating ranging error, the robot updates the predicted position point by executing step D and moves to the latest predicted position point. This reduces the ranging error caused by the distortion of the line connecting two adjacent target detection points in the camera, until the distance between the currently identified target graphic label and the camera, and the deflection angle of the currently identified target graphic label relative to the camera, both meet preset positioning conditions. At this point, the ranging error caused by the distortion of the line connecting two adjacent target detection points in the camera equals zero. This suppresses the error caused by the label pattern distortion, improves the accuracy of distance and angle calculations in step B, and thus improves the positioning accuracy of the predicted position point.

[0168] Combination Figure 4 It can be seen that, among two adjacent target detection points C1 and D1, the robot records the plane that passes through the side to be measured where the first detection point C1 is located and is parallel to the pinhole plane Lf of the camera as the first virtual plane Lv1, and the plane that passes through the side to be measured where the second detection point D1 is located and is parallel to the pinhole plane Lf of the camera as the second virtual plane Lv2. Then, the distance between the first virtual plane Lv1 and the second virtual plane Lv2 is set to be equal to the ranging error caused by the distortion of the line connecting the two adjacent target detection points C1 and D1. According to the geometric relationship of triangles, the ranging error is equal to the product of the length U1 of line segment C1D1 and the sine of the tilt angle a1 of the plane where the label pattern is located. When the distance between the first virtual plane Lv1 and the second virtual plane Lv2 is smaller, the ranging error caused by the distortion of the label pattern (including the distortion of the line connecting the two adjacent target detection points C1 and D1) is smaller, and the tilt angle a1 of the plane where the label pattern is located is also smaller.

[0169] Combination Figure 5It can be seen that, among two adjacent target detection points C2 and D2 distributed in a rectangular label, the ranging error caused by the distortion of the line connecting the two adjacent target detection points C2 and D2, according to the geometric relationship of triangles, is equal to the product of the length U2 of line segment C2D2 and the sine of the tilt angle a2 of the plane where the label pattern is located. The tilt angle a2 of the plane where the label pattern is located is represented by the angle between the plane of the object to be measured Lr2 and the first virtual plane Lv3. The first virtual plane Lv3 is a virtual plane that passes through the side C2E where the target detection point C2 is located and is parallel to the pinhole plane Lf of the camera. The second virtual plane Lv4 is a virtual plane that passes through the target... The virtual plane where detection point D2 is located on the side D2F to be measured and is parallel to the pinhole plane Lf of the camera, has a length U2 of line segment C2D2 and the sine of the tilt angle a2 of the plane where the label pattern is located. The product of these two values ​​is equal to the distance between the first virtual plane Lv3 and the second virtual plane Lv4. The smaller the tilt angle a2 of the plane where the label pattern is located, and / or the smaller the length U2 of line segment C2D2 is set, the smaller the distance between the first virtual plane Lv3 and the second virtual plane Lv4 becomes. This results in a smaller ranging error caused by the distortion of the line connecting the two adjacent target detection points C2 and D2, and consequently, a smaller ranging error caused by the distortion of the label pattern.

[0170] As one embodiment, when the plane of the object to be measured is not parallel to the pinhole plane of the camera, if the robot detects that the side of the rectangular label perpendicular to the target line is parallel to the pinhole plane of the camera, the robot determines that the ranging error caused by the distortion of the side of the rectangular label parallel to the target line in the camera is not equal to 0, thus confirming that the side parallel to the target line is distorted in the camera; if the ranging error caused by the distortion of the side to be measured in the rectangular label is equal to 0, then the side to be measured in the rectangular label is not distorted in the camera. Therefore, by calculating the ranging error generated by each side of the rectangular label, it is determined whether distortion has occurred, saving computation and ensuring ranging accuracy. Specifically, the side to be measured in the rectangular label includes the side to be measured where the first detection point is located and the side to be measured where the second detection point is located. Since both the side to be measured where the first detection point and the side to be measured where the second detection point are located are parallel to the pinhole plane of the camera, in step B, when calculating the distances from the camera to the side to be measured where the first and second detection points of the rectangular label are located, the robot determines that the ranging error caused by the distortion of the side to be measured where the first and second detection points of the rectangular label are located in the camera is equal to 0, and the ranging error caused by the distortion of the side to be measured where the second detection point of the rectangular label is located in the camera is also equal to 0. Therefore, the calculation of the distances from the camera to the side to be measured where the first and second detection points of the rectangular label are located in step B can be performed using the pinhole imaging model, saving computational effort and ensuring ranging accuracy.

[0171] Preferably, the visual positioning control method further includes: when the distance between two adjacent target detection points obtained by the robot is less than 0.5*(dx1+dx2), the robot sets the ranging error caused by the distortion of the line connecting the two adjacent target detection points in the camera to be equal to a value of 0. Here, dx1+dx2 represents the sum of the distances dx1 from the first side to be measured to the camera and dx2 from the second side to be measured to the camera in the same label pattern, and 0.5*(dx1+dx2) represents half of this sum. If half of this sum is greater than the length of the line connecting the two adjacent target detection points, i.e., greater than the width of the label pattern, then the distance in the label pattern where the line connecting the two adjacent target detection points is located is so small as to be negligible, and the actual imaging width in the imaging plane may not differ much from the ideal imaging width, thus the distortion generated by the label pattern can also be ignored.

[0172] Alternatively, when the distance between two adjacent target detection points obtained by the robot is less than or equal to 0.5*(dx1+dx2), the smaller the distance between the two adjacent target detection points detected by the robot, the smaller the ranging error caused by the distortion of the line connecting the two adjacent target detection points; when the distance between the two adjacent target detection points obtained by the robot is much less than 0.5*(dx1+dx2), the robot can even ignore the ranging error caused by the distortion of the label pattern.

[0173] In summary, before executing the visual positioning control method, the label pattern on the plane of the object to be measured can be set to be relatively small, thereby minimizing the ranging error caused by distortion when the plane of the object to be measured is not parallel to the imaging plane of the camera, i.e. suppressing the distance error caused by angle changes.

[0174] As one embodiment, if the two vertices of the edge to be tested where the first detection point is located are updated to the two adjacent target detection points, then the edges originally distributed along the target straight line direction in the same triangular label are set as the edge to be tested where one of the updated target detection points is located, and the edge to be tested where the original second detection point is located is set as the edge to be tested where another updated target detection point is located, so that the target straight line direction is updated to be parallel to the edge to be tested where the original first detection point is located; then the distance from the camera to the edge to be tested where one of the updated target detection points is located is set to equal dx1, and the distance from the camera to the edge to be tested where another updated target detection point is located is set to equal dx2; when the distance between the two adjacent target detection points after the update is less than 0.5*(dx1+dx2), the robot sets the ranging error caused by the distortion of the line connecting the two adjacent target detection points in the camera to equal the value 0. Thus, during the execution of the visual positioning control method, the label patterns containing the two adjacent target detection points with a distance less than 0.5*(dx1+dx2) are specifically selected for distance and angle calculation, improving the accuracy of the planar positioning calculation of the object to be tested.

[0175] Combination Figure 4It can be seen that, in a triangular label, the ranging error caused by the distortion of the line connecting two adjacent target detection points C1 and D1 in the camera is equal to the product of the length U1 of line segment C1D1 and the sine of the tilt angle a1 of the plane containing the label pattern, according to the geometric relationship of triangles. The tilt angle a1 of the plane containing the label pattern is represented by the angle between the plane Lr1 of the object to be measured and the first virtual plane Lv1. The first virtual plane Lv1 is the plane passing through the side TC1 of the target detection point C1 and parallel to the pinhole plane Lf of the camera. The first virtual plane Lv1 and the second virtual plane Lv2 are virtual planes that pass through the edge TD1 where the target detection point D1 is located and are parallel to the pinhole plane Lf of the camera. The product of the length U1 of line segment C1D1 and the sine of the tilt angle a1 of the plane where the label pattern is located is also equal to the distance between the first virtual plane Lv1 and the second virtual plane Lv2. The smaller the tilt angle a1 of the plane where the label pattern is located, and / or the smaller the length U1 of line segment C1D1 is set, the smaller the distance between the first virtual plane Lv1 and the second virtual plane Lv2, and the smaller the ranging error caused by the distortion of the label pattern.

[0176] If the two vertices of the edge C1T where the first detection point C1 is located are updated to the two adjacent target detection points, that is, the two adjacent target detection points are updated to the first detection point C1 and the second detection point T, then the edge C1D1 that was originally distributed along the target line direction in the same triangle label is set as the edge to be tested where the first detection point C1 is located, and the edge D1T where the original second detection point D1 is located is set as the edge to be tested where the updated target detection point T is located, so that the target line direction is updated to be parallel to the edge C1T where the original first detection point is located; then the distance from the camera to the edge to be tested where the updated target detection point C1 is located is set to be equal to dx1, and the distance from the camera to the edge to be tested where the updated target detection point T is located is set to be equal to dx2; when the distance between the updated two adjacent target detection points C1 and T is less than 0.5*(dx1+dx2), the robot sets the ranging error caused by the distortion of the line C1T connecting the updated two adjacent target detection points in the camera to be equal to the value 0.

[0177] Similarly, when the updated two adjacent target detection points are D1 and T respectively, and the distance between the two adjacent target detection points D1 and T is less than 0.5*(dx1+dx2), the robot sets the ranging error caused by the distortion of the line D1T connecting the updated two adjacent target detection points in the camera to a value of 0. Therefore, it can be determined that the preset distortion error value disclosed in the aforementioned embodiment is equal to 0.5*(dx1+dx2).

[0178] In summary, during the execution of the aforementioned visual positioning control method, the label patterns containing two adjacent target detection points with a distance of less than 0.5*(dx1+dx2) are specifically selected for distance and angle calculation, thereby improving the accuracy of planar positioning calculation of the object under test.

[0179] It should be noted that the pinhole plane of the camera is parallel to the imaging plane of the camera; the optical center of the camera is located in the pinhole plane, and the distance from the optical center to the imaging plane is equal to the focal length of the camera. Specifically, the incident light from the target detection point corresponding to each triangular label intersects the optical center of the camera; the target detection points corresponding to each triangular label are distributed along a straight line in the plane of the object to be measured, and are identified sequentially during the repeated execution of step A by the robot, and then used in step B to calculate the ranging error caused by the distortion of the label pattern. The ranging error here includes the ranging error caused by the distortion of the line connecting two adjacent target detection points in the camera. Of course, only two adjacent target detection points corresponding to the same label pattern are needed to calculate this ranging error.

[0180] As one embodiment, in step A, the preprocessing of the image captured by the robot's camera includes: first, the robot converts the captured image to grayscale to obtain a binarized image; then, the binarized image is Gaussian smoothed to remove image noise, resulting in a smoothed image; then, edge detection is performed on the smoothed image to obtain a target edge map, preferably using the Canny operator. The robot can extract closed curves from the target edge map, and these closed curves can form the closed shape disclosed in the aforementioned embodiment. The target edge map is then dilated to obtain the preprocessed image, which allows the robot to identify the vertices and graphic attributes of the label pattern from the preprocessed image. Based on the graphic attributes of the label pattern, it is possible to distinguish... Figure 3 The preprocessed positioning graphic labels and target graphic labels in the image. Figure 3 The pre-processed label pattern on the left is a regular polygonal graphic arranged with a pre-processed rectangular label (a black square surrounded by a white closed edge line) as the center of symmetry. This regular polygonal graphic is composed of multiple identical black squares surrounded by white closed edge lines, which is the pre-processed target graphic label. Figure 3The preprocessed label pattern on the right consists of a preprocessed triangular label (a black triangle surrounded by a white closed edge line), i.e., a preprocessed positioning graphic label. Before and after preprocessing, the positional relationship between the target graphic label and each positioning graphic label remains unchanged, and the shape and size of each label pattern change only slightly. The only difference is that the preprocessed edge lines are smoother and sharper, enabling clear imaging on the camera's imaging plane. This overcomes the problems of image blurring and noise interference within a specific lighting field of view. Therefore, a single monocular camera with a fixed focal length can achieve clear imaging, capturing all vertices of the label pattern at once, thereby improving the accuracy of calculating the length of the edges connecting the vertices.

[0181] Preferably, in step A, the method for searching for the vertices of the label pattern in the preprocessed image includes: extracting a closed shape in the preprocessed image so that the closed shape represents the label pattern that the robot needs to search for; wherein the closed shape is generated in the edge detection; then extracting the vertices of the label pattern and the graphic attributes of the label pattern based on the closed shape, and then filtering the extracted vertices of the label pattern and the graphic attributes of the label pattern to suppress noise in the label pattern; then setting the filtered vertices as the vertices of the label pattern searched by the robot in the preprocessed image, and setting the filtered graphic attributes as the graphic attributes of the label pattern searched by the robot in the preprocessed image, so as to distinguish labels of different shapes. In this embodiment, the filtered graphic attributes are... Figure 3 The edges reflected on the white lines are clearly imaged and sharp, enabling the robot to obtain accurate graphic attributes and vertices.

[0182] Preferably, the closed shape is formed by multiple line segments connected end-to-end, or by multiple pixels connected sequentially. Pixels at the corners of the closed shape are identified as vertices. The graphic attributes of the label pattern are the edge features of the closed shape, including the number of sides and vertices. In the closed shape, the robot identifies the line connecting two adjacent vertices as a side forming the closed shape. When the robot searches for the graphic attributes and vertices of the label pattern in the preprocessed image, it distinguishes different shapes of label patterns by detecting the number of sides forming the closed shape. Therefore, in the process of detecting the number of sides of a closed shape in the acquired image, when the robot detects three sides, it identifies the currently detected closed shape as a triangular label and determines the identified location graphic label; when the robot detects four sides, it identifies the currently detected closed shape as a rectangular label, thus distinguishing different shapes of labels based on graphic attributes with less noise interference.

[0183] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A visual positioning control method based on label patterns, characterized in that, Visual positioning control methods include: Step A: The robot preprocesses the images captured by its camera, and then searches for the graphic attributes and vertices of the label pattern in the preprocessed images, wherein the label pattern is set on the surface of the device to be positioned. Step B: Based on the graphic attributes and vertices of the label pattern, calculate the distance between the label pattern and the camera, as well as the deflection angle of the label pattern relative to the camera, using the monocular ranging principle. Step C: Determine whether the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera, meet the preset positioning conditions. If yes, the visual positioning of the device to be positioned is completed, so that the robot can align with or contact the corresponding label pattern in the device to be positioned. Otherwise, proceed to step D. Step D: Based on the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera, the robot moves to the predicted position point, then uses the camera to capture an image of the label pattern, and then executes steps A to C. In step B, the method for calculating the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera, using the monocular ranging principle includes: The robot sets the target straight line direction as the extension direction of the plane of the object under test on the horizontal plane, and sets the target straight line direction to be parallel to the robot's walking plane; The robot sets two vertices of a label pattern distributed along the target straight line as two adjacent target detection points in the plane of the object under test; where the distance Ux between the two adjacent target detection points in the plane of the object under test is obtained in advance; the plane of the object under test represents the surface of the charging dock where the label pattern is located; Among the two adjacent target detection points, the robot records one of the target detection points as the first detection point and uses the pinhole imaging model to calculate the distance dx1 from the edge to be tested where the first detection point is located to the camera; the robot records the other target detection point as the second detection point and uses the pinhole imaging model to calculate the distance dx2 from the edge to be tested where the second detection point is located to the camera. Based on the side-angle relationships of a triangle, the deflection angle of the label pattern relative to the camera is calculated using the following formula: ； ； The angle between the perpendicular segment from the camera to the side to be tested where the first detection point is located and the pinhole plane of the camera in the opposite direction of the target straight line is denoted as bx11, and the angle between the perpendicular segment from the camera to the side to be tested where the second detection point is located and the pinhole plane of the camera in the target straight line is denoted as bx21. The robot sets the angle between the plane of the object to be tested and the pinhole plane of the camera in the target straight line direction as the tilt angle of the plane of the object to be tested, where ax is the tilt angle of the plane of the object to be tested; the tilt angle of the plane where the label pattern is located is the tilt angle of the plane of the object to be tested. The deflection angle of the label pattern relative to the camera includes the angle between the perpendicular segment of the camera to the side to be tested where the first detection point is located and the optical axis, denoted as bx12, and the angle between the perpendicular segment of the camera to the side to be tested where the second detection point is located and the optical axis, denoted as bx22. The distance between the label pattern and the camera mentioned in step B includes the distance from the edge to be measured where the first detection point is located to the camera, and the distance from the edge to be measured where the second detection point is located to the camera; the monocular ranging principle mentioned in step B includes the pinhole imaging model.

2. The visual positioning control method according to claim 1, characterized in that, In step D, the following exists: If all the label patterns involved in step C are positioning graphic labels, and if the distance between the positioning graphic label and the camera, or the deflection angle of the positioning graphic label relative to the camera, does not meet the preset positioning conditions, then the robot sets the predicted position point and moves to the predicted position point based on the distance between the positioning graphic label and the camera, and the deflection angle of the positioning graphic label relative to the camera. The robot updates the predicted position point each time it executes step D. If, in step C, it is determined that the distance between the positioning graphic label and the camera meets the preliminary positioning conditions, after the robot completes step D, it begins to identify the target graphic label at the newly set predicted position point, and then configures the target graphic label as the label pattern in steps B to D. Specifically, determining that the visual positioning of the device to be positioned is completed includes completing the visual positioning of the corresponding label pattern in the device to be positioned; the label pattern is a target graphic label or a positioning graphic label; the corresponding label pattern in the device to be positioned is a target graphic label, which is used to indicate the assembly port in the device to be positioned that is specifically for robot contact.

3. The visual positioning control method according to claim 2, characterized in that, Before recognizing the target graphic label, the robot sequentially traverses each recognized positioning graphic label during steps A to D to guide the robot to move from the recognized positioning graphic labels on both sides to the unrecognized area in the middle. The robot identifies a rectangular label at the center of the target graphic label at the last determined predicted location point, and determines that the distance between the rectangular label and the camera meets the preset positioning conditions, thus confirming the existence of the target graphic label in the unidentified area in the middle, and determining that there are multiple positioning graphic labels distributed on both sides of the target graphic label, which is composed of multiple rectangular labels. Moreover, the robot adjusts its pose to align with the rectangular label, and determines that the deflection angle of the rectangular label relative to the camera meets the preset positioning conditions, so that the last determined predicted location point becomes the location of the charging interface on the robot's walking plane. The device to be positioned is a charging dock, and the target graphic label is used to represent the label of the charging interface of the charging dock, with the vertical direction of its plane indicating the docking direction of the charging interface; the assembly port in the device to be positioned that is specifically for robot contact is the charging interface of the charging dock.

4. The visual positioning control method according to claim 3, characterized in that, The size of a positioning graphic label on the charging dock surface is larger than the size of any rectangular label included within the target graphic label on the charging dock surface, so that: When the robot identifies the positioning graphic label, it does not identify the rectangular label, and therefore does not identify the target graphic label. At this point, the robot is at the first predicted position. Moreover, when the robot recognized the rectangular label, it did not recognize the positioning graphic label. At this point, the robot was at the second predicted position point. The distance between the first predicted location point and the charging interface is greater than the distance between the second predicted location point and the charging interface; the positioning graphic label does not include a rectangular label.

5. The visual positioning control method according to claim 3, characterized in that, In step A, the robot identifies the localized graphic label and / or the target graphic label from multiple label patterns at once based on the graphic attributes of the label pattern, so as to obtain the vertices of the identified label pattern and the graphic attributes of the identified label pattern. Whenever multiple positioning graphic tags are identified, in step B, based on the graphic attributes and vertices of each positioning graphic tag, the distance between each positioning graphic tag and the camera, as well as the deflection angle of each positioning graphic tag relative to the camera, are calculated using the monocular ranging principle. Then, the distance between each identified positioning graphic label and the camera is traversed sequentially. When the robot determines in step C that the distance between the two positioning graphic labels with different placement patterns located on both sides of the middle position and the closest distance is not the two sets of distances with the smallest values ​​among the identified positioning graphic labels and the camera, it is determined that the distance between each currently identified label pattern and the camera does not meet the preset positioning conditions.

6. The visual positioning control method according to claim 5, characterized in that, The robot performs step D to set a predicted position point in front of two positioning graphic labels that are located on both sides of the middle position and have different placement patterns. Then, the robot adjusts its pose and moves to the currently set predicted position point according to the deflection angle of the currently set predicted position point relative to the camera, so as to shorten the distance between the robot and the target graphic label. Then the robot repeats steps A to D until, in step C, it determines that the distance between the two positioning graphic tags, which are located on opposite sides of the middle position and have different placement patterns, and the camera is the two smallest distances among all the identified positioning graphic tags and the camera. At this point, it is determined that the distance between the positioning graphic tag and the camera meets the preliminary positioning conditions, but the distance between each currently identified tag pattern and the camera does not meet the preset positioning conditions. After completing step D, the robot begins to identify the target graphic tag in step A. Before the robot identifies the target graphic label in step A, the distance between each currently identified label pattern and the camera is not allowed to meet the preset positioning conditions.

7. The visual positioning control method according to claim 6, characterized in that, After the robot identifies the target graphic label in step A, it then identifies the individual rectangular labels that make up the target graphic label. Then, in step B, based on the graphic attributes and vertices of each rectangular label, the distance between each rectangular label and the camera, as well as the deflection angle of each rectangular label relative to the camera, are calculated using the monocular ranging principle. Then, iterate through the distances between each identified rectangular label and the camera; When the robot determines in step C that the distance between the rectangular label at the center of the target graphic label and the camera is not the smallest among the distances between the identified rectangular labels and the camera, it determines that the distance between the currently identified target graphic label and the camera does not meet the preset positioning conditions; then in step D, the robot moves towards the rectangular label at the center of the target graphic label, until the distance between the rectangular label at the center of the target graphic label and the camera is the smallest among the distances between the identified rectangular labels and the camera, and determines that the distance between the currently identified target graphic label and the camera meets the preset positioning conditions, wherein the currently moved position is the latest determined predicted position. Then, at the newly determined predicted position point, the robot's movement direction is adjusted to be parallel to the vertical direction of the rectangular label at the center position of the target graphic label, and the distance between the currently identified target graphic label and the camera, as well as the deflection angle of the currently identified target graphic label relative to the camera, are determined to meet the preset positioning conditions.

8. The visual positioning control method according to claim 3, characterized in that, The robot recognizes a positioning graphic tag as being composed of a triangular tag and determines the side length of the triangular tag by its vertex. The arrangement of the positioning graphic tags on one side of the target graphic tag is different from that on the other side. The positioning graphic tags on each side of the target graphic tag are evenly spaced along a straight line on the surface of the charging base. The robot identifies a target graphic label as being composed of multiple identical rectangular labels arranged together, and determines the side length of the rectangular labels by their vertices. A target graphic label is a regular polygonal graphic arranged with a rectangular label as its center. Within the target graphic label, the number of rectangular labels distributed in the two neighboring areas on both sides of its center position is different, so as to distinguish the two sides of the target graphic label.

9. The visual positioning control method according to claim 8, characterized in that, In the target graphic label, three rows of rectangular labels are set. There are three rectangular labels in the row that passes through the center position. Among them, the center position of the target graphic label is filled with a rectangular label that is parallel to the central axis of the charging dock, and the other two rectangular labels are located on both sides of the center position. The row passing through the center position is called the middle row. The row above the middle row has two rectangular labels, which are placed in the same column as the rectangular label at the center position and the rectangular label on one side of it, respectively. The row below the middle row has two rectangular labels, which are placed in the same column as the rectangular label at the center position and the rectangular label on the other side of it, respectively, to indicate the direction directly above, directly below, to the left, and to the right of the target graphic label.

10. The visual positioning control method according to claim 8, characterized in that, Methods for identifying the location graphic label and / or target graphic label from multiple label patterns at once based on the graphic attributes of label patterns include: When the robot searches for the graphic attributes and vertices of the label pattern in the preprocessed image, it detects the number of edges that form a closed shape. The graphic attributes of the label pattern are the edge features of the closed shape, which include the number of edges and vertices that form the closed shape. In the closed shape, the robot identifies the line connecting two adjacent vertices as an edge that forms the closed shape. When the robot detects that there are 3 sides forming a closed shape, it identifies the currently detected closed shape as a triangle label and determines the location of the graphic label. When the robot detects that there are 4 sides forming a closed shape, it identifies the currently detected closed shape as a rectangular label. If the number of rectangular labels detected by the robot in the same frame is the total number of all rectangular labels required to form a target graphic label, and the cumulatively detected rectangular labels are symmetrically set with one of the rectangular labels as the center position, and the number of rectangular labels distributed in the neighborhood on both sides of the center position is different, then a target graphic label is identified.

11. The visual positioning control method according to claim 10, characterized in that, In the label pattern, the side that forms a certain angle with the target straight line direction is recorded as the side to be measured in the label pattern; Methods for calculating the distance between a label pattern and a camera using a pinhole imaging model include: The lens focal length f of the camera, the side length w of the side to be measured, and the pixel width p formed by the side to be measured in the imaging plane of the camera are obtained in advance. The distance between the edge to be measured and the camera is calculated using the following formula: If one endpoint of the edge to be tested is the first detection point, then the edge to be tested is the edge to be tested where the first detection point is located, and then d is set to be equal to the distance dx1 from the camera to the edge to be tested where the first detection point is located; If one endpoint of the edge to be tested is the second detection point, then the edge to be tested is the edge to be tested where the second detection point is located, and then d is set to be equal to the distance dx2 from the camera to the edge to be tested where the second detection point is located; Where the plane of the object to be tested is not parallel to the pinhole plane of the camera, dx1 is not equal to dx2, and the intersection line of the plane of the object to be tested and the pinhole plane of the camera is set perpendicular to the direction of the target line.

12. The visual positioning control method according to claim 11, characterized in that, When the label pattern is represented as a triangular label, the two adjacent target detection points are the two vertices of the base of the triangular label. The target detection points corresponding to each triangular label are distributed along the target straight line in the plane of the object to be measured. The line connecting the two vertices of a triangular label that form a first angle with the target straight line is denoted as the side to be measured where the first detection point is located. The line connecting the two vertices of the same triangular label that form a second angle with the target straight line is denoted as the side to be measured where the second detection point is located. The sum of the second and first angles is equal to 180 degrees. The lengths of the sides to be measured where the first and second detection points are located are both preset to be less than a preset distortion error value. When the label pattern includes a rectangular label, the two adjacent target detection points are two vertices of the side of the rectangular label that is parallel to the target straight line direction. The two sides of the rectangular label that are perpendicular to the target straight line direction are the test side where the first detection point is located and the test side where the second detection point is located, respectively. The test side where the first detection point is located and the test side where the second detection point is located are both parallel to the pinhole plane of the camera.

13. The visual positioning control method according to claim 12, characterized in that, Step D further includes the robot selecting the region formed by the angles formed by the perpendicular segments of the tested sides of the identified triangular labels with different placement patterns on both sides of the unidentified area from the camera, and then setting the predicted position point in the region formed by the selected angle with the smallest angle, so that the robot can determine in step C at the predicted position point that the distance between the camera and the two triangular labels with different placement patterns that are located on both sides of the middle position and are closest to each other is the two sets of distances with the smallest values ​​among the distances between the identified triangular labels and the camera. Each time step D is executed, the angle with the smallest selected angle by the robot is updated so that the predicted position point is updated, thereby guiding the robot to move from the identified triangular labels on both sides to the unidentified area in the middle. The distance between a triangular tag and the camera includes the distance from the camera to the side to be measured where the first detection point of the triangular tag is located, and the distance from the camera to the side to be measured where the second detection point of the triangular tag is located, which constitutes a set of distances for a triangular tag.

14. The visual positioning control method according to claim 12, characterized in that, After the robot identifies the rectangular label in the target graphic label, during the process of moving towards the rectangular label at the center of the target graphic label through step D, if the robot is detected to be to the left of the rectangular label at the center of the target graphic label based on the deflection angle of the currently identified rectangular label relative to the camera, the robot moves to the right to the new predicted position point; or, if the robot is detected to be to the right of the rectangular label at the center of the target graphic label based on the deflection angle of the currently identified rectangular label relative to the camera, the robot moves to the left to the new predicted position point; until at the latest determined predicted position point, it is determined that the distance between the rectangular label at the center of the target graphic label and the camera is the smallest among the distances between all the identified rectangular labels and the camera, and it is determined that the distance between the currently identified target graphic label and the camera meets the preset positioning conditions; The distance between a rectangular label and a camera includes the distance from the camera to the side to be measured where the first detection point of the rectangular label is located, and the distance from the camera to the side to be measured where the second detection point of the rectangular label is located, which constitutes a set of distances for a rectangular label. The distance between the target graphic label and the camera includes the distances between all the rectangular labels required to make up the target graphic label and the camera.

15. The visual positioning control method according to claim 14, characterized in that, After the robot determines that the distance between the currently identified target graphic label and the camera meets the preset positioning conditions, the robot adjusts the optical axis of the camera to be perpendicular to the plane of the object under test by rotating, so that the robot's movement direction becomes parallel to the vertical direction of the rectangular label at the center position of the target graphic label, thereby determining that the deflection angle of the currently identified target graphic label relative to the camera meets the preset positioning conditions.

16. The visual positioning control method according to claim 14, characterized in that, If the robot detects that there is an edge in the label pattern that is not parallel to the pinhole plane of the camera, it confirms that the edge that is not parallel to the pinhole plane of the camera is distorted in the camera, and thus determines that the label pattern is not parallel to the pinhole plane of the camera and that the label pattern is distorted in the camera. If the robot detects that there is an edge in the label pattern that is parallel to the pinhole plane of the camera, it confirms that the edge parallel to the pinhole plane of the camera will not cause distortion in the camera.

17. The visual positioning control method according to claim 16, characterized in that, When the robot identifies two adjacent target detection points on the plane of the object under test, it calculates the product Ux of the distance Ux between the two adjacent target detection points on the plane of the object under test and the sine of the tilt angle of the plane of the object under test. sin(ax), then Ux sin(ax) is set as the ranging error caused by the distortion of the line connecting two adjacent target detection points in the camera.

18. The visual positioning control method according to claim 17, characterized in that, When the plane of the object to be measured is not parallel to the pinhole plane of the camera, and the robot detects that the side to be measured that is perpendicular to the straight line direction of the target in the rectangular label is parallel to the pinhole plane of the camera, the robot determines that the ranging error caused by the distortion of the side of the rectangular label that is parallel to the straight line direction of the target in the camera is not equal to the value of 0, and the ranging error caused by the distortion of the side to be measured in the rectangular label in the camera is equal to the value of 0. The edges to be tested in the rectangle label include the edges where the first detection point is located and the edges where the second detection point is located.

19. The visual positioning control method according to claim 16, characterized in that, The distance between two adjacent target detection points obtained by the robot is less than 0.

5. When (dx1+dx2), the robot sets the ranging error caused by the distortion of the line connecting two adjacent target detection points in the camera to be equal to the value 0.

20. The visual positioning control method according to claim 16, characterized in that, If the two vertices of the edge to be tested where the first detection point is located are updated to the two adjacent target detection points, then the edges originally distributed along the target line direction in the same triangle label are set as the edge to be tested where one of the target detection points is located after the update, and the edge to be tested where the original second detection point is located is set as the edge to be tested where another target detection point is located after the update, so that the target line direction is updated to be parallel to the edge to be tested where the original first detection point is located; then the distance from the camera to the edge to be tested where one of the target detection points is located after the update is set to be equal to dx1, and the distance from the camera to the edge to be tested where another target detection point is located after the update is set to be equal to dx2; When the distance between two adjacent target detection points after the update is less than 0.5 When (dx1+dx2), the robot sets the ranging error caused by the distortion of the line connecting the two adjacent target detection points in the camera to a value of 0, wherein the preset distortion error value is equal to 0.

5. (dx1+dx2).

21. The visual positioning control method according to claim 1, characterized in that, In step A, the robot preprocesses the images captured by its camera in the following ways: The robot first converts the acquired image to grayscale to obtain a binarized image; then it performs Gaussian smoothing on the binarized image to remove image noise, resulting in a smoothed image; then it performs edge detection on the smoothed image to obtain a target edge map; and finally, it performs dilation on the target edge map to obtain the preprocessed image, which allows the robot to identify the vertices of the label pattern and the graphic attributes of the label pattern from the preprocessed image.

22. The visual positioning control method according to claim 21, characterized in that, In step A, the method for searching for the vertices of the label pattern within the preprocessed image includes: Within the preprocessed image, closed shapes are extracted to represent the label pattern that the robot needs to search for; wherein, the closed shapes are generated during the edge detection. Then, based on the closed shape, the vertices and graphic attributes of the label pattern are extracted, and the extracted vertices and graphic attributes of the label pattern are filtered to suppress the noise of the label pattern. Then, the filtered vertices are set as the vertices of the label pattern that the robot searches for in the preprocessed image, and the filtered graphic attributes are set as the graphic attributes of the label pattern that the robot searches for in the preprocessed image to distinguish label patterns of different shapes.

Images