Charging device, charging interface docking method based on tag recognition, and charging system

Abstract

The application discloses a charging device, a charging interface docking method based on label recognition and a charging system. The charging device is provided with a charging interface for charging a robot. A target graphic label and a plurality of direction correction labels are arranged on the surface of the charging device. The target graphic label is used for indicating the label of the charging interface of the charging device. The plurality of direction correction labels are distributed on both sides of the target graphic label to guide the robot to move from the direction correction labels on both sides to the target graphic label. The vertical direction of the plane where the target graphic label is located indicates the orientation of the charging interface, so as to correspond to the direction of the robot docking the charging interface. The charging interface docking method is used for controlling the robot to position and dock the charging interface of the charging device. The charging system comprises the robot and the charging device.

Description

Technical Field

[0001] This application relates to the technical field of visual positioning, and in particular to charging devices, tag-based charging interface docking methods, and charging systems. Background Technology

[0002] Visual ranging is a crucial technology in robotics, with wide applications in visual localization, target tracking, and visual obstacle avoidance. Common visual ranging methods include monocular ranging, which boasts a simple structure and fast computation speed, thus offering promising application prospects.

[0003] Existing monocular ranging technology generally simplifies the monocular vision system into a camera projection model, establishes a ranging model through geometric derivation, obtains the transformation relationship between image coordinates and the world coordinate system, and finally calculates the distance to obstacles through geometric calculations (including the pinhole imaging model and the geometric proportion relationship of similar triangles). However, this approach requires manual alignment of the camera's optical axis, and since the focal length of the monocular camera is fixed, image blurring is likely to occur when the camera captures object images.

[0004] When a robot equipped with a monocular camera searches for a charging port near a charging dock, it will remain in motion in order to locate the charging dock or plan a return route. The robot will collect image information of the charging port of the same charging dock from different angles at different times. The camera is prone to ranging errors, which is not conducive to accurately contacting the charging port for charging. Summary of the Invention

[0005] This application discloses a charging device, a charging interface docking method based on tag identification, and a charging system. The specific technical solutions are as follows:

[0006] The charging device includes a charging interface for the robot. Its surface features a target graphic label and multiple orientation correction labels. The target graphic label identifies the charging interface, while the orientation correction labels, located on either side of the target graphic label, guide the robot's movement from the orientation correction labels towards the target graphic label. The vertical direction of the plane containing the target graphic label indicates the orientation of the charging interface, corresponding to the robot's docking direction. Thus, the charging device, through its surface-mounted orientation correction labels and target graphic label, pre-sets the charging interface's position for the robot. This facilitates accurate docking with the charging interface after being guided or corrected by the other label patterns, overcoming the ranging error of the camera. Especially when the surface of the charging device is not perpendicular to the optical axis of the monocular camera, the robot gradually approaches the charging interface using the guidance provided by the specific arrangement of the target graphic label and orientation correction labels, improving the ranging accuracy of the monocular camera.

[0007] Furthermore, the orientation correction label and the target graphic label are positioned on the vertical surface of the charging device along the target straight line direction; the target straight line direction is set as the extension direction of the surface of the charging device on the horizontal ground, and is parallel to the horizontal ground; when the charging device is placed on the horizontal ground, the vertical surface of the charging device is set at an angle to the horizontal ground. This facilitates docking with the charging electrodes on the side of the robot body.

[0008] Furthermore, the arrangement of the orientation correction tags on one side of the target graphic label differs from that on the other side; the orientation correction tags on each side of the target graphic label are evenly spaced on the surface of the charging device; the charging interface is located below the target graphic label to accommodate the height of the robot's charging electrodes. Therefore, the robot can systematically identify each orientation correction tag along a straight line, distinguishing those with the same arrangement, and thus determining the position of each orientation correction tag on the surface of the charging device.

[0009] Furthermore, a orientation correction label consists of a triangular label, with two vertices of the triangular label forming one side, and the number of sides forming the triangular label is 3. A target graphic label consists of multiple identical rectangular labels, with adjacent vertices of the rectangular labels forming a vertical edge line in the vertical direction and adjacent vertices of the rectangular labels forming a horizontal edge line in the horizontal direction. Both the vertical and horizontal edge lines of the rectangular labels are considered edges of the rectangular labels, and the number of sides forming the rectangular label is 4. The target straight line direction is parallel to the base of the triangular label and parallel to the horizontal edge line of the rectangular label. This eliminates the diagonals of the rectangular labels and corrects the robot's pose in both vertical and horizontal directions. The specifically arranged rectangular labels guide the robot closer to the charging interface, thereby achieving contact between the robot and the charging interface at a specific location.

[0010] Furthermore, a target graphic label is a regular polygonal shape arranged around a rectangular label as its center of symmetry. Within the target graphic label, no rectangular labels are distributed in the neighborhood of its center of symmetry, ensuring that all rectangular labels distributed around the center of symmetry do not exceed one circle. The number of rectangular labels distributed in the neighborhoods on either side of the center of symmetry within the target graphic label is different. The charging interface includes charging electrodes from a charging device, which include positive and negative electrodes, respectively positioned directly below the target graphic label. This allows the robot to identify a target graphic label by searching the positions of four or more corner points and their distribution relative to the center of symmetry, improving the positioning accuracy of the target graphic label and the accuracy of the robot's docking and charging, thereby enabling the robot to locate and even point towards the charging interface.

[0011] Furthermore, the target graphic label contains three rows of rectangular labels. The row passing through the center of symmetry contains three rectangular labels: one at the center of symmetry, parallel to the central axis of the charging device, and the other two on either side of the center of symmetry. The row passing through the center of symmetry is designated as the middle row. The row above the middle row contains two rectangular labels, arranged in the same column as the rectangular label at the center of symmetry and the label on one side of it. The row below the middle row also contains two rectangular labels, arranged in the same column as the rectangular label at the center of symmetry and the label on the other side of it. The directional correction labels on one side and the other side of the target graphic label are centrally symmetrical about the center of symmetry of the target graphic label. This distinguishes the left and right sides of the target graphic label.

[0012] Furthermore, within the vertical surface, the coverage area of ​​a triangular label is larger than that of a rectangular label; no side of the triangular label is perpendicular to the target straight line direction, while each rectangular label has two parallel sides perpendicular to the target straight line direction, so that multiple triangular labels are distributed on both sides of the rectangular label. When the target straight line direction is parallel to the pinhole plane of the camera, but the vertical surface is not parallel to the pinhole plane of the camera, the triangular labels serve as correction tags for the robot's movement direction, guiding the robot from the positions corresponding to the triangular labels on both sides to the position corresponding to the rectangular label in the middle (the sides of the rectangular label perpendicular to the target straight line direction are undistorted), reducing the ranging error caused by distortion of the triangular labels in the camera. Therefore, in guiding the robot to contact the charging interface from far to near, the process progresses from recognizing triangular labels to recognizing rectangular labels, from having ranging errors to eliminating ranging errors, thus achieving high-precision closing of the distance between the robot and the charging interface represented by the rectangular labels and aligning it with the charging interface.

[0013] A charging interface docking method based on tag recognition is used to control a robot to locate and dock with the charging interface of the charging device. The charging interface docking method includes: Step A, the robot preprocesses the image captured by its camera, and then searches for the graphic attributes and vertices of the tag pattern in the preprocessed image, wherein the tag pattern is a target graphic tag or a direction correction tag; Step B, based on the graphic attributes and vertices of the tag pattern, the distance between the tag pattern and the camera, and the deflection angle of the tag pattern relative to the camera are calculated using the monocular ranging principle; Step C, a predicted position point is set according to the distance between the tag pattern and the camera, and the deflection angle of the tag pattern relative to the camera; Step D, the robot moves to the predicted position point, and then uses the camera to capture an image of the tag pattern. Steps A to D are repeated until the latest predicted position point is the position point occupied by the charging interface in the robot's walking plane. The robot's movement direction is parallel to the perpendicular direction of the plane where the target graphic tag is located, so that the robot docks with the charging interface. Compared to existing technologies, this method uses the vertices of the identified label pattern to calculate the distance and angle information of the label relative to the same camera, thereby predicting the relative position of the charging interface and the camera. This provides accurate positioning information for the robot to navigate back to the charging device for docking and charging. In the process of iteratively processing the distance between the label pattern and the camera, as well as the deflection angle of the label pattern relative to the camera, the robot's movement direction is corrected to the orientation of the charging interface represented by the target graphic label. This improves the visual positioning accuracy of the camera and the accuracy of docking and charging as the robot approaches the charging interface.

[0014] Further, step C specifically includes: determining whether the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera, meet the preset positioning conditions. If yes, no new predicted position point is set; otherwise, a new predicted position point is set. The preset positioning conditions include that the newly set predicted position point is the position of the charging interface in the robot's walking plane, and that the robot's movement direction is perpendicular to the plane where the target graphic label is located and points towards the target graphic label. Based on these preset positioning conditions, the robot is adjusted and controlled to gradually approach the device to be positioned, guided by the specific arrangement order of the label patterns.

[0015] Furthermore, in step A, the robot identifies a direction correction label based on the graphic attributes of the label pattern, so that the currently identified direction correction label represents the label pattern, and is used to repeat steps A to D; the robot identifies a target graphic label at the last set predicted position point, so that the currently identified target graphic label represents the label pattern, and is used to execute step D, so that when the latest set predicted position point is the position point occupied by the charging interface in the robot's walking plane, and the robot's movement direction is perpendicular to the plane where the target graphic label is located and points towards the target graphic label, the visual positioning of the charging interface is determined to be complete. In the process from identifying the direction correction label to identifying the target graphic label, the visual positioning accuracy of the corresponding target graphic label in the charging interface is improved.

[0016] Furthermore, before recognizing the target graphic label, during steps A to D, the robot sequentially traverses each identified orientation correction label to guide its movement from the identified orientation correction labels on both sides towards the unidentified area in the center. At the last determined predicted position point, the robot identifies the rectangular label at the center of the target graphic label and determines that the distance between this rectangular label and the camera meets preset positioning conditions. This confirms the presence of the target graphic label within the unidentified area in the center and identifies multiple orientation correction labels distributed on both sides of the target graphic label, which is composed of multiple rectangular labels. Moreover, the robot adjusts its pose so that its movement direction is parallel to the perpendicular direction of the plane containing the target graphic label and points towards the target graphic label. The robot's movement direction is then aligned with the charging interface, confirming the completion of the positioning correction for the charging device or charging interface.

[0017] Furthermore, when the robot identifies the orientation correction label, it does not identify the rectangular label, and therefore does not identify the target graphic label. At this point, the robot is at the first predicted position point. When the robot identifies the rectangular label, it does not identify the orientation correction label. At this point, the robot is at the second predicted position point. The distance between the first predicted position point and the charging interface is greater than the distance between the second predicted position point and the charging interface. The orientation correction label does not include the rectangular label. The label size of an orientation correction label on the surface of the charging device is greater than the label size of any rectangular label included in the target graphic label on the surface of the charging device.

[0018] Therefore, in scenarios where the camera is far from the charging device, during the repeated execution of steps A to D, the robot sequentially identifies each directional correction label before starting to identify the target graphic label. When approaching a rectangular label, for example, at a position where the distance to one of the rectangular labels is less than or equal to the focal length, all rectangular labels can be identified, thus confirming the identification of a target graphic label. This can also be understood as the robot confirming the identification of a target graphic label after it has continuously identified multiple rectangular labels. This guides the robot to move from the identified directional correction labels on both sides towards the unidentified image area in the middle.

[0019] Furthermore, in step A, the robot identifies orientation correction labels and / or target graphic labels from multiple label patterns at once based on the graphic attributes of the label patterns, thereby obtaining the vertices and graphic attributes of the identified label patterns. Whenever multiple orientation correction labels are identified, in step B, based on the graphic attributes and vertices of each orientation correction label, the robot calculates the distance between each orientation correction label and the camera, as well as the deflection angle of each orientation correction label relative to the camera, using the monocular ranging principle. Then, the robot sequentially iterates through the distances between each identified orientation correction label and the camera. When the robot determines that the distances between the two closest orientation correction labels (with different placement patterns) located on either side of the center position and not the two smallest distances among the identified orientation correction labels and the camera, it determines that the distances between the currently identified label patterns and the camera do not meet the preset positioning conditions. Therefore, this technical solution accurately excludes unsuitable positioning locations and guides the subsequently set predicted position points to the area between the two closest orientation correction labels located on either side of the center position.

[0020] Further, in step C, the robot sets a predicted position point in front of the two directional correction labels with different placement patterns that are closest to each other on both sides of the central position. Then, the robot adjusts its pose and moves to the currently set predicted position point based on the deflection angle of the predicted position point relative to the camera, thereby reducing the distance between the robot and the target graphic label. The robot repeats steps A to C until it determines that the distance between the two directional correction labels with different placement patterns that are closest to each other on both sides of the central position and the camera is the two smallest distances among the distances between all identified directional correction labels and the camera. Then, the robot moves to the currently set predicted position point and identifies the target graphic label in step A. Before the robot identifies the target graphic label in step A, the distance between each currently identified label pattern and the camera is not allowed to meet the preset positioning condition. Therefore, multiple directional correction labels serve to correct the robot's movement direction.

[0021] Further, after identifying the target graphic label in step A, the robot then identifies the individual rectangular labels that make up the target graphic label; then in step B, based on the graphic attributes and vertices of each rectangular label, it calculates the distance between each rectangular label and the camera, as well as the deflection angle of each rectangular label relative to the camera, using the monocular ranging principle; then it sequentially iterates through the distances between each identified rectangular label and the camera; when the robot determines in step C that the distance between the rectangular label at the center of the target graphic label and the camera is not the smallest among the distances between the identified rectangular labels and the camera, it determines that the distance between the currently identified target graphic label and the camera does not meet the preset positioning conditions; then in step D, the robot... The robot moves towards the rectangular label at the center of the target graphic label. The distance between the rectangular label at the center of the target graphic label and the camera is the smallest among all the distances between the identified rectangular labels and the camera. The distance between the currently identified target graphic label and the camera satisfies a preset positioning condition. The currently moved position is the latest predicted position. Then, at the latest predicted position, the robot's movement direction is adjusted to be parallel to the perpendicular direction of the rectangular label at the center of the target graphic label. The distance between the currently identified target graphic label and the camera, as well as the deflection angle of the currently identified target graphic label relative to the camera, are determined to satisfy the preset positioning condition. In summary, this technical solution first determines the position of the target graphic label based on the identified direction correction label, achieving a coarse positioning effect for the charging interface. Then, by continuously changing the robot's required navigation position through the rectangular labels within the target graphic label, the distance between the robot and the charging interface is continuously reduced until the latest calculated distance and angle satisfy the preset positioning condition, thus achieving precise positioning of the charging interface and improving the accuracy of the robot's return to the charging device for docking.

[0022] Furthermore, the robot identifies a direction correction label as consisting of a triangular label and determines the side length of the triangular label by its vertex; the arrangement of the direction correction labels on one side of the target graphic label is different from the arrangement of the direction correction labels on the other side of the target graphic label; each direction correction label on each side of the target graphic label is equally spaced along a straight line on the surface of the charging device; the robot identifies a target graphic label as consisting of multiple identical rectangular labels arranged together and determines the side length of the rectangular labels by their vertices; a target graphic label is a regular polygonal shape arranged with a rectangular label as its center position, and the number of rectangular labels distributed in the two neighboring areas on both sides of the center position of the target graphic label is different to distinguish the two sides of the target graphic label.

[0023] Furthermore, the method for identifying orientation correction labels and / or target graphic labels from multiple label patterns simultaneously based on the graphic attributes of label patterns includes: when the robot searches for the graphic attributes and vertices of the label patterns within the preprocessed image, the robot detects the number of edges forming a closed shape; wherein, the graphic attributes of the label patterns are the edge features of the closed shape, and the edge features of the closed shape include the number of edges forming the closed shape and the number of vertices of the closed shape; in the edge lines of the closed shape, the robot identifies the line connecting two adjacent vertices as an edge forming the closed shape; when the robot detects that the number of edges forming the closed shape is 3, the currently detected edge is... The robot identifies closed shapes as triangular labels and determines them as orientation correction labels. When the robot detects that there are four sides forming a closed shape and two sets of sides are perpendicular, it identifies the currently detected closed shape as a rectangular label, where each set has two parallel sides. If the number of rectangular labels detected by the robot in the same frame is equal to the total number of rectangular labels required to form a target graphic label, and the detected rectangular labels are symmetrically arranged with one of the rectangular labels as the center, and the number of rectangular labels distributed in the neighboring areas on both sides of the center is different, then a target graphic label is identified. This allows the robot to distinguish labels of different shapes based on the graphic attributes of the label pattern, and then identify the target graphic label and orientation correction label by combining the number and distribution of label patterns of different shapes.

[0024] Further, in the label pattern, the robot records the edge that forms a certain angle with the target straight line as the edge to be measured in the label pattern; the distance between the label pattern and the camera in step B includes the distance from the edge to be measured at the first detection point to the camera and the distance from the edge to be measured at the second detection point to the camera; the monocular ranging principle in step B includes a pinhole imaging model; the method for calculating the distance between the label pattern and the camera using the pinhole imaging model includes: obtaining in advance the lens focal length f of the camera, the side length w of the edge to be measured, and the pixel width p formed by the edge to be measured in the imaging plane of the camera; the distance between the edge to be measured and the camera is calculated using the following formula:

[0025]

[0026] If one endpoint of the edge to be tested is the first detection point, then the edge to be tested is the edge to be tested where the first detection point is located, and then d is set to be equal to the distance dx1 from the camera to the edge to be tested where the first detection point is located; if one endpoint of the edge to be tested is the second detection point, then the edge to be tested is the edge to be tested where the second detection point is located, and then d is set to be equal to the distance dx2 from the camera to the edge to be tested where the second detection point is located; wherein, the plane of the object to be tested is the surface of the charging device where the label pattern is located; when the plane of the object to be tested is not parallel to the pinhole plane of the camera, the intersection line of the plane of the object to be tested and the pinhole plane of the camera is set perpendicular to the direction of the target straight line. This technical solution can be viewed as constructing a pinhole imaging model from the perspective of a side view between the plane of the object under test, the pinhole plane of the camera, and the imaging plane of the camera (in fact, it uses the geometric relationship of similar triangles from the side view of each plane (preferably the direction perpendicular to the horizontal ground)). The length of the perpendicular line segment from the camera to the side of the object under test where the first detection point or the second detection point is located can be calculated. Distance calculation can be performed using the pinhole imaging model without ignoring the distortion caused by the vertical or horizontal direction, thus saving the amount of calculation of the distance between the label pattern and the camera.

[0027] Further, in step B, the method for calculating the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera using the monocular ranging principle, includes: the robot sets the target straight line direction as the extension direction of the plane of the object to be measured on the horizontal plane, and sets the target straight line direction as parallel to the robot's walking plane; the robot sets two vertices distributed along the target straight line direction in a label pattern as two adjacent target detection points in the plane of the object to be measured; wherein, the distance Ux between two adjacent target detection points in the plane of the object to be measured is obtained in advance; the plane of the object to be measured represents the surface of the charging device where the label pattern is located, perpendicular to the horizontal ground or the robot's walking plane; among the two adjacent target detection points, the robot records one of the target detection points as the first detection point, and calculates the distance dx1 from the side of the first detection point to the camera using the pinhole imaging model; the robot records the other target detection point as the second detection point, and calculates the distance dx2 from the side of the second detection point to the camera using the pinhole imaging model; based on the side-angle relationship of the triangle, the deflection angle of the label pattern relative to the camera is obtained by the following formula:

[0028]

[0029] dx2*sin(bx21)=Ux*sin(ax)+dx1*sin(bx11);

[0030] Ux*cos(ax)=dx2*cos(bx21)+dx1*cos(bx11);

[0031] bx12 = 90 - bx11;

[0032] bx22 = 90 - bx21;

[0033] Wherein, the angle between the perpendicular segment from the camera to the side to be tested at the first detection point and the pinhole plane of the camera in the opposite direction of the target straight line is denoted as bx11, and the angle between the perpendicular segment from the camera to the side to be tested at the second detection point and the pinhole plane of the camera in the target straight line is denoted as bx21; the robot sets the angle between the plane of the object to be tested and the pinhole plane of the camera in the target straight line as the tilt angle of the plane of the object to be tested, where ax is the tilt angle of the plane of the object to be tested; the tilt angle of the plane where the label pattern is located is the tilt angle of the plane of the object to be tested; the deflection angle of the label pattern relative to the camera includes the angle between the perpendicular segment from the camera to the side to be tested at the first detection point and the optical axis, bx12, and the angle between the perpendicular segment from the camera to the side to be tested at the second detection point and the optical axis, bx22; the distance between the label pattern and the camera mentioned in step B includes the distance from the side to be tested at the first detection point to the camera and the distance from the side to be tested at the second detection point to the camera; the monocular ranging principle mentioned in step B includes the pinhole imaging model. In summary, when calculating the deflection angle of the label pattern relative to the camera, this technical solution will construct triangular geometric calculation formulas for the corresponding target detection points and their sides from both vertical and horizontal perspectives to ensure the comprehensiveness and representativeness of the angle calculation.

[0034] Furthermore, when the label pattern is represented as a triangular label, the two adjacent target detection points are the two vertices of the base of the triangular label. Each triangular label's target detection points are distributed along the target straight line in the plane of the object to be measured. The line connecting the two vertices of a triangular label that form a first angle with the target straight line is denoted as the side to be measured where the first detection point is located. The line connecting the two vertices of the same triangular label that form a second angle with the target straight line is denoted as the side to be measured where the second detection point is located. The sum of the second and first angles is 180 degrees. When the label pattern includes a rectangular label, the two adjacent target detection points are the two vertices of the side of the rectangular label parallel to the target straight line. The two sides of the rectangular label perpendicular to the target straight line are the sides to be measured where the first and second detection points are located, respectively. Thus, based on the two adjacent target detection points already identified by the robot, the sides to be measured where the first and second detection points are located are extracted within the same triangular label. It also extracts the edge to be tested from two adjacent target detection points that have been identified by the robot within the same rectangular label, and provides it to the robot to calculate the distance between the label pattern and the camera.

[0035] Furthermore, if any side of the triangular label to be measured is not perpendicular to the target straight line, then that side will be distorted in the camera, causing a distance measurement error in the distance between the triangular label and the camera calculated by the robot using the pinhole imaging model. If any side of the rectangular label to be measured is perpendicular to the target straight line, then that side will not be distorted in the camera. Therefore, when the target straight line is parallel to the pinhole plane of the camera but the vertical plane is not parallel to the pinhole plane of the camera, the triangular label is used as a label for correcting the robot's movement direction. The robot first identifies and locates the triangular label, and then identifies and locates the rectangular label, so that the robot is guided from the positions corresponding to the triangular labels on both sides to the position corresponding to the rectangular label in the middle (the side of the rectangular label perpendicular to the target straight line is not distorted), reducing the distance measurement error caused by the distortion of the triangular label in the camera. Therefore, in guiding the robot to contact the charging interface from far to near, the process involves changing from recognizing triangular tags to recognizing rectangular tags, and from having distance measurement errors to eliminating distance measurement errors, so as to achieve high precision in closing the distance between the robot and the charging interface represented by the rectangular tags and aligning it with the charging interface.

[0036] If the lengths of the sides to be measured where the first detection point of the label pattern is located and the lengths of the sides to be measured where the second detection point of the same label pattern is located are both preset to be less than a preset distortion error value, then the distance between the label pattern and the camera calculated by the robot using the pinhole imaging model is set to have no ranging error. Therefore, step B, which calculates the distance from the camera to the rectangular or triangular label, can be calculated using the pinhole imaging model, saving computational effort and ensuring ranging accuracy.

[0037] Furthermore, if any side of the triangular label to be measured is not perpendicular to the target line, while any side of the rectangular label to be measured is perpendicular to the target line, the robot repeats steps A to D. First, it identifies the triangular labels located on both sides of the rectangular label and calculates the distance between each triangular label and the camera, as well as the deflection angle of the triangular label relative to the camera. Then, it identifies the rectangular label and calculates the distance between the rectangular label and the camera, as well as the deflection angle of the rectangular label relative to the camera. Therefore, in guiding the robot to approach the charging interface from a distance, the robot calculates the distance from the triangular label to the rectangular label using monocular ranging principles, reducing the distance from having ranging errors to eliminating them. This allows for high-precision aligning of the robot with the charging interface represented by the rectangular label.

[0038] Furthermore, in step C, the robot selects the region formed by the angles formed by the perpendicular segments of the perpendicular lines from the identified triangular labels with different arrangements on both sides of the unidentified area, passing through the camera. Then, the predicted position point is set within this selected region of smallest angle. Each time step C is executed, the smallest angle selected by the robot is updated, thus updating the predicted position point and continuously providing the robot with flexible image acquisition positions. This guides the robot to move from the identified triangular labels on both sides towards the unidentified area in the center.

[0039] The distance between a triangular tag and the camera includes the distance from the camera to the side to be measured where the first detection point of the triangular tag is located, and the distance from the camera to the side to be measured where the second detection point of the triangular tag is located, which constitutes a set of distances for a triangular tag.

[0040] Furthermore, after the robot identifies the rectangular label in the target graphic label, during step D, as the robot moves towards the rectangular label closer to the center of the target graphic label, if the robot is detected to be to the left of the rectangular label at the center of the target graphic label based on the deflection angle of the currently identified rectangular label relative to the camera, the robot moves to the right to the new predicted position point; or, if the robot is detected to be to the right of the rectangular label at the center of the target graphic label based on the deflection angle of the currently identified rectangular label relative to the camera, the robot moves to the left to the new predicted position point; until at the latest determined predicted position point, it is determined that the distance between the rectangular label at the center of the target graphic label and the camera is the smallest among the distances between all identified rectangular labels and the camera, and the currently identified target graphic label is determined to be the rectangular label at the center of the target graphic label. The distance between the graphic label and the camera meets preset positioning conditions. Simultaneously, while determining that the distance between the currently identified target graphic label and the camera meets these conditions, the robot adjusts the camera's optical axis to be perpendicular to the plane of the object under test by rotation. This changes the robot's movement direction to be parallel to the vertical direction of the rectangular label at the center of the target graphic label and pointing towards it. This determines that the deflection angle of the currently identified target graphic label relative to the camera meets the preset positioning conditions. The distance between a rectangular label and the camera includes the distance from the camera to the side of the rectangular label containing its first detection point and the distance from the camera to the side of the rectangular label containing its second detection point, forming a set of distances for the rectangular label. The distance between the target graphic label and the camera includes the distances between all the rectangular labels required to form the target graphic label and the camera. This allows the robot to overcome ranging errors caused by image distortion by adjusting its movement direction until the robot's movement direction is aligned with the charging interface, thus correcting the robot's recharge direction, reducing errors during the recharge process, and improving recharge efficiency.

[0041] Furthermore, when the robot identifies two adjacent target detection points on the plane of the object under test, it calculates the product of the distance Ux between the two adjacent target detection points and the sine of the tilt angle of the object under test, Ux*sin(ax). Ux*sin(ax) is then set as the ranging error caused by the distortion of the line connecting the two adjacent target detection points in the camera. If the robot detects that one edge of the label pattern is not parallel to the pinhole plane of the camera, it confirms that this non-parallel edge is distorted in the camera, thus determining that the label pattern is not parallel to the pinhole plane and is distorted in the camera. If the robot detects that one edge of the label pattern is parallel to the pinhole plane of the camera, it confirms that this parallel edge is not distorted in the camera. This achieves the detection of whether the label pattern on the plane of the object under test is distorted in the camera.

[0042] Furthermore, the robot updates the predicted position point by performing step D and moves to the latest predicted position point to reduce the ranging error caused by the distortion of the acquired label pattern. Until the latest predicted position point is the location of the charging port on the robot's walking plane, and the robot's movement direction is perpendicular to the plane of the target graphic label and pointing towards the target graphic label, the ranging error caused by the distortion of the line connecting two adjacent target detection points in the camera falls within a pre-set target error range, where the pre-set target error range includes the value 0. This suppresses the error caused by the label pattern distortion, improves the accuracy of distance and angle calculations in step B, and thus improves the positioning accuracy of the predicted position point.

[0043] Furthermore, when the distance between two adjacent target detection points obtained by the robot is less than 0.5*(dx1+dx2), the robot sets the ranging error caused by the distortion of the line connecting the two adjacent target detection points in the camera to be equal to 0. In summary, before executing the charging interface docking method, the label pattern on the plane of the object to be tested can be set to a relatively small size, thereby minimizing the ranging error caused by distortion when the plane of the object to be tested is not parallel to the imaging plane of the camera, i.e., suppressing the distance error caused by angle changes.

[0044] A charging system includes a robot and a charging device. The robot is configured to execute a tag-based charging interface docking method to dock with the charging interface and perform charging. The robot is equipped with a monocular camera to collect data on multiple tags on the surface of the charging device. Compared to existing technologies, this charging system controls the robot to calculate the distance and angle information of the tags relative to the same camera using the vertices of the identified tag patterns. This predicts the relative position of the charging interface and the camera, providing accurate positioning information for the robot to navigate back to the charging device for docking and charging. When the robot systematically traverses the tags in a single direction, it can distinguish whether the robot is approaching or moving away from the charging interface, thus correcting the robot's movement direction. Therefore, in the iterative processing of the distance between the tag pattern and the camera, and the deflection angle of the tag pattern relative to the camera, the robot's movement direction is corrected to the orientation of the charging interface represented by the target graphic tag. This improves the visual positioning accuracy of the camera and the accuracy of docking and charging as the robot approaches the charging interface. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating the charging interface docking method based on tag patterns disclosed in the embodiments of this application.

[0046] Figure 2 This is a schematic diagram showing the distribution of triangular and rectangular tags on the surface of the charging device according to an embodiment of this application.

[0047] Figure 3 This is a schematic diagram of the image obtained after preprocessing the label pattern disclosed in the embodiments of this application.

[0048] Figure 4 This is a schematic diagram illustrating the principle of calculating the distance between the camera and the triangular label from a side view and calculating the deflection angle of the triangular label relative to the camera from a top view, as disclosed in this application embodiment.

[0049] Figure 5 This is a schematic diagram illustrating the principle of calculating the distance between the camera and the rectangular label from a side view and calculating the deflection angle of the rectangular label relative to the camera from a top view, as disclosed in this application embodiment.

[0050] Figure 6 This is a schematic diagram illustrating the principle of connecting a charging interface using multiple triangular tags and multiple rectangular tags, as disclosed in an embodiment of this application. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described below are only for explaining the present invention and are not intended to limit the present invention.

[0052] In the description of the invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", and "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention.

[0053] In existing rechargeable charging applications, when a robot equipped with a monocular camera searches for a charging port near a charging dock, it remains in motion to locate the dock or plan a recharge route. The robot collects images of the charging port from different angles at different times. For example, the image plane captured by the monocular camera may be at an angle relative to the camera's imaging plane, resulting in ranging errors. To address this, this application uses specially designed labels with unique shapes for the charging devices (charging docks, charging piles, or charging base stations) that the robot needs to locate and dock with. The specific shape of the charging device is not limited, but a vertical surface is provided for attaching each label. One label represents the charging port, and they are arranged according to a specific pattern to ensure that the label corresponding to the charging port is located at the position captured by the robot after positioning correction, such as in the middle of multiple labels. Furthermore, the robot only recognizes and locates the label corresponding to the charging port when it approaches the charging device, facilitating accurate docking with the charging port after being guided or corrected by other labels.

[0054] As one embodiment, to improve the visual positioning accuracy of a robot towards a charging device, this embodiment discloses a charging device. The charging device includes a charging interface for the robot to charge. A target graphic label and multiple orientation correction labels are disposed on the surface of the charging device. The placement of these labels can be captured by an external camera and presented in the camera's imaging plane in a visual imaging manner. In some embodiments, they are categorized as a set of label patterns or disposed on the surface of the charging device in the form of a QR code. The target graphic label represents the label of the charging interface in the charging device. Multiple orientation correction labels are distributed on both sides of the target graphic label to guide the robot to move from the orientation correction labels on both sides towards the target graphic label. Preferably, the multiple orientation correction labels are evenly distributed on both sides of a target graphic label, and each orientation correction label is a label of the same type in the charging device, but different in type from the target graphic label. The multiple orientation correction labels can be placed at equal intervals near the charging interface, and the charging interface itself also has a corresponding target graphic label, forming orientation labels to guide the robot back to charging. When the robot systematically traverses the correction labels in a single direction, it can distinguish whether the robot is approaching or moving away from the charging interface, thus correcting the robot's direction of movement. The docking structure of the robot's charging electrodes and the charging interface is adapted to each other. The vertical direction of the plane containing the target graphic label (the surface of the charging device) indicates the orientation of the charging interface.

[0055] To improve the accuracy of robot docking with the charging interface, the target graphic label is typically positioned in the middle of multiple direction correction labels. The vertical direction of the plane containing the target graphic label is generally perpendicular to the outside of the vertical plane of the charging device, corresponding to the robot's docking direction with the charging interface—that is, the direction in which the robot returns to the charging device and aligns with the charging interface. When it is determined that the robot's movement direction is pointing towards the charging interface of the charging device, or is nearly parallel to the orientation of the charging interface, it can be considered that the robot is moving along its current direction to accurately complete the return-to-charge docking operation. If the target graphic label corresponding to the charging interface is identified, the movement direction is corrected through the various direction correction labels to obtain the opposite direction of the charging interface's orientation. Then, the robot can move in a straight line to complete the docking and charging of the robot's charging electrodes with the charging interface. Thus, the charging device uses multiple orientation correction tags and a target graphic tag on its surface to design the position information of the charging interface for the robot. This allows the robot to accurately dock with the charging interface after being guided or corrected by the other tag patterns, overcoming the influence of the camera's ranging error. Especially when the surface of the charging device under test is not perpendicular to the optical axis of the monocular camera, the robot gradually approaches the charging interface by relying on the guidance direction formed by the specific arrangement of the target graphic tag and the orientation correction tag, which helps to improve the ranging accuracy of the monocular camera for the charging device.

[0056] Specifically, both the target graphic label and the orientation correction label may include labels of a specific shape and their vertex positions. The graphic attributes of both the target graphic label and the orientation correction label include information about the line segments that connect the labels of a specific shape. The line segments that connect the labels of a specific shape include the line segments that connect end to end. These line segments are placed on the surface of the charging device that has a contact and docking structure with the charging electrode components of the robot, and can be detected by the camera set at the front of the robot (i.e., fall within the detection field of view of the camera).

[0057] In the above embodiments, the orientation correction label and the target graphic label are disposed on the vertical surface of the charging device along the target straight line direction. The vertical surface of the charging device is part of the surface of the charging device, and is disposed above a horizontal surface or vertically on the base of the charging device, so that image information of the orientation correction label and the target graphic label, including edge contour information, can be captured by an external camera. The target straight line direction is set as the direction in which the surface of the charging device extends on the horizontal surface, such as... Figure 2 The target straight line shown is set to be parallel to the horizontal ground and from left to right. Here, left refers to the left side of the rectangular label in the middle position, and right refers to the right side of the rectangular label in the middle position. When the charging device is placed on the horizontal ground, the vertical surface of the charging device is set at an angle to the horizontal ground, generally at 90 degrees, which is beneficial for docking with the charging electrodes on the side of the robot body.

[0058] Specifically, in combination Figure 2 It is understood that the vertical surface can be set with a set of label patterns along the target straight line direction. This set of label patterns includes multiple triangular labels of the same shape and size or rectangular labels of different shapes and sizes. Moreover, multiple triangular labels can be symmetrically set on the vertical surface as orientation correction labels. When multiple rectangular labels in the middle position represent the labels of the charging interface, the orientation of these rectangular labels relative to the triangular labels reflects the direction corresponding to the charging interface.

[0059] Preferably, the orientation correction labels of the same shape type are the same in shape and size, but the positions of each orientation correction label are different. Specifically, they can be arranged along the same straight line at various positions on the surface of the charging device. It can be understood that each orientation correction label is set in the label area, so that it can be identified by the robot before the robot identifies the target graphic label, so as to find the location of the target graphic label. Of course, the target graphic label is also set in the label area.

[0060] As one embodiment, the orientation correction labels on one side of the target graphic label are arranged differently from those on the other side; combined with Figure 2It can be seen that the target graphic label is composed of multiple rectangular labels, and the shape and size of each rectangular label are equivalent to the rectangular label with vertices C2 and D2 set. The direction correction label is as follows: Figure 2 As shown by the triangular pattern on the left and the inverted triangular pattern on the right, the three directional correction labels arranged to the left of the rectangular label are all triangular in shape and are evenly spaced, as follows: Figure 2 The triangle containing vertex T1, the triangle containing vertex T2, and the triangle containing vertex T3; the three directional correction labels arranged to the right of the rectangle label are all inverted triangles, and are equally spaced, represented as follows: Figure 2 The triangle containing vertex T4, the triangle containing vertex T5, ​​and the triangle containing vertex T6. This embodiment will... Figure 2 The triangles and inverted triangles shown are both considered as triangle labels. Moreover, the triangle and the inverted triangle are triangle labels with different placement patterns. For example, in the triangle label to the left of the rectangular label, one vertex of the triangle label is located above the base of the triangle label; in the triangle label to the right of the rectangular label, one vertex of the triangle label is located below the base of the triangle label, forming triangle labels with different placement patterns on both sides of the middle position.

[0061] Each directional correction label on each side of the target graphic label is equally spaced on the surface of the charging device. Among the directional correction labels on the same side of the target graphic label, any one directional correction label can be considered as being obtained by translating another directional correction label along the extension direction of the charging device surface (or vertical surface) on the horizontal ground. This can be understood as the position on the charging device surface being translated from left to right or from right to left. The robot then maintains a straight line to orderly identify each directional correction label, distinguishing directional correction labels with the same placement, and thus determining the position of each directional correction label on the charging device surface.

[0062] The charging interface is positioned below the target graphic label to accommodate the height of the robot's charging electrodes. The charging device for charging the robot, specifically the charging interface mounted on the base surface, consists of charging electrodes and other mechanical structures that cooperate with the robot's charging electrodes, including but not limited to magnetic adsorption, elastic adsorption components, and snap-fit ​​fixing components. Preferably, the charging interface is mounted directly below the target graphic label, and the mounting height of the mechanical structures cooperating with the robot's charging electrodes in the vertical plane is equal to the height of the robot's charging electrodes.

[0063] To distinguish the orientation features of the orientation correction labels relative to the target graphic label, the orientation correction labels located on both sides of the target graphic label are set to different placement forms. For example, one orientation correction label has a vertex facing upwards and a bottom edge parallel to the horizontal direction, with the vertex located above the bottom edge; the other orientation correction label has a vertex facing downwards and a bottom edge parallel to the horizontal direction, with the vertex located below the bottom edge. Generally, two orientation correction labels with different placement forms located on both sides of the middle position and closest to each other can be considered as the two orientation correction labels located on both sides of the middle position and closest to each other. Since robots typically move on horizontal surfaces, the orientation correction tags used to correct the robot's direction of movement can be arranged in a horizontal row. The middle position here is the center of the area where multiple identified orientation correction tags are located, specifically the center of the area set by a row of orientation correction tags. The two orientation correction tags closest to each other on either side of the middle position are the left and right adjacent orientation correction tags in the same row. The distance between each orientation correction tag and the camera represents the distance between each orientation correction tag and the robot. Since orientation correction tags have at least two sides in the vertical or horizontal direction, the distance between an orientation correction tag and the robot may have at least two distance values ​​from the same camera to at least two sides. Therefore, the at least two distance values ​​corresponding to an orientation correction tag are recorded as a group of distances. The more orientation correction tags or the more rectangular tags required to form a target graphic tag, the more distance groups are obtained, and the higher the accuracy of the positioning information provided by the charging device to the robot.

[0064] In some embodiments, the orientation correction label is used Figure 2 The triangle label indicates that, in Figure 2 The combination of rectangular labels in the middle position represents the target graphic label. The three vertices of the triangular label to the left of the target graphic label are arranged with two vertices at the top and the remaining vertex at the bottom. The three vertices of the triangular label to the right of the target graphic label are arranged with one vertex at the top and the remaining two vertices at the bottom, thus forming two groups of triangular labels with different arrangements. Each group contains three triangular labels. Figure 2 and Figure 6It is known that the target graphic label is located between the triangle label with vertex T3 and the triangle label with vertex T4. The center of the target graphic label contains a rectangle label with vertices C2 and D2. At present, it is only determined that the target graphic label is located between the triangle label with vertex T3 and the triangle label with vertex T4. Moreover, the triangle label with vertex T3 and the triangle label with vertex T4 have been located. Therefore, the area where the located target graphic label is located can be delineated, and the robot can continue to perform visual localization.

[0065] Specifically, a direction correction label consists of a triangle label, where two vertices of the triangle label form one side of the triangle. The number of sides forming the triangle label is 3, obtained by connecting the vertices of the triangle's edge. A target graphic label consists of multiple identical rectangular labels, where adjacent vertices in the vertical direction form the vertical edge of the rectangle, and adjacent vertices in the horizontal direction form the horizontal edge of the rectangle. Figure 2 The line connecting vertices C2 and D2 is shown; combined with Figure 2 It is known that the target straight line direction is parallel to the base of the triangular label, and the target straight line direction is parallel to the horizontal edge line of the rectangular label. The vertical and horizontal edges of the rectangular label are both sides of the rectangular label, and the number of sides forming the rectangular label is 4, obtained by connecting the vertices of the rectangle's edge lines, excluding the diagonals. In the vertical plane, multiple orientation correction labels are distributed on both sides of the target graphic label. The target graphic label is preferably composed of multiple rows and columns of rectangular labels, which allows for the correction of the robot's pose in both vertical and horizontal directions. The specifically arranged rectangular labels guide the robot closer to the charging interface, thereby achieving contact between the robot and the charging interface at a specific location.

[0066] As one embodiment, a target graphic label is a regular polygonal graphic arranged around a rectangular label as its center of symmetry. Within this target graphic label, a portion of its neighborhood around the center of symmetry has no rectangular labels distributed, ensuring that all rectangular labels distributed around the center of symmetry do not exceed one circle. This can be represented by the different numbers of rectangular labels distributed in the neighborhoods on either side of the center position, thus distinguishing the two sides of the target graphic label. For example, a reserved area... Figure 2The top left and bottom right corners of the target graphic label are not filled with rectangular labels, which helps to distinguish the orientation of the target graphic label. On the other hand, in this embodiment, the charging interface includes charging electrodes of the charging device, which include positive and negative electrodes, respectively located directly below the target graphic label. Specifically, within the planar area covered by the charging interface, the positive electrode can be located on the left side, directly below the rectangular label to the left of the center of symmetry of the target graphic label, and the negative electrode can be located on the right side, directly below the rectangular label to the right of the center of symmetry of the target graphic label; alternatively, the positive electrode can be located on the top side, and the negative electrode on the bottom side, both directly below the target graphic label. In this embodiment, the rectangular label at the center of symmetry of the target graphic label is used as the label of the charging interface. When the robot identifies and locates the label at the center of symmetry, the orientation of the charging interface relative to the robot can also be determined. The height of the target graphic label and the charging interface on the surface of the charging device can be adapted to the assembly height of the robot's charging electrodes. The robot can identify a target graphic label by searching the positions of four or more corner points and their distribution relative to the center of symmetry. This improves the positioning accuracy of the target graphic label and the accuracy of the robot docking with the charging port, thereby enabling the robot to locate or even point to the charging port.

[0067] Preferably, each orientation correction label is set at the same height, and the orientation correction label and the target graphic label are also set at the same height, so that the vertices of the orientation correction label and the target graphic label are located on the same straight line. Then, the orientation correction label and the target graphic label are distributed between two straight lines, and the height of the target graphic label and the height of the orientation correction label are equal to the distance between the two straight lines. If the image is formed within the camera and the vertical surface is not parallel to the pinhole plane of the camera, then the orientation correction label and the target graphic label may not be set at the same height in the same imaging plane.

[0068] As one example, see Figure 2 It can be seen that the target graphic label contains three rows of rectangular labels. Each row passing through the center of symmetry contains three rectangular labels, preferably square labels. A rectangular label is placed at the center of symmetry of the target graphic label and is parallel to the central axis (or vertical plane) of the charging device. The other two rectangular labels are located on either side of the center of symmetry. A rectangular label is placed at the center of symmetry as shown in the image. Figure 2The rectangular labels at vertices C2 and D2 are shown in the diagram. The row passing through the center of symmetry is designated as the middle row. Two rectangular labels are placed in the row above the middle row, aligned with the rectangular label at the center of symmetry and the rectangular label on one side of it, respectively. Two rectangular labels are also placed in the row below the middle row, aligned with the rectangular label at the center of symmetry and the rectangular label on the other side of it, respectively, to indicate the directions directly above, below, to the left, and to the right of the target graphic label. It should be noted that the direction correction labels on one side of the target graphic label are centrally symmetrically positioned with respect to the center of symmetry of the target graphic label, such that the triangular label on one side of the target graphic label is obtained by rotating the triangular label on the other side of the target graphic label 180 degrees around the center of symmetry of the target graphic label. Therefore, the center of symmetry of the target graphic label is the common center of symmetry of the multiple triangular and rectangular labels placed within the vertical plane.

[0069] Specifically, in combination Figure 2 It can be seen that the records in the same row are arranged from left to right, and the records in the same column are arranged from top to bottom. The leftmost end of the first row of the target graphic labels does not have a rectangular label to indicate the top of the target graphic label; the rightmost end of the last row of the target graphic labels does not have a rectangular label to indicate the bottom of the target graphic label; the top of the first column of the target graphic labels does not have a rectangular label to indicate the left side of the target graphic label; and the bottom of the last column of the target graphic labels does not have a rectangular label to indicate the right side of the target graphic label. Figure 2 The diagram shows an octagon composed of seven rectangular labels. Therefore, the number of these rectangular labels distributed in the neighborhoods on either side of the center position of the target graphic label differs, allowing the robot to accurately distinguish the left and right sides of the target graphic label during the traversal of the label pattern.

[0070] In the aforementioned embodiment, within the vertical surface, the coverage area of ​​a triangular label is greater than that of a rectangular label; therefore, the label size of a triangular label on the surface of the charging device is greater than the label size of any rectangular label included within the target graphic label on the surface of the charging device. Within the same focal length environment, when the robot's camera is at a position far from the charging device, for example, at a distance from any of the rectangular labels that is much greater than the focal length, the robot will first recognize the graphic attributes of the triangular label (the complete edge information surrounding the triangular label, specifically including the number of sides) due to the larger label size, and can determine that it has identified a direction correction label. However, because of the greater distance and the smaller size of the rectangular label, the camera lacks the imaging capabilities. Therefore, the robot at the same location will not recognize the rectangular label, and thus will not recognize the target graphic label. When the robot identifies the orientation correction label, it fails to identify the rectangular label, and consequently, the target graphic label. At this point, the robot is at the first predicted position. Furthermore, when the robot identifies the rectangular label, it fails to identify the orientation correction label, placing it at the second predicted position. The distance between the first predicted position and the charging port is greater than the distance between the second predicted position and the charging port. Therefore, when the distance between the orientation correction label and the camera is relatively large, the smaller rectangular labels that make up the target graphic label appear blurred in the camera's imaging plane.

[0071] On the other hand, none of the sides of the triangular labels are perpendicular to the target straight line, while each rectangular label has two parallel sides perpendicular to the target straight line, so that multiple triangular labels are distributed on both sides of the rectangular label. When the target straight line is parallel to the pinhole plane of the camera, but the vertical surface is not parallel to the pinhole plane, the triangular labels serve as correction tags for the robot's movement direction, guiding the robot from the positions corresponding to the triangular labels on both sides to the position corresponding to the rectangular label in the middle (the sides of the rectangular label perpendicular to the target straight line are undistorted), reducing the ranging error caused by distortion of the triangular labels in the camera. Therefore, in guiding the robot to contact the charging interface from far to near, the process progresses from recognizing triangular labels to recognizing rectangular labels, and from having ranging errors to eliminating them, achieving high-precision closing of the distance between the robot and the charging interface represented by the rectangular labels, and aligning it with the charging interface.

[0072] Preferably, if at least one side of the triangular label has a length less than a preset distortion error value, then that side of the triangular label is considered to have no distortion in the camera, so as to suppress the distortion of that side in the camera when the robot recognizes the triangular label and improve the accuracy of positioning.

[0073] Based on the foregoing embodiments, this application also discloses a charging interface docking method based on tag recognition. The charging interface docking method is used to control a robot to locate and dock with the charging device disclosed in the foregoing embodiments. It should be noted that the main body executing the charging interface docking method is a robot equipped with a camera. The robot locates and contacts the charging device disclosed in the foregoing embodiments by executing the charging interface docking method, so as to guide the robot to gradually approach or even align with the charging device with the tag pattern using the image information of the collected tag pattern, and to contact the device in the correct posture. This overcomes the influence of the ranging error of the camera, especially when the surface of the charging device under test is not perpendicular to the optical axis of the monocular camera. The robot gradually approaches the charging interface by relying on the guidance direction formed by the specific arrangement order of the target graphic tag and the orientation correction tag, which helps to improve the ranging accuracy of the camera for the charging device.

[0074] See Figure 1 It is known that the charging interface docking method includes: Step A, the robot preprocesses the image captured by its camera, and then searches for the graphic attributes and vertices of the label pattern in the preprocessed image, wherein the label pattern is a target graphic label or a direction correction label; thereby determining the position information of the label pattern on the surface of the device to be inspected and identifying its shape features; and then step B is executed. In step A, the robot first controls a camera at a defined location to capture images. This camera is a monocular camera, and its field of view covers all the label patterns set in the charging device. The robot's camera can capture images of the label patterns set on the surface of the charging device's base, but it may not be able to distinguish the information of various shaped label patterns, including the direction corresponding to the charging interface. The label pattern may include labels of a specific shape and their vertex positions. The graphic attributes of the label pattern include information on the line segments connecting the labels of a specific shape. The line segments connecting the labels of a specific shape include the line segments connecting the beginning and the end, which can be identified by the robot from the image and the number of identified line segments can be counted. Instead, the labels are all placed on the surface of the charging device's base. Specifically, these labels can be placed at equal intervals near the charging interface, and the charging interface itself also has a corresponding special pattern to form a directional label to guide the robot back to the charging port.

[0075] Preferably, the robot performs preprocessing on the image captured by the camera in step A to remove noise interference, obtain higher precision graphic attributes and vertices of the label pattern, improve the recognition accuracy of the label pattern, and solve the problem of image blurring that easily occurs when the monocular camera equipped on the robot captures images with a fixed focal length.

[0076] Step B: Based on the graphic attributes and vertices of the label pattern, the distance between the label pattern and the camera, as well as the deflection angle of the label pattern relative to the camera, are calculated using the monocular ranging principle. This further determines the ranging error caused by label pattern distortion. Then, step C is executed. When the label pattern is placed on the surface of the charging device and associated with the charging interface, the robot first identifies a portion of the larger, multi-directional correction labels at once (limited by the distance between the camera and the charging device). Then, the positions of the pre-identified label patterns on the surface of the charging device are stored for distance and angle calculations. This allows the robot to determine the positional characteristics of each vertex that makes up the label pattern on the surface of the charging device, such as equidistant distribution or central symmetry about the charging interface. Specifically, based on the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera, the robot can determine the relative positional relationship between the robot and the label pattern. In addition to determining the robot's current position information (coordinates and angle), the positional information of the label pattern and the plane it occupies can also be determined. In this embodiment, the deflection angle of the label pattern relative to the camera can be represented by the angle between the incident light at the corner of the label pattern extracted from the surface of the device to be positioned and the optical axis of the camera. The tilt angle of the plane where the label pattern is located can be the angle between the plane where the label pattern is located and the pinhole plane of the camera. The distance between the label pattern and the camera can be represented by the distance between the edge of the label pattern perpendicular to the horizontal ground and the pinhole plane of the camera (which can be reduced to the center position of the camera lens) to suppress the distortion effect of the label pattern in the camera (lens).

[0077] Step C: Based on the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera, a predicted position point is set. This predicted position point can be a navigation position point relatively close to the label pattern corresponding to the charging interface or the direction corresponding to the charging interface. Then, step D is executed. In some embodiments, the robot uses the angle and distance calculated in real time in step B in step C to convert into coordinate information in the corresponding polar coordinate system. This not only determines the coordinate information of the robot's current position point, but also initially determines the coordinate information of the orientation correction label and target graphic label identified in the charging device, thereby initially establishing the relative positional relationship between the charging device (or even the charging interface) and the robot's current position point. Based on this relative positional relationship, the robot adjusts its pose to move to a predicted position point. The predicted position point can be adjusted according to the actual focal length of the camera. By correcting the robot's movement direction, the label pattern corresponding to the charging interface can be identified, and the robot's charging electrodes can be controlled to move closer to the charging interface of the charging device.

[0078] Preferably, in step C, the method for setting the predicted position point based on the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera, specifically includes: determining whether the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera, meet preset positioning conditions; if yes, no new predicted position point is set; otherwise, a new predicted position point is set; wherein, the preset positioning conditions include that the newly set predicted position point is the position point occupied by the charging interface in the robot's walking plane, and that the robot's movement direction is perpendicular to the plane where the target graphic label is located and points towards the target graphic label. Specifically, determining whether the distance between the label pattern and the camera meets preset positioning conditions includes determining whether the distance between the label pattern and the camera falls within the distance error range specified by the preset positioning conditions, or determining whether the distance between the currently identified label pattern and the camera is the smallest among all identified label patterns, to confirm that the robot has gradually explored the label area with the most positioning value from the identified label patterns; at the same time, determining whether the deflection angle of the label pattern relative to the camera meets preset positioning conditions, including determining whether the deflection angle of the currently identified label pattern relative to the camera is at the location of a specific label pattern (such as the aforementioned label area with the most positioning value, generally set as the middle position of a group of label patterns). The angle corresponding to the opposite direction of the perpendicular direction of the plane (generally perpendicular to the outer surface of the device to be positioned), but the robot's forward direction or the optical axis of the camera is generally pointing towards the device to be positioned, is required for the camera to capture an image of the label pattern. Whether the deflection angle of the label pattern relative to the camera meets the preset positioning conditions also includes determining whether the deflection angle of the label pattern relative to the optical axis of the camera falls within the angle error range specified by the preset positioning conditions. This angle error range can represent the minimum allowable angle range between the optical axis of the camera and the perpendicular direction of the plane containing the specific label pattern (e.g., the label area with the most positioning value mentioned above, generally set as the middle position of a group of label patterns) (generally perpendicular to the outer surface of the device to be positioned). Thus, based on the preset positioning conditions, the robot is adjusted and controlled to gradually approach the device to be positioned by relying on the guidance direction formed by the specific arrangement order of the label patterns.

[0079] Step D: The robot moves to the predicted position point, then uses the camera to capture an image of the label pattern. Steps A to D are then repeated until the latest predicted position point is the location of the charging interface on the robot's walking plane. The robot's movement direction is parallel to the perpendicular direction of the plane containing the target graphic label, allowing the robot to dock with the charging interface. In this embodiment, each time the robot moves to the predicted position point, the camera captures an image, and steps A to D are repeated to update the predicted position point. This also ensures that the robot's movement direction tends towards the charging interface of the charging device, and that the final determined predicted position point is located at the charging interface. The latest predicted position point, which is the location of the charging interface on the robot's walking plane, and the adjusted robot movement direction are then used as the visual positioning result. Simultaneously, the robot stops executing steps A to D, indicating that the robot has finished executing the charging interface docking method.

[0080] Specifically, in this embodiment, whenever the robot moves to a new predicted position and continues to acquire surrounding images, steps A to D are repeated. In step D, the robot uses the angle and distance calculated in real time in step B to convert them into coordinate information in the corresponding polar coordinate system. This not only determines the robot's current position but also preliminarily determines the relative positional relationship between the multiple identified tags in the charging device and the unidentified area, thereby initially establishing the relative positional relationship between the charging position of the charging device and the robot's current position. Based on this relative positional relationship, the robot adjusts its pose and moves to a predicted position. The predicted position can be adjusted according to the actual focal length of the camera. By correcting the robot's movement direction, the label pattern corresponding to the charging interface can be identified, and the robot's charging electrodes can be controlled to move closer to the charging position of the charging device. By repeatedly executing steps A to D to correct the robot's position (updating the predicted position), the influence of ranging errors caused by image distortion acquired by the camera can be overcome. Once the robot's movement direction is determined to be pointing towards the charging interface of the charging device, or to be parallel to the orientation of the charging interface, it can be considered that the robot has accurately completed the recharge docking operation by moving along the current movement direction. That is, when the target graphic label corresponding to the charging interface is identified, the robot completes the docking and charging of the robot's charging electrodes and the charging interface while moving in a straight line along the orientation of the charging interface.

[0081] Compared to existing technologies, this method uses the vertices of the identified label pattern to calculate the distance and angle information of the label relative to the same camera, thereby predicting the relative position of the charging interface and the camera. This provides accurate positioning information for the robot to navigate back to the charging device for docking and charging. In the process of iteratively processing the distance between the label pattern and the camera, as well as the deflection angle of the label pattern relative to the camera, the robot's movement direction is corrected to the orientation of the charging interface represented by the target graphic label. This improves the visual positioning accuracy of the camera and the accuracy of docking and charging as the robot approaches the charging interface.

[0082] As one embodiment, during the execution of step D, the robot identifies a direction correction label based on the graphic attributes of the label pattern in step A, so that the currently identified direction correction label represents the label pattern, and is used to repeat steps A to D. That is, even if the robot does not identify the target graphic label, the label pattern in steps A to D executed by the robot is the direction correction label. The robot identifies the target graphic label at the last set predicted position point, so that the currently identified target graphic label represents the label pattern, and is used to execute step D. This ensures that, when the latest set predicted position point is the position point occupied by the charging interface in the robot's walking plane, and the robot's movement direction is parallel to the perpendicular direction of the plane where the target graphic label is located and points towards the target graphic label, the visual positioning of the charging interface is determined to be complete. This improves the visual positioning accuracy of the corresponding target graphic label in the charging interface during the process from identifying the direction correction label to identifying the target graphic label.

[0083] Whenever the robot acquires the graphic attributes of a label pattern and the vertices of the same label pattern, the robot already has the parameter basis to recognize the currently acquired label pattern. Among them, the graphic attributes of a label pattern and the vertices of the same label pattern constitute the basic judgment elements for the robot to recognize a label pattern of a specific shape.

[0084] In this embodiment, the robot sequentially traverses multiple orientation correction tags before successfully recognizing a target graphic tag. Since the robot's movement direction can represent the optical axis direction of the camera, indicating the lens orientation, the robot's movement direction is aligned with the target graphic tag when it successfully recognizes it. Therefore, the multiple orientation correction tags serve to correct the robot's movement direction. Preferably, after the robot sets the predicted position point to a distance of less than a preset correction distance from the plane containing the tag pattern, the robot cannot recognize the orientation correction tag based on the graphic attributes of the tag pattern in the newly executed step A. Instead, it recognizes the target graphic tag so that the currently recognized target graphic tag represents the tag pattern. The preset correction distance is determined by the focal length of the camera and is positively correlated with the size of the imaging area of ​​the tag pattern or the actual coverage area on the surface of the charging device (the aforementioned vertical surface), reducing the difficulty for the robot to recognize the target graphic tag and the number of times the predicted position point needs to be set.

[0085] As one embodiment, before recognizing the target graphic label, the robot sequentially traverses each direction correction label during steps A to D to guide the robot from the recognized direction correction labels on both sides to the unrecognized label area in the middle, enabling the robot to approach and dock with the charging interface. In this embodiment, the unrecognized label area in the middle is the area where the target graphic label is set on the surface of the charging device, but the image is blurred in the imaging plane of the camera. Each direction correction label is used to guide the robot's movement direction towards the charging interface to shorten the distance between the robot and the charging interface of the charging device. Of course, these labels of the same type have the same shape and size, but the position of each label is different. Specifically, they can be arranged along the same straight line at the corresponding position on the surface of the charging device. It can be understood that each direction correction label is set in the label area of ​​the vertical surface, so that it can be recognized by the robot before the target graphic label is recognized, so as to find the location of the target graphic label. Of course, the target graphic label is also set in the label area. Accordingly, during step D, the robot will first sequentially identify the directional correction labels distributed on the left side of the target graphic label; then, it will identify the target graphic label itself. The robot identifies the directional correction labels from left to right (e.g., ...). Figure 2The horizontal arrows pointing to the target straight line are identified at certain intervals. Each identified label graphic includes the identification of the label graphic's vertices, edges, shape, or the area covered by the graphic itself. Alternatively, during step D, the robot starts from the rightmost direction correction label and identifies each direction correction label sequentially from right to left until it identifies the target graphic label in the middle position. At the last determined predicted position point, the robot identifies the rectangular label at the center of the target graphic label and determines that the distance between the rectangular label and the camera meets the preset positioning conditions, thus confirming the existence of the target graphic label in the unidentified area in the middle. It also determines that multiple direction correction labels are distributed on both sides of the target graphic label, and that the target graphic label is composed of multiple rectangular labels. Furthermore, by adjusting its pose so that the robot's movement direction is parallel to the vertical direction of the plane where the target graphic label is located and points towards the target graphic label, the robot's movement direction is aligned with the charging interface, and the positioning correction of the charging device or charging interface is completed.

[0086] In the aforementioned embodiments, during the execution of step D, or as a repetition of steps A to C, the robot sequentially identifies each direction correction label along a straight line. Generally, it identifies each direction correction label from left to right or from right to left on the plane containing the label pattern, distinguishing the direction correction labels with the same arrangement from those on the same side of the target graphic label. In some embodiments, the robot's identification direction within the plane containing the label pattern is parallel to the robot's walking plane and can be parallel to the base of each triangular label, so that the triangular label on one side of the target graphic label is obtained by rotating the triangular label on the other side of the target graphic label 180 degrees around the center of symmetry of the target graphic label. Therefore, corresponding to... Figure 2 In the diagram, the three triangular labels to the left of the rectangular label at the center are obtained by rotating the three triangular labels to the right of the rectangular label at the center by 180 degrees around the center. During the process of traversing the direction correction labels, they are used to distinguish the left and right sides of the target graphic label, so that the direction correction labels distributed on the left and right sides of the target graphic label correspond to labels with different placement patterns. This makes it easier to guide the robot located on the left side of the target graphic label to move to the right, or guide the robot located on the right side of the target graphic label to move to the left, so that the robot's movement direction continuously approaches the target graphic label.

[0087] In one embodiment, in step A, the robot identifies the orientation correction label and / or the target graphic label from multiple label patterns at once based on the graphic attributes of the label pattern, so as to obtain the vertices and graphic attributes of the identified label pattern. Whenever the robot obtains the graphic attributes of a label pattern and the vertices of the same label pattern, the robot already has the parameter basis to identify the currently obtained label pattern. The graphic attributes of a label pattern and the vertices of the same label pattern constitute the basic judgment elements for the robot to identify a label pattern of a specific shape. Whenever multiple orientation correction labels are identified, in step B, based on the graphic attributes and vertices of each orientation correction label, the distance between each orientation correction label and the camera, as well as the deflection angle of each orientation correction label relative to the camera, are calculated using the monocular ranging principle. Then, the distances between each identified orientation correction label and the camera are traversed sequentially, which can be done along a straight line to systematically reflect the changing patterns of the distances between each orientation correction label and the camera. When the robot determines in step C that the distances between the two orientation correction labels with different placement patterns located on either side of the middle position and the camera are not the two sets of distances with the smallest values ​​among the identified orientation correction labels and the camera, it is determined that the distances between the currently identified label patterns and the camera do not meet the preset positioning conditions, that is, the robot is far away from the two orientation correction labels on either side of the middle position. To distinguish the orientation features of the orientation correction labels relative to the target graphic label, the orientation correction labels located on both sides of the target graphic label are set to different placement forms. For example, one orientation correction label has a vertex facing upwards and a bottom edge parallel to the horizontal direction, with the vertex located above the bottom edge; the other orientation correction label has a vertex facing downwards and a bottom edge parallel to the horizontal direction, with the vertex located below the bottom edge. Generally, two orientation correction labels with different placement forms located on both sides of the middle position and closest to each other can be considered as the two orientation correction labels located on both sides of the middle position and closest to each other.Since robots typically move on level ground, the orientation correction tags used to correct the robot's direction of movement can be arranged in a horizontal row. The middle position here represents the center of the area where multiple identified orientation correction tags are located; specifically, it's the center of the area set by a row of orientation correction tags. The two closest orientation correction tags on either side of the middle position are the left and right adjacent orientation correction tags within the same row. The distance between each orientation correction tag and the camera represents the distance between each orientation correction tag and the robot. Because orientation correction tags have at least two edges in either the vertical or horizontal direction, the distance between a single orientation correction tag and the robot may have at least two distance values ​​from the same camera to at least two edges. Therefore, the at least two distance values ​​corresponding to a single orientation correction tag are recorded as a group of distances. The more distances, the higher the positioning accuracy. The comparison of multiple groups of distances is done by comparing the distances corresponding to edges set in the same direction within each group. If one group has the smallest distance across all directions, it is classified as the group with the smallest distance. Similarly, if two groups have the smallest distance across all directions, they are classified as the two groups with the smallest distances. Therefore, this embodiment accurately excludes unsuitable locations and guides the subsequently set predicted location points to the area between the two closest directional correction labels located on either side of the middle position.

[0088] Based on the above embodiment, the robot performs step C to set a predicted position point in front of the two labels with different orientations that are located on both sides of the middle position and are closest to each other. Then, the robot adjusts its pose (including the robot's direction of movement) according to the deflection angle of the currently set predicted position point relative to the camera and moves to the currently set predicted position point to shorten the distance between the robot and the target graphic label, or to shorten the distance between the robot and the charging interface of the charging device, so that the final set predicted position point is the charging interface of the charging device.

[0089] Then the robot repeats steps A to D until step C determines that the distance between the two closest directional correction labels on either side of the middle position and the camera is the two smallest distances among all the identified directional correction labels and the camera. At this point, the distance between the directional correction labels and the camera satisfies the preliminary positioning condition. However, the distance between the currently identified label patterns and the camera does not meet the preset positioning condition, so new predicted position points need to be set to move towards the reduced unknown area. At this time, the robot has not yet identified the target graphic label or the rectangular label contained within it, but the distance relative to the target graphic label is closer.

[0090] Therefore, after performing step D, the robot begins to identify the target graphic label in step A; if the orientation correction label is used... Figure 2 The triangle label indicates that, in Figure 2 The combination of rectangular labels in the middle position represents the target graphic label. The three vertices of the triangular label to the left of the target graphic label are arranged with two vertices at the top and the remaining vertex at the bottom. The three vertices of the triangular label to the right of the target graphic label are arranged with one vertex at the top and the remaining two vertices at the bottom, thus forming two groups of triangular labels with different arrangements. Each group contains three triangular labels. Figure 2 and Figure 6 It is known that the target graphic label is located between the triangle label with vertex T3 and the triangle label with vertex T4. The center of the target graphic label contains a rectangle label with vertices C2 and D2. At present, it is only determined that the target graphic label is located between the triangle label with vertex T3 and the triangle label with vertex T4. Moreover, the triangle label with vertex T3 and the triangle label with vertex T4 have been located. Therefore, the area where the located target graphic label is located can be delineated, and then handed over to the subsequent steps B to D for visual positioning.

[0091] It is worth noting that before the robot identifies the target graphic label in step A, the distance between each currently identified label pattern and the camera is not allowed to meet the preset positioning conditions. That is, it is constrained by the predicted position points set in each step D, forming a path that tends towards the target graphic label but not towards the direction correction label. Therefore, in this embodiment, the robot will sequentially traverse multiple direction correction labels before successfully identifying the target graphic label located in the middle. Since the robot's movement direction can represent the optical axis direction of the camera, representing the lens orientation, when the robot successfully identifies a target graphic label, its movement direction is aligned with that target graphic label. Thus, the multiple direction correction labels serve to correct the robot's movement direction.

[0092] Based on the above embodiments, after identifying the target graphic label in step A, the robot further identifies the individual rectangular labels that make up the target graphic label. Then, in step B, based on the graphic attributes and vertices of each rectangular label, the robot calculates the distance between each rectangular label and the camera, as well as the deflection angle of each rectangular label relative to the camera, using the monocular ranging principle. Then, the robot sequentially traverses the distances between each identified rectangular label and the camera. This can be done along a pre-set straight line; if there are vertical and horizontal directions, the distances between each rectangular label and the camera need to be traversed row by row and column by column. When the robot determines in step C that the distance between the rectangular label at the center of the target graphic label and the camera is not the smallest among the previously identified distances between the identified rectangular labels and the camera, it determines that the distance between the currently identified target graphic label and the camera does not meet the preset positioning conditions. It should be noted that the center position here can be the center position within the area set by the multi-row, multi-column target graphic labels, such as the symmetrical center in the aforementioned embodiment concerning the charging device. The set of distances between each rectangular label and the camera represents the distance between the target graphic label and the robot. Since a rectangular label typically has two sides in either the vertical or horizontal direction, the distance between a single label and the robot may have two values: the distance from the same camera to each side. Therefore, the two distance values ​​corresponding to a single rectangular label are grouped together. The more rectangular labels there are, the more distance groups there are for the same target graphic label, resulting in higher accuracy for positioning using the target graphic label. The comparison of multiple distance groups is done by comparing the distances corresponding to the sides set in the same direction within each group. The group with the smallest distance across all directions is considered the smallest.

[0093] Then, in step D, the robot moves towards the rectangular label at the center of the target graphic label. The distance between the rectangular label at the center of the target graphic label and the camera is the smallest among the distances between the identified rectangular labels and the camera. This ensures that the distance between the currently identified target graphic label and the camera meets preset positioning conditions. The currently moved position is the latest predicted position, which can be denoted as the location of the charging interface. Then, at the latest predicted position, the robot's movement direction is adjusted to be parallel to the perpendicular direction of the rectangular label at the center of the target graphic label, specifically pointing towards the interior of the device to be positioned where the target graphic label is located. Therefore, it can be determined that the distance between the currently identified target graphic label and the camera, and the deflection angle of the currently identified target graphic label relative to the camera, both meet the preset positioning conditions. This completes the visual positioning of the charging device, including the visual positioning of the charging interface, which is essentially precise positioning after overcoming distortion errors caused by various viewing angles. Figure 2 and Figure 6 It is known that the target graphic label is located between the triangle label with vertex T3 and the triangle label with vertex T4. At the center of the target graphic label, there is a rectangular label with vertices C2 and D2. Currently, not only can the specific position information of each rectangular label of the target graphic label be determined (including the angle and distance relative to the camera), but also the rectangular label with vertices C2 and D2 can be accurately located. That is, the distance between the rectangular label with vertices C2 and D2 and the camera is the smallest value among the distances between the identified rectangular labels and the camera. This allows the robot to walk in a straight line towards the rectangular label with vertices C2 and D2 until it comes into contact with the charging interface.

[0094] In summary, this embodiment first determines the position of the identifiable target graphic label based on the identified direction correction label, which achieves the effect of coarse positioning of the charging interface. Then, the rectangular label inside the target graphic label continuously changes the position that the robot needs to navigate to, and continuously reduces the distance between the robot and the charging interface until the latest calculated distance and angle meet the preset positioning conditions. Only then is the precise positioning of the charging interface completed, which improves the accuracy of the robot returning to the charging device for charging.

[0095] As one embodiment, the robot identifies a orientation correction label as consisting of a triangular label, and determines the side length of the triangular label based on its vertices. Specifically, within the camera's imaging plane, the length of the line connecting adjacent vertices is calculated from the shape and vertices of the orientation correction label, serving as the side length of the triangular label and thus determining the sides that enclose the triangular label; the vertices of the label pattern include the three vertices of each triangular pattern. For example... Figure 2 As shown by the triangular pattern on the left and the inverted triangular pattern on the right, the three directional correction labels arranged to the left of the rectangular label are all triangular in shape and are evenly spaced, as follows: Figure 2 The triangle containing vertex T1, the triangle containing vertex T2, and the triangle containing vertex T3; the three directional correction labels arranged to the right of the rectangle label are all inverted triangles, and are equally spaced, represented as follows: Figure 2 The triangle containing vertex T4, the triangle containing vertex T5, ​​and the triangle containing vertex T6. This embodiment will... Figure 2 The triangles and inverted triangles shown are both considered as triangular labels, and the triangles and inverted triangles are triangular labels with different placement patterns. The placement pattern of the orientation correction labels on one side of the target graphic label is different from that on the other side of the target graphic label; the orientation correction labels on each side of the target graphic label are evenly spaced on the surface of the charging device; moreover, among the orientation correction labels on the same side of the target graphic label, any one orientation correction label is set to be obtained by translating another orientation correction label along the target straight line direction, which can be understood as translating its position on the charging device surface from left to right or from right to left; thus, the robot maintains an orderly identification of each orientation correction label along a straight line direction to distinguish the orientation correction labels with the same placement pattern, and thus can determine the position of each orientation correction label on the surface of the charging device.

[0096] like Figure 2 As shown, the robot recognizes a target graphic label as consisting of multiple identical rectangular labels (e.g., Figure 2 The labels are arranged in a grid of black-filled cells, with rectangular labels preferably being square labels; a target graphic label is a regular polygonal graphic arranged with a rectangular label as its center of symmetry. Figure 2 (The diagram shows an octagon composed of eight rectangular labels); specifically, a charging interface is located directly below the rectangular label at the center of symmetry of the target graphic label. The specific assembly height is adapted to the assembly height of the robot's docking electrode within the robot body to improve the accuracy of robot docking and charging. When the robot's movement direction is parallel to the vertical direction of the rectangular label at the center of symmetry, and the robot's movement direction is towards the rectangular label at the center of symmetry, the robot's movement direction is aligned with the charging interface. This can be understood as the robot's movement direction being opposite to the orientation of the charging interface.

[0097] It is worth noting that when the robot identifies the orientation correction label, it does not identify the rectangular label, and therefore does not identify the target graphic label. At this point, the robot is at the first predicted position point. When the robot identifies the rectangular label, it does not identify the orientation correction label. At this point, the robot is at the second predicted position point. The distance between the first predicted position point and the charging interface is greater than the distance between the second predicted position point and the charging interface. The orientation correction label does not include the rectangular label. The label size of an orientation correction label on the surface of the charging device is greater than the label size of any rectangular label included in the target graphic label on the surface of the charging device. Therefore, on the surface of the charging device where the label pattern is located, if the coverage area of ​​a triangular label on the surface of the charging device is greater than that of a rectangular label, then within the same focal length environment, when the robot's camera is at a position far away from the charging device, such as a position where the distance from any of the label patterns is much greater than the focal length, the robot will first identify the graphic attributes of the triangular label (the complete edge information of the triangular label, specifically including the number of sides) and can determine that it has identified a direction correction label. However, due to the greater distance and the smaller size of the rectangular label, the camera does not have the conditions for imaging. Therefore, the robot at the same position will not identify the rectangular label, and thus will not identify the target graphic label. Therefore, in scenarios where the camera is far from the charging device, during the repeated execution of steps A to D, the robot sequentially identifies each directional correction label before starting to identify the target graphic label. When approaching a rectangular label, for example, at a position where the distance to one of the rectangular labels is less than or equal to the focal length, all rectangular labels can be identified, thus confirming the identification of a target graphic label. This can also be understood as the robot confirming the identification of a target graphic label after it has continuously identified multiple rectangular labels. This guides the robot to move from the identified directional correction labels on both sides towards the unidentified image area in the middle.

[0098] As one embodiment, the robot's identification of orientation correction labels based on the graphic attributes of the label pattern specifically involves the robot identifying triangular and rectangular labels based on the graphic attributes of the label pattern. The corresponding method includes: when the robot searches for the graphic attributes and vertices of the label pattern within the preprocessed image, the robot detects the number of edges forming a closed shape. The graphic attributes of the label pattern are the edge features of the closed shape, including the number of edges and vertices forming the closed shape. Preferably, the closed shape is formed by multiple line segments connected end-to-end, or by multiple pixels connected sequentially. Pixels at the corner positions of the closed shape are identified as vertices of the closed shape. Within the edge lines of the closed shape, the robot identifies the line connecting two adjacent vertices as an edge forming the closed shape. Therefore, during the process of detecting the number of sides of a closed shape in the acquired image, when the robot detects three sides forming a closed shape, it identifies the currently detected closed shape as a triangle label and determines that it is an orientation correction label. When the robot detects four sides forming a closed shape, and there are two sets of mutually perpendicular sides, it identifies the currently detected closed shape as a rectangle label, where each set has two mutually parallel sides. Furthermore, if the number of rectangle labels detected by the robot in the same frame is the total number of rectangle labels required to form a target graphic label, and the detected rectangle labels are arranged with one of the rectangle labels as the center of symmetry, then a target graphic label is determined. This enables the differentiation of labels of different shapes based on the graphic attributes of the label pattern, and then the identification of the target graphic label and orientation correction label by combining the number and distribution of label patterns of different shapes.

[0099] In one embodiment, in the label pattern, the robot designates the edge that forms a certain angle with the target straight line as the edge to be measured in the label pattern. Specifically, this can be divided into the edge to be measured at the first detection point and the edge to be measured at the second detection point. The target straight line is preferably along the extension direction of the vertical surface on the horizontal ground. The angle of inclination between the edge to be measured and the target straight line is not 0, and the edge to be measured is not the edge connecting the first and second detection points. In this embodiment, the distance from the edge to be measured at the first detection point to the camera, and the distance from the edge to be measured at the second detection point to the camera, both belong to the distance between the label pattern and the camera mentioned in step B. This can be understood as the distance between the label pattern and the camera mentioned in step B including the distance from the edge to be measured at the first detection point to the camera and the distance from the edge to be measured at the second detection point to the camera. The monocular ranging principle mentioned in step B includes a pinhole imaging model. Therefore, without considering distortion errors, the method by which the robot can calculate the distance between the label pattern and the camera using a pinhole imaging model includes: pre-obtaining the camera's lens focal length f (a pre-set fixed lens parameter), the side length w of the side to be measured (serving as the object height of the pinhole imaging model), and the pixel width p formed by the side to be measured within the camera's imaging plane (the actual image height that varies with the side length w of the side to be measured). Here, the object plane to be measured is the surface of the charging device where the label pattern is located; when the object plane to be measured is not parallel to the pinhole plane of the camera, the intersection line of the object plane to be measured and the pinhole plane of the camera is set perpendicular to the direction of the target line.

[0100] It should be noted that if the intersection of the plane of the object to be tested and the pinhole plane of the camera is set perpendicular to the target straight line direction, the edge to be tested that is perpendicular to the target straight line direction will not be distorted in the camera. However, if the plane of the object to be tested is not parallel to the pinhole plane of the camera, the edge of the label pattern that is parallel to the target straight line direction will be distorted in the camera.

[0101] Then, the distance between the relevant edge to be measured and the camera is calculated using the following formula:

[0102]

[0103] If one endpoint of the edge to be tested is the first detection point, then the edge to be tested is the edge to be tested where the first detection point is located. Then, d is set to be equal to the distance dx1 from the edge to be tested where the first detection point is located to the camera. The first detection point belongs to the target detection point. Specifically, the case where one endpoint of the edge to be tested is the first detection point is expressed as follows: if the intersection of the line connecting the two adjacent target detection points mentioned above and the edge to be tested is the first detection point, then the edge to be tested is the first edge to be tested. w represents the side length of the first edge to be tested. Substituting into Formula 1, the distance dx1 from the edge to be tested where the first detection point is located to the camera can be obtained. Similarly, if one endpoint of the edge to be tested is the second detection point, then the edge to be tested is the edge to be tested where the second detection point is located. Then, d is set to be equal to the distance dx2 from the edge to be tested where the second detection point is located to the camera. The second detection point is also a target detection point. Specifically, the case where one endpoint of the edge to be tested is the second detection point is expressed as follows: if the intersection of the line connecting the two adjacent target detection points mentioned above and the edge to be tested is the second detection point, then the edge to be tested is the second edge to be tested. w represents the side length of the second edge to be tested. Substituting into Formula 1, the distance dx2 from the edge to be tested where the second detection point is located to the camera can be obtained.

[0104] It should be noted that the lens focal length f and the side length w of the side to be measured can be obtained through mechanical design, or by simple ruler measurement, or by querying the lens's factory parameters (including radius of curvature, distance, refractive index, etc.). The pixel width p formed by the side to be measured in the imaging plane of the camera can be obtained in the coordinate system of the imaging plane of the camera.

[0105] It is worth noting that when the plane of the object under test is not parallel to the pinhole plane of the camera, dx1 is not equal to dx2, and it is even less equal to the distance from the camera to the plane of the object under test when the optical axis of the camera is perpendicular to the plane of the object under test (which is directly calculated from the pinhole imaging model). Therefore, the tilt angle between the plane of the object under test and the pinhole plane of the camera will produce a ranging error. It is necessary to perform the aforementioned steps A to D to deal with the impact of the ranging error on the robot's positioning and alignment with the charging interface until it is determined that the distance between the currently identified target graphic label and the camera, and the deflection angle of the currently identified target graphic label relative to the camera, meet the preset positioning conditions.

[0106] As an implementation method for calculating the distance between the measured side of a triangular label and the camera, when the label pattern is represented as a triangular label, the two adjacent target detection points are the two vertices of the base of the triangular label. Each triangular label's corresponding target detection points are distributed along the target straight line in the plane of the object to be measured (equivalent to the vertical surface disclosed in the aforementioned embodiments). The line connecting two vertices of a triangular label distributed at a first angle to the target straight line is denoted as the measured side where the first detection point is located, and is called the first measured side. The line connecting two vertices of the same triangular label distributed at a second angle to the target straight line is denoted as the measured side where the second detection point is located, and is called the second measured side. The sum of the second and first angles is equal to 180 degrees. Thus, based on the two adjacent target detection points already identified by the robot, the measured sides where the first and second detection points are located are extracted within the same triangular label.

[0107] Preferably, the lengths of the side to be tested where the first detection point is located and the lengths of the side to be tested where the second detection point is located are both preset to be less than a preset distortion error value. Because the distance between two adjacent target detection points obtained by the robot is so small that it can be ignored, it can also be regarded as the ranging error caused by the distortion of the line connecting the two adjacent target detection points falling within the preset target error range. In fact, the distance error is caused by the change in the angle between the line connecting the two adjacent target detection points and the pinhole plane of the camera. Therefore, in this embodiment, the distance between two adjacent target detection points set on the surface of the object to be tested (specifically the surface of the charging dock) by reducing the label pattern is initially reduced to reduce the ranging error caused by distortion, thereby achieving the effect of initial correction of angle distortion.

[0108] See Figure 4 It can be seen that the label pattern recognized by the robot is Figure 4When the robot selects triangle TC1D1, it sets two vertices C1 and D1, distributed along the target line, as two target detection points in the plane Lr1 of the object to be measured. The robot designates the left-leaning target detection point C1 as the first detection point C1. Ignoring distortion of the side containing the first detection point C1 in the camera, the robot designates the distance from the side containing the first detection point C1 to the camera's location O (denoted as the optical center of the camera) as d1, where d1 can be replaced by d in the aforementioned formula one. Ignoring distortion of the side containing the second detection point D1 in the camera, the robot designates the right-leaning target detection point D1 as the second detection point D1, and the distance from the side containing the second detection point D1 to the camera's location O (denoted as the optical center of the camera) as d2, where d2 can be replaced by d in the aforementioned formula one. The line connecting two vertices of a triangular label distributed along a first angle with the target line is designated as the first side to be measured of the triangular label, corresponding to... Figure 4 Let side C1T of the triangle be the side to be tested, where the first detection point C1 is located. Figure 4 In the formula, the side length of the first side C1T to be tested is represented by w11, which is equivalent to w in Formula 1; the pixel width formed by the first side C1T in the imaging plane of the camera is represented by p11, which is equivalent to p in Formula 1. Similarly, the line connecting two vertices of the same triangular label that are distributed along the second angle with the direction of the target line is denoted as the second side to be tested of the triangular label, corresponding to... Figure 4 The side D1T of the triangle is used as the side to be tested where the second detection point D1 is located. The side length of the second side to be tested D1T is represented by w12, which is equivalent to w in Formula 1. The pixel width formed by the second side to be tested D1T in the imaging plane of the camera is represented by p12, which is equivalent to p in Formula 1.

[0109] exist Figure 4 In the schematic diagram from the side view on the right, the optical center O of the camera is located at the pinhole plane Lf of the camera, and the distance from the optical center O of the camera to the imaging plane Lp of the camera is equal to the fixed lens focal length f.

[0110] Based on the similar triangle theorem, the distance from the edge to be tested where the first detection point C1 is located to the location O of the camera (denoted as the optical center of the camera) can be obtained using the following formula:

[0111]

[0112] Therefore, ignoring the distortion of the first and second sides in the camera, the distance d1 between the first side C1T and the optical center O of the camera can be considered as constructing a pinhole imaging model from the side view perspective between the object plane Lr1, the pinhole plane Lf, and the imaging plane Lp of the camera (actually, it is calculated from the side view perspective of each plane using the geometric relationship of similar triangles; see details). Figure 4 Using the similar triangle relationship constructed on the right, the length of the perpendicular segment from the camera to the side to be tested where the first detection point C1 is located is calculated, and then the distance d1 from the first side to be tested to the camera is obtained from the top view. Similarly, the distance d2 between the second side to be tested D1T and the optical center O of the camera can be regarded as constructing a pinhole imaging model between the plane Lr1 of the object to be tested, the pinhole plane Lf of the camera, and the imaging plane Lp of the camera from the side view (actually, it is calculated from the side view of each plane using the geometric relationship of similar triangles), the length of the perpendicular segment from the camera to the side to be tested where the second detection point D1 is located is calculated, and then the distance d2 from the second side to be tested to the camera is obtained from the top view.

[0113] It is worth noting that when the plane of the object under test and the pinhole plane of the camera are considered parallel, the distance from the camera to the first measured side of the label pattern is equal to the distance from the camera to the second measured side of the same label pattern. In this case, the label pattern does not produce distortion in the camera; in fact, combined with Figure 4 It can be seen that the plane Lr1 of the object to be measured is not parallel to the pinhole plane Lf of the camera. Among the two adjacent target detection points C1 and D1 distributed in a triangular label, the distance d1 from the camera to the first side C1T to be measured is not equal to the distance d2 from the camera to the second side D1T to be measured, thus generating a distance difference, which corresponds to a distance measurement error in one direction caused by the distortion of the label pattern.

[0114] As an implementation method for calculating the distance between the side to be measured of a rectangular label and the camera, when the label pattern is represented as a rectangular label, the two adjacent target detection points are two vertices of the side of the rectangular label that is parallel to the target line direction. The two sides of the rectangular label that are perpendicular to the target line direction are the side to be measured where the first detection point is located and the side to be measured where the second detection point is located, respectively. The side to be measured where the first detection point is located is marked as the first side to be measured, and the side to be measured where the second detection point is located is marked as the second side to be measured. Preferably, when both the side to be measured where the first detection point is located and the side to be measured where the second detection point is located are parallel to the pinhole plane of the camera, the side to be measured will not produce distortion in the camera. Thus, based on the two adjacent target detection points that have been identified by the robot, the side to be measured is extracted within the same rectangular label and provided to the robot for calculating the distance between the label pattern and the camera.

[0115] See Figure 5It can be seen that the label pattern recognized by the robot is Figure 5 When the rectangle C2D2FE is defined, the robot sets two vertices C2 and D2, distributed along the target line direction, as two target detection points in the plane Lr2 of the object to be measured. The robot designates the left-leaning target detection point C2 as the first detection point C2, and the distance from the edge to be measured containing the first detection point C2 to the camera's location O (denoted as the optical center of the camera) as d3, where d3 can be replaced by d in the aforementioned formula one. The robot designates the right-leaning target detection point D2 as the second detection point D2, and the distance from the edge to be measured containing the second detection point D2 to the camera's location O (denoted as the optical center of the camera) as d4, where d4 can be replaced by d in the aforementioned formula one. The line connecting two vertices of a rectangular label distributed along a first angle with the target line direction is designated as the first edge to be measured of the rectangular label, corresponding to... Figure 5 The side C2E of the rectangle is taken as the side to be tested where the first detection point C2 is located. Figure 5 In the formula, the side length of the first side C2E to be tested is represented by w21, which is equivalent to w in Formula 1; the pixel width formed by the first side C2E to be tested in the imaging plane of the camera is represented by p21, which is equivalent to p in Formula 1. Similarly, the line connecting two vertices of the same triangular label that are distributed along the second angle with the direction of the target line is denoted as the second side to be tested of the triangular label, corresponding to... Figure 5 The side D2F of the triangle is used as the side to be tested where the second detection point D2 is located. The side length of the second side to be tested D2F is represented by w22, which is equivalent to w in Formula 1. The pixel width formed by the second side to be tested D2F in the imaging plane of the camera is represented by p22, which is equivalent to p in Formula 1.

[0116] exist Figure 5 In the schematic diagram from the side view on the right, the optical center O of the camera is located at the pinhole plane Lf of the camera, and the distance from the optical center O of the camera to the imaging plane Lp of the camera is equal to the fixed lens focal length f.

[0117] Based on the similar triangle theorem, the distance from the edge to be tested where the first detection point C2 is located to the location O of the camera (denoted as the optical center of the camera) can be obtained using the following formula:

[0118]

[0119] Therefore, the distance d3 between the first side C2E to be measured and the optical center O of the camera can be considered as constructing a pinhole imaging model from the side view perspective between the plane Lr2 of the object to be measured, the pinhole plane Lf of the camera, and the imaging plane Lp of the camera (actually, it is calculated from the side view perspective of each plane using the geometric relationship of similar triangles; see details). Figure 5Using the similar triangle relationship constructed on the right, the length of the perpendicular segment from the camera to the side to be tested where the first detection point C2 is located is calculated, and then the distance d3 from the first side to be tested to the camera is obtained from the top view. Similarly, the distance d4 between the second side to be tested D2F and the optical center O of the camera can be considered as constructing a pinhole imaging model between the plane Lr2 of the object to be tested, the pinhole plane Lf of the camera, and the imaging plane Lp of the camera from the side view (actually, the calculation is performed using the geometric relationship of similar triangles from the side view of each plane), calculating the length of the perpendicular segment from the camera to the side to be tested where the second detection point D2 is located, and then obtaining the distance d4 from the second side to be tested to the camera from the top view. It is worth noting that when the plane of the object to be tested is parallel to the pinhole plane of the camera, the distance from the camera to the first side to be tested of the label pattern is equal to the distance from the camera to the second side to be tested of the same label pattern. In this case, the label pattern does not produce distortion in the camera; in fact, combined with Figure 5 It can be seen that the plane Lr2 of the object to be measured is not parallel to the pinhole plane Lf of the camera. Among the two adjacent target detection points C2 and D2 distributed in a rectangular label, the distance d3 from the camera to the first side to be measured C2E is not equal to the distance d4 from the camera to the second side to be measured D2F, thus generating a distance difference, which corresponds to the distance measurement error caused by the distortion of the label pattern.

[0120] In summary, this embodiment can be considered as constructing a pinhole imaging model between the plane of the object under test, the pinhole plane of the camera, and the imaging plane of the camera from the perspective of a side view (in fact, it uses the geometric relationship of similar triangles from the side view of each plane (preferably the direction perpendicular to the horizontal ground)). The length of the perpendicular line segment from the camera to the side of the object under test where the first detection point or the second detection point is located is calculated. Under the premise of ignoring the distortion caused by the vertical or horizontal direction, and assuming that the plane of the object under test, the pinhole plane of the camera, and the imaging plane of the camera are parallel to each other from the side view, the calculation of the distance between the label pattern and the camera is saved.

[0121] As one embodiment, in step B, the method for calculating the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera using the monocular ranging principle includes: the robot sets the target straight line direction as the extension direction of the plane of the object to be measured on the horizontal plane, and sets the target straight line direction as parallel to the robot's walking plane. The robot sets the direction of the target straight line direction as parallel to the extension direction of the horizontal coordinate axis in the imaging plane of the camera on the horizontal plane, and sets the horizontal coordinate axis in the imaging plane of the camera as parallel to the robot's walking plane, so as to facilitate the identification of the label pattern in the imaging plane of the camera. Moreover, the vertical plane perpendicular to the target straight line direction is perpendicular to the robot's walking plane, or it can be parallel to the pinhole plane of the camera to suppress the influence of distortion error. The pinhole plane of the camera is parallel to the imaging plane of the camera. For example, the vertices of the identified label pattern are traversed along the positive direction of the horizontal coordinate axis in the imaging plane. Here, the identified label pattern is the label pattern that has been clearly imaged in the imaging plane of the camera, and its graphic attributes and vertices are determined by the robot.

[0122] Then, the robot sets two vertices of a label pattern distributed along the target straight line as two adjacent target detection points in the plane of the object under test, serving as the two endpoints of an edge of the label pattern. The distance Ux between two adjacent target detection points in the plane of the object under test is obtained in advance, representing the actual width of a label pattern. This distance can be derived through mechanical design, measured with a simple ruler, or directly obtained by querying lens factory parameters (including radius of curvature, distance, refractive index, etc.). The distance Ux between two adjacent target detection points can be calculated when the graphic attributes and vertices of the label pattern are identified. The plane of the object under test refers to the surface of the charging dock where the label pattern is located; in some embodiments, it can be perpendicular to the horizontal ground or the robot's walking plane.

[0123] In this embodiment, the monocular ranging principle includes a pinhole imaging model and the geometric proportions of similar triangles. Among two adjacent target detection points, the robot designates one as the first detection point, typically the left-hand target detection point, and uses the pinhole imaging model to calculate the distance dx1 from the edge to be measured at the first detection point to the camera, which is correlated with the side length in the label pattern perpendicular to the target line direction. The robot designates the other target detection point as the second detection point, typically the right-hand target detection point, and uses the pinhole imaging model to calculate the distance dx2 from the edge to be measured at the second detection point to the camera. Correspondingly, the camera lens has a pinhole plane passing through the lens center (optical center of the camera), and an imaging plane located behind the lens, which are respectively... Figures 4 to 6In the diagram, planes Lf (pinhole plane) and Lp (imaging plane) are represented. These planes are shown from both a top-down view (perpendicular to the robot's walking plane) and a side-view view (parallel to the robot's walking plane), and are both displayed as line segments. The plane of the object under test represents the charging dock plane where the label patterns are located. Since all label patterns are placed on the same plane, the plane of the object under test can also represent the plane where the charging interface is located. The pinhole plane, in the calculation scenario of the pinhole imaging model, is the plane where the optical center of the camera is located; the optical center of the camera is the center of the lens.

[0124] Based on the side-angle relationship of a triangle, the deflection angle of the label pattern relative to the camera is calculated using the following formula:

[0125]

[0126] dx2*sin(bx21)=Ux*sin(ax)+dx1*sin(bx11);

[0127] Ux*cos(ax)=dx2*cos(bx21)+dx1*cos(bx11);

[0128] bx12 = 90 - bx11;

[0129] bx22 = 90 - bx21;

[0130] The deflection angle of the label pattern relative to the camera includes the angle bx12 between the perpendicular segment from the camera to the side to be tested where the first detection point is located and the optical axis, and the angle bx22 between the perpendicular segment from the camera to the side to be tested where the second detection point is located and the optical axis. The angle bx11 is the angle formed by the perpendicular segment from the camera to the side to be tested where the first detection point is located and the pinhole plane of the camera in the opposite direction to the target straight line. The perpendicular segment from the camera to the side to be tested where the first detection point is located can be understood as the perpendicular segment passing through the optical center of the camera and perpendicular to the edge of the label pattern passing through the first detection point. This is applicable to the calculation of the distance between the camera and the side to be tested where the first detection point is located in the pinhole imaging model. The deflection angle of the label pattern relative to the camera includes the angle bx12 between the perpendicular segment from the camera to the side to be tested where the first detection point is located and the optical axis, which is equal to the difference between 90 degrees and bx11. This can be understood by relating it to the correspondence between top-down and side-down views. That is, the distance dx1 from the edge to be tested where the first detection point is located to the camera can be regarded as constructing a pinhole imaging model between the plane of the object to be tested, the pinhole plane of the camera, and the imaging plane of the camera from the side-down view (in fact, it is to use the geometric relationship of similar triangles from the side-down view of each plane (preferably the direction perpendicular to the horizontal ground). The length of the perpendicular segment from the camera to the edge to be tested where the first detection point is located is calculated, and then the distance dx1 from the edge to be tested where the first detection point is located to the camera is obtained from the top-down view. Based on this, the length of the perpendicular segment from the camera to the edge to be tested where the first detection point is located is equivalent to the distance dx1 from the edge to be tested where the first detection point is located to the camera.

[0131] The angle between the perpendicular segment from the camera to the side to be tested where the second detection point is located and the pinhole plane of the camera in the direction of the target straight line is denoted as bx21. The perpendicular segment from the camera to the side to be tested where the second detection point is located can be understood as the perpendicular segment passing through the optical center of the camera and perpendicular to the edge of the label pattern passing through the second detection point. It is applicable to the calculation of the distance between the camera and the side to be tested where the second detection point is located in the pinhole imaging model. The deflection angle of the label pattern relative to the camera includes the angle between the perpendicular segment from the camera to the side to be tested where the second detection point is located and the optical axis, bx22, which is equal to the difference between 90 degrees and bx21. This can be understood by relating it to the correspondence between the top-down and side-down perspectives. That is, the distance dx2 from the edge to be tested where the second detection point is located to the camera can be regarded as constructing a pinhole imaging model between the plane of the object to be tested, the pinhole plane of the camera, and the imaging plane of the camera from the side-down perspective (in fact, it is to use the geometric relationship of similar triangles from the side-down perspectives of each plane (preferably the direction perpendicular to the horizontal ground). The length of the perpendicular segment from the camera to the edge to be tested where the second detection point is located is calculated, and then the distance dx2 from the edge to be tested where the second detection point is located to the camera is obtained from the top-down perspective. Based on this, the length of the perpendicular segment from the camera to the edge to be tested where the second detection point is located is equivalent to the distance dx2 from the edge to be tested where the second detection point is located to the camera.

[0132] It should be noted that the distance between the label pattern and the camera mentioned in step B includes the distance from the edge to be measured where the first detection point is located to the camera, and the distance from the edge to be measured where the second detection point is located to the camera; the monocular ranging principle mentioned in step B includes the pinhole imaging model.

[0133] When the label pattern is a target graphic label, the distance between the target graphic label and the camera includes the distance from the side to be tested where the first detection point of each rectangular label that makes up the target graphic label is located to the camera, and the distance from the side to be tested where the second detection point of the same rectangular label is located to the camera. Similarly, the deflection angle of the target graphic label relative to the camera includes the deflection angle of each rectangular label that makes up the target graphic label relative to the camera.

[0134] The robot sets the angle between the plane of the object under test and the pinhole plane of the camera as the tilt angle of the object under test, where ax is the tilt angle of the object under test; the tilt angle of the plane where the label pattern is located is the tilt angle of the object under test. In Formula 4, distances dx1, dx2, and Ux are known quantities, while angles bx11, bx21, bx12, bx22, and ax are unknown quantities. The equation constructed from Formula 4 can be used to obtain angles bx11, bx21, bx12, bx22, and ax. Thus, by constructing a functional relationship between the cosine theorem and the sides and angles of a triangle corresponding to the aforementioned formula through the pinhole imaging model, the tilt angle of the object under test (the plane where the charging interface is located) and the azimuth information of the label pattern at the first and second detection points relative to the camera can be calculated. Therefore, a monocular camera with a fixed focal length can be used to determine the specific angle information of the carrier plane where a label pattern is located. In summary, distances dx1 and dx2 are calculated from the side view, while distances Ux, angles bx11, bx21, bx12, bx22, and ax can be obtained from the top view. The side and top views can be considered as two perpendicular directions, with one preferably vertical and the other horizontal. Therefore, in this embodiment, when calculating the deflection angle of the label pattern relative to the camera, Formula 4 is constructed sequentially from both vertical and horizontal perspectives for the corresponding target detection points and their edges. This is used for trigonometric calculations to ensure the comprehensiveness and representativeness of the angle calculations. Furthermore, based on the tilt angle of the object's plane, the influence of various label patterns on the ranging error generated by the camera can be calculated. For a detailed analysis, please refer to the aforementioned embodiments for triangular and rectangular labels.

[0135] As an implementation method for calculating the position information of the triangular label relative to the camera, see [link to relevant documentation]. Figure 4 It can be seen that the label pattern recognized by the robot is Figure 4 In the implementation scenario of triangle TC1D1, step B involves calculating the distance between the triangular tag and the camera, and the deflection angle of the triangular tag relative to the camera, using the monocular ranging principle. This includes combining... Figure 4It can be seen that the robot sets two vertices C1 and D1, distributed along the target line direction in a triangle TC1D1, as two target detection points in the plane Lr1 of the object to be measured. The distance between target detection point C1 and target detection point D1 is denoted as U1, which is the length of the base of the triangle. U1 is equal to Ux in the aforementioned formula four. The robot designates the target detection point C1, which is biased to the left, as the first detection point C1, and the distance from the side to be measured where the first detection point C1 is located to the location O of the camera (denoted as the optical center of the camera) is denoted as d1. d1 is equal to dx1 in the aforementioned formula four and is obtained in advance using the pinhole imaging model. The robot designates the target detection point D1, which is biased to the right, as the second detection point D1, and the distance from the side to be measured where the second detection point D1 is located to the location O of the camera (denoted as the optical center of the camera) is denoted as d2. d2 is equal to dx2 in the aforementioned formula four and is obtained in advance using the pinhole imaging model.

[0136] The line connecting two vertices of a triangular label that form a first angle with the target line is denoted as the first side to be measured of the triangular label, corresponding to... Figure 4 The side C1T of the triangle is taken as the side to be tested where the first detection point C1 is located; similarly, the line connecting two vertices of the same triangle label that are distributed along the second angle with the target line is denoted as the second side to be tested of the triangle label, corresponding to... Figure 4 The side D1T of the triangle is taken as the side to be measured where the second detection point D1 is located; wherein the sum of the angles of the second included angle and the first included angle is equal to 180 degrees. It should be noted that the incident light from the first detection point C1 and the incident light from the second detection point D1 intersect at the optical center O of the camera, the optical center O of the camera is located on the pinhole plane Lf of the camera, and the distance from the optical center O of the camera to the imaging plane Lp of the camera is equal to the focal length f of the camera lens. Preferably, in order to reduce the influence of camera distortion, especially when the plane of the object to be measured where the triangular label is located is not parallel to the pinhole plane of the camera, the lengths of the side to be measured where the first detection point is located and the lengths of the side to be measured where the second detection point is located are both preset to be less than the preset distortion error value; or even much less than the preset distortion error value, then d1 and d2 can be directly calculated in advance using the pinhole imaging model.

[0137] Based on the side-angle relationship of a triangle, the deflection angle of the triangle label TC1D1 relative to the camera is calculated using the following formula:

[0138]

[0139] d2*sin(b21)=U1*sin(a1)+d1*sin(b11);

[0140] U1*cos(a1)=d2*cos(b21)+d1*cos(b11);

[0141] b12 = 90 - b11;

[0142] b22 = 90 - b21;

[0143] exist Figure 4 In the diagram, the plane Lv1 passing through the first detection point C1 (actually a virtual plane passing through the first side to be tested C1T and parallel to the pinhole plane Lf of the camera), the plane Lv2 passing through the second detection point D1 (actually a virtual plane passing through the second side to be tested D1T and parallel to the pinhole plane Lf of the camera), the pinhole plane Lf, and the imaging plane Lp are parallel to each other.

[0144] The robot sets the angle between the plane Lr1 of the object to be tested, where the target detection points C1 and D1 are located, and the pinhole plane Lf (or plane Lv1, or imaging plane Lp) of the camera as the tilt angle a1 of the plane of the object to be tested. a1 is equivalent to ax disclosed in Formula 4 above and is an angle value that urgently needs to be calculated and obtained.

[0145] The angle between the perpendicular segment from the camera to the side to be tested at the first detection point C1 and the pinhole plane Lf of the camera in the opposite direction of the target straight line is denoted as b11, which is equivalent to the angle between the perpendicular segment from the camera to the side to be tested at the first detection point C1 and the plane Lv1 in the target straight line direction. The angle between the perpendicular segment from the camera to the side to be tested at the second detection point D1 and the pinhole plane Lf of the camera in the target straight line direction is denoted as b21, which is equivalent to the angle between the perpendicular segment from the camera to the side to be tested at the second detection point D1 and the plane Lv1 in the opposite direction of the target straight line direction. The angle between the perpendicular line segment passing through the optical center O of the camera and perpendicular to the first side C1T to be measured and the plane Lv1 is denoted as b11, which is equivalent to the angle bx11 disclosed in Formula 4 above; the angle between the perpendicular line segment passing through the optical center O of the camera and perpendicular to the second side D1T to be measured and the plane Lv1 is denoted as b21, which is equivalent to the angle bx21 disclosed in Formula 4 above; the robot sets the angle between the plane Lr1 of the object to be measured and the imaging plane Lp (or the pinhole plane Lf) of the camera as the tilt angle of the plane of the object to be measured, where a1 is the tilt angle of the plane of the object to be measured, which is equivalent to the angle ax disclosed in Formula 4 above.

[0146] The deflection angle of the triangular tag relative to the camera includes the angle b12 between the perpendicular segment from the camera to the side to be measured where the first detection point is located and the optical axis, and the angle b22 between the perpendicular segment from the camera to the side to be measured where the second detection point is located and the optical axis. Specifically, the angle b12, which includes the angle between 90 degrees and b11, is equivalent to the angle bx12 disclosed in Formula 4 above. Similarly, the angle b22, which includes the angle between 90 degrees and b21, is equivalent to the angle bx22 disclosed in Formula 4 above.

[0147] exist Figure 4 In Formula 5 disclosed in the illustrated embodiment, distances d1, d2, and U1 are known quantities, while angles b11, b21, b12, b22, and a1 are unknown quantities. The equation constructed from Formula 5 can be used to obtain angles b11, b21, b12, b22, and a1. Specifically, the distance d1 between the first side C1T to be measured and the optical center O of the camera can be considered as constructing a pinhole imaging model from a side-view perspective between the plane Lr1 of the object under test, the pinhole plane Lf of the camera, and the imaging plane Lp of the camera (actually, it is calculated using the geometric relationships of similar triangles from the side-view perspectives of each plane; see details in [reference needed]). Figure 4 Using the similar triangle relationship constructed on the right, the length of the perpendicular segment from the camera to the side to be tested, where the first detection point C1 is located, is calculated. Then, the distance d1 from the first side to be tested to the camera is obtained from the top view. Similarly, the distance d2 between the second side to be tested D1T and the optical center O of the camera can be considered as constructing a pinhole imaging model from the side view between the plane Lr1 of the object to be tested, the pinhole plane Lf of the camera, and the imaging plane Lp of the camera (actually, it is calculated from the side view of each plane using the geometric relationship of similar triangles; see details). Figure 4 (Construct similar triangles on the right), calculate the length of the perpendicular segment from the camera to the side to be tested where the second detection point D1 is located, and then obtain the distance d2 from the second side to be tested to the camera from the top view.

[0148] As an implementation method for calculating the position information of the rectangular label relative to the camera, see [link to relevant documentation]. Figure 5 It can be seen that the label pattern recognized by the robot is Figure 5 The rectangle C2D2FE is located on a plane Lr2 that is not parallel to the camera's imaging plane Lp. In step B, the method for calculating the distance between the rectangular tag and the camera, and the deflection angle of the rectangular tag relative to the camera, using the monocular ranging principle, includes: combining... Figure 5It can be seen that the robot sets two vertices C2 and D2, distributed along the target line direction in a rectangle C2D2FE, as two target detection points in the plane Lr2 of the object to be measured. The distance between target detection point C2 and target detection point D2 is denoted as U2, which is the length of the side C2D2 of the rectangle. U2 is equal to Ux disclosed in Formula 4 above. The robot designates the target detection point C2 on the left as the first detection point C2, and the distance from the side to be measured where the first detection point C2 is located to the location O of the camera (denoted as the optical center of the camera) is denoted as d3. d3 is equal to dx1 disclosed in Formula 4 above, which is obtained in advance using the pinhole imaging model. The robot designates the target detection point D2 on the right as the second detection point D2, and the distance from the side to be measured where the second detection point D2 is located to the location O of the camera (denoted as the optical center of the camera) is denoted as d4. d4 is equal to dx2 disclosed in Formula 4 above, which is obtained in advance using the pinhole imaging model.

[0149] The line connecting two vertices of a rectangular label, perpendicular to the target straight line, is denoted as the first side to be measured of the rectangular label, corresponding to... Figure 5 The side C2E of the rectangle is taken as the side to be tested where the first detection point C2 is located. From a side view, it is preferably parallel to the pinhole plane Lf of the camera. Similarly, the line connecting two vertices of the same rectangular label distributed perpendicular to the target line direction is denoted as the second side to be tested of that rectangular label, corresponding to... Figure 5 The side D2F of the rectangle is used as the side to be measured where the second detection point D2 is located. From the side view, it is preferably parallel to the pinhole plane Lf of the camera. It should be noted that the incident light from the first detection point C2 and the incident light from the second detection point D2 intersect at the optical center O of the camera. The optical center O of the camera is located in the pinhole plane Lf of the camera, and the distance from the optical center O of the camera to the imaging plane Lp of the camera is equal to the focal length f of the camera lens.

[0150] Based on the side-angle relationships of a triangle (obtained using the theorem of similar triangles), the deflection angle of the rectangular label C2D2FE relative to the camera is calculated using the following formula:

[0151]

[0152] d4*sin(b41)=U2*sin(a2)+d3*sin(b31);

[0153] U2*cos(a2)=d4*cos(b41)+d3*cos(b31);

[0154] b32 = 90 - b31;

[0155] b42 = 90 - b41;

[0156] exist Figure 5 In the diagram, the plane Lv3 passing through the first detection point C2 (actually a virtual plane passing through the first side to be tested C2E and parallel to the pinhole plane Lf of the camera), the plane Lv4 passing through the second detection point D2 (actually a virtual plane passing through the second side to be tested D2F and parallel to the pinhole plane Lf of the camera), the pinhole plane Lf, and the imaging plane Lp are parallel to each other. The focal length of the camera remains unchanged relative to any of the aforementioned embodiments. The robot sets the angle between the plane Lr2 of the object to be tested, where the target detection point C2 and the target detection point D2 are located, and the pinhole plane Lf (or imaging plane Lp) of the camera as the tilt angle a2 of the plane of the object to be tested. a2 is equivalent to ax disclosed in the aforementioned formula four and is an angle value that urgently needs to be calculated and obtained.

[0157] The angle between the perpendicular line segment passing through the optical center O of the camera and perpendicular to the first side to be measured C2E and the plane Lv3 is denoted as b31, which is equivalent to the angle bx11 disclosed in Formula 4 above; the angle between the perpendicular line segment passing through the optical center O of the camera and perpendicular to the second side to be measured D2F and the plane Lv3 is denoted as b41, which is equivalent to the angle bx21 disclosed in Formula 4 above; the robot sets the angle between the plane Lr2 of the object to be measured and the imaging plane Lp (or the pinhole plane Lf) of the camera as the tilt angle of the plane of the object to be measured, where a2 is the tilt angle of the plane of the object to be measured, which is equivalent to the angle ax disclosed in Formula 4 above.

[0158] The deflection angle of the rectangular label relative to the camera includes the angle b32 between the perpendicular segment of the camera to the side to be measured (located at the first detection point C2) and the optical axis, and the angle b42 between the perpendicular segment of the camera to the side to be measured (located at the second detection point D2) and the optical axis. Specifically, the angle b32, including the angle between the perpendicular segment of the camera to the side to be measured (located at the first detection point C2) and the optical axis, is equal to the difference between 90 degrees and b31, and b32 is equivalent to the angle bx12 disclosed in Formula 4 above. Similarly, the angle b42, including the angle between the perpendicular segment of the camera to the side to be measured (located at the second detection point D2) and the optical axis, is equal to the difference between 90 degrees and b41, and b42 is equivalent to the angle bx22 disclosed in Formula 4 above.

[0159] exist Figure 5In Formula 6 disclosed in the illustrated embodiment, distances d3, d4, and U2 are known quantities, while angles b31, b41, b32, b42, and a2 are unknown quantities. The equations constructed using Formula 6 can be used to obtain angles b31, b41, b32, b42, and a2. Specifically, the distance d3 between the first measured side C2E and the optical center O of the camera can be considered as constructing a pinhole imaging model from a side-view perspective between the measured object plane Lr2, the camera's pinhole plane Lf, and the camera's imaging plane Lp (actually, it is calculated using the geometric relationships of similar triangles from the side-view perspectives of each plane; see details in [reference needed]). Figure 5 Using the similar triangle relationship constructed on the right, the length of the perpendicular segment from the camera to the side to be tested, where the first detection point C2 is located, is calculated. Then, the distance d3 from the first side to be tested, C2E, to the camera is obtained from the top view. Similarly, the distance d3 between the second side to be tested, D2F, and the optical center O of the camera can be considered as constructing a pinhole imaging model from the side view between the plane Lr2 of the object to be tested, the pinhole plane Lf of the camera, and the imaging plane Lp of the camera (actually, it is calculated from the side view of each plane using the geometric relationship of similar triangles; see details). Figure 5 Using the similar triangle relationship constructed on the right, the length of the perpendicular segment from the camera to the side to be tested where the second detection point D2 is located is calculated. Then, the distance d4 from the second side to be tested to the camera is obtained from the top view. Thus, using two mutually perpendicular viewpoints (top view and side view), Formula Six disclosed in this embodiment is constructed for the target detection point and the side it is located in the rectangular label. Trigonometric calculations are performed in this way to obtain the angle while avoiding the influence of camera distortion, thereby improving the accuracy of the deflection angle of the rectangular label relative to the camera. Furthermore, when the deflection angles of all the rectangular labels required to form a target graphic label relative to the camera are calculated in sequence, the deflection angle of the target graphic label relative to the camera can be obtained. Since all the rectangular labels required to form a target graphic label are parallel to each other or on the same plane, and the graphic attributes, shape and size of each rectangular label are the same, it is only necessary to calculate the angle between one of the rectangular labels and the imaging plane Lp (or pinhole plane Lf) of the camera, which can be used as the angle between the target graphic label and the imaging plane Lp (or pinhole plane Lf) of the camera.

[0160] As one embodiment, in step C, the robot selects the region formed by the angles formed by the perpendicular segments of the perpendicular lines from the camera to the sides of the identified triangular labels with different placement patterns on both sides of the unidentified image area (obtained by combining the top and side views of the same plane). Then, the robot sets the predicted position point within the region formed by the selected angle with the smallest angle, so that the robot can determine at the predicted position point that the distance between the two triangular labels with different placement patterns that are located on both sides of the middle position and are closest to the camera is the two sets of distances with the smallest values ​​among the distances between the identified triangular labels and the camera. The distance between the camera and the two triangular tags, which are located on opposite sides of the center position and have different placement patterns, includes: the distance from the camera to the side to be measured of the first detection point of the triangular tag on the side of the center position along the target straight line, and the distance from the camera to the side to be measured of the second detection point of the triangular tag on the opposite side of the target straight line; wherein, the distance between a triangular tag and the camera includes the distance from the camera to the side to be measured of the first detection point of the triangular tag and the distance from the camera to the side to be measured of the second detection point of the triangular tag, which constitute a set of distances for a triangular tag.

[0161] In essence, this can be understood as distinguishing the triangular labels located on either side of the target graphic label by their placement. This involves selecting the test sides of the identified triangular labels from both sides of an unrecognized image area (occupied by the target graphic label), then obtaining the angles formed by the perpendicular segments from the optical center of the same camera to the test sides of the selected triangular labels on both sides. The region with the smallest angle is then selected as the candidate region for the predicted location point. The robot can recognize triangular labels with a larger coverage area but cannot recognize rectangular labels with a smaller coverage area when it is far from the charging dock or charging port. Therefore, the robot recognizes the triangular label first when performing step A for the first time. The direction correction label was not identified, but no target graphic label composed of multiple rectangular labels was detected. Therefore, the rectangular label required for the area with the smallest included angle could not be identified when the robot first executed step A. However, during the subsequent repeated execution of steps A to D, the robot would explore and approach the position of the rectangular label, preferably located at the center of all identified triangular labels (which can also be understood as the axis of symmetry), specifically at the center of the target graphic label. Thus, the area formed by the perpendicular lines from the optical center of the camera to the perpendicular lines from the perpendicular lines of the closest triangular labels with different arrangements on both sides of the rectangular label is the area formed by the aforementioned smallest included angle. Therefore, each time step C is executed, the included angle with the smallest angle selected by the robot is updated, so that the predicted position point is updated, thereby guiding the robot to move from the identified triangular labels on both sides towards the unidentified area in the middle.

[0162] Because in Figure 6 A triangle label is represented by one of the vertices, so in Figure 6 In this diagram, the line connecting the vertex to the optical center of the camera represents the perpendicular line segment from the camera to the edge to be measured where that vertex is located. Specifically, the perpendicular line segments from the optical center O of the camera to the edges to be measured of the nearest triangle labels on either side of the rectangular label are line segments OT3 and OT4, respectively, corresponding to... Figure 2 The leftmost triangle label (with one vertex at the top and two vertices at the bottom) is the closest to the mid-distance rectangular label. Figure 2 The rightmost inverted triangle label (one vertex at the bottom, two vertices at the top) is the closest to the mid-distance rectangular label. Having sequentially identified the triangle labels represented by vertices T1, T2, T3, T4, T5, and T6, the area formed by the angle with the smallest included angle can be set as... Figure 6In the angle region formed by the midline segments OT3 and OT4, the robot determines through step C that the distance between the triangle label corresponding to T3 and the camera and the distance between the triangle label corresponding to T4 and the camera are the two sets of distances with the smallest values ​​among the distances between vertices T1, T2, T3, T4, T5, and T6 and the camera. The distance between the triangle label corresponding to each vertex and the camera is recorded as the corresponding set of distances.

[0163] The perpendicular bisector of line segment C2D2, or the perpendicular line of the rectangle label it contains, is the axis of symmetry between the left vertices (including T1, T2, T3) and the right vertices (T4, T5, T6) of line segment C2D2. The robot then sets the predicted position point within the area formed by the smallest angle selected so far, corresponding to... Figure 6 In this process, the direction in which the predicted location point is set can be from the vertices of the triangular labels representing the identified triangles distributed on both sides of line segment C2D2 to one endpoint of line segment C2D2. This can be achieved by setting the predicted location point after determining that the distance between the label and the camera satisfies the preliminary positioning conditions disclosed in the aforementioned embodiments, thus ensuring the direction correction meets the requirements. Figure 6 Within the angle region formed by line segments OT3 and OT4; specifically, from a top-down perspective, the robot successively zooms in on the predicted position point from the angle region formed by line segment OT1 and other line segments, the angle region formed by line segment OT2 and other line segments, and the angle region formed by line segment OT3 and other line segments, until... Figure 6 Within the angle region formed by line segments OT3 and OT4, or by successively narrowing the predicted position point from the angle region formed by line segment OT6 and other line segments, the angle region formed by line segment OT5 and other line segments, and the angle region formed by line segment OT4 and other line segments, to... Figure 6 Within the angle region formed by line segments OT3 and OT4, during the repeated execution of steps A to D, the robot is guided to move from the identified labels on both sides toward the unidentified image region in the middle, so that the predicted position point is set at the center of symmetry between all the identified triangular labels, thereby reducing the distance between the robot and the perpendicular line of the plane containing the rectangular label C2D2. Here, the predicted position point is only limited to the region formed by the smallest angle currently selected, without limiting the specific position point. It can be located on the side of the angle or in other angle regions with smaller angles, continuously providing the robot with flexible image acquisition positions.

[0164] In some embodiments, before the ranging error caused by the latest determined label pattern distortion falls within a pre-set target error range, the predicted position point set in each execution of step C is closer to the charging port than the previously set predicted position point. That is, the distance between the predicted position point and the charging port directly below the rectangular label located at the center of the target graphic label (which may be the center of symmetry disclosed in the aforementioned embodiments) is shortened. In some embodiments, during this distance shortening process, the tilt angle of the plane containing the label pattern (which can also be considered the rectangular label) can be adjusted to reduce the ranging error caused by the latest determined label pattern distortion, even to near zero. Alternatively, when the robot sets the predicted position point within the area formed by the smallest angle currently selected, the robot can set the predicted position point on the perpendicular line of the rectangular label located at the center of the target graphic label, corresponding to... Figure 6 The perpendicular line between the rectangle label containing vertices C2 and D2 is the perpendicular bisector of line segment C2D2 from a top-down view. Moreover, the predicted position point set in each execution step C will be closer to the charging interface than the predicted position point set in the previous execution, along the perpendicular line of the rectangle label located at the center of the target graphic label.

[0165] Preferably, when the robot does not recognize the rectangular label, the robot calculates that the distance from the camera to the side to be measured of the rectangular label is within the range defined by the distance between the camera and the two triangular labels placed at opposite sides of the central position with different arrangements. Specifically, among the two triangular labels, the sides to be measured of the two triangular labels are the second side to be measured of the triangular label closest to the left of the rectangular label and the first side to be measured of the triangular label closest to the right of the rectangular label. The vertical line segments from the optical center O of the camera to the sides to be measured of the two triangular labels closest to the rectangular label are line segments OT3 and OT4, respectively. From a top-down perspective, the lower limit and upper limit of the distance range defined by the distance from the camera to the sides to be measured of the two triangular labels are represented by the lengths of line segments OT3 and OT4, respectively. Therefore, the distance from the camera to the sides to be measured of the two triangular labels can be set as the upper and lower limits of the distance from the camera to the sides to be measured of the rectangular label. Furthermore, once the robot has identified the rectangular label, it has also identified the two adjacent target detection points C2 and D2 of the rectangular label, as well as the test edges located at the vertices C2 and D2 of the rectangular label from the optical center O of the camera. Therefore, the lengths of the perpendicular segments from the optical center O of the camera to the test edges on both sides of the rectangular label can be calculated. These lengths fall within the distance range defined by the distance from the camera to the test edges of the two triangular labels, further limiting the distance from the camera to the test edges of the rectangular label. In particular, this is refined to the distance information of the rectangular label representing the charging interface relative to the camera, correspondingly forming the distance range of the charging interface represented by the target graphic label relative to the camera.

[0166] As one embodiment, after the robot identifies the rectangular label in the target graphic label, during the process of moving towards the rectangular label closer to the center of the target graphic label by executing step D, if the robot is detected to be to the left of the rectangular label at the center of the target graphic label based on the deflection angle of the currently identified rectangular label relative to the camera, then the robot moves to the right to the new predicted position point; or, if the robot is detected to be to the right of the rectangular label at the center of the target graphic label based on the deflection angle of the currently identified rectangular label relative to the camera, then the robot moves to the left to the new predicted position point. The robot continues its operation until it determines that the distance between the rectangular label at the center of the target graphic label and the camera is the smallest among all the previously identified rectangular labels and their distances to the camera. This ensures that the distance between the currently identified target graphic label and the camera meets preset positioning conditions. Simultaneously, the robot rotates to adjust the camera's optical axis to be perpendicular to the plane of the object being measured. This changes the robot's movement direction to be parallel to the perpendicular direction of the rectangular label at the center of the target graphic label and pointing towards it. This further confirms that the deflection angle of the currently identified target graphic label relative to the camera meets the preset positioning conditions. This allows the robot to overcome ranging errors caused by image distortion by adjusting its movement direction until it aligns its movement direction with the charging interface, thus correcting the robot's recharge direction, reducing errors during the recharge process, and improving recharge efficiency.

[0167] It should be noted that the predicted position point set in each execution step D can be closer to the charging interface than the previously set predicted position point, that is, the distance between the predicted position point and the charging interface directly below the rectangular label located at the center of the target graphic label (which can be the center of symmetry disclosed in the aforementioned embodiments) is shortened. This continues until the distance between the rectangular label located at the center of the target graphic label and the camera is determined to be the smallest among the distances between all the identified rectangular labels and the camera at the latest determined predicted position point, and it is determined that the distance between the currently identified target graphic label and the camera meets the preset positioning conditions. During this distance shortening process, the tilt angle of the plane where the label pattern (which can also be regarded as the rectangular label) is located can be adjusted to reduce the ranging error caused by the edge distortion of the label pattern newly determined by the robot, or even approach a value of 0. Here, the distance between a rectangular label and the camera includes the distance from the camera to the side to be measured where the first detection point of the rectangular label is located, and the distance from the camera to the side to be measured where the second detection point of the rectangular label is located, constituting a set of distances for a rectangular label; the distance between the target graphic label and the camera includes the distances between all the rectangular labels and the camera required to form the target graphic label.

[0168] Preferably, if the robot fails to recognize the rectangular label, or if the robot moves to the predicted position point determined in the last executed step C, the robot calculates that the distance from the camera to the side to be measured of the rectangular label is within the range defined by the distance from the camera to the sides to be measured of the two triangular labels. Specifically, the two triangular labels are considered as the label patterns closest to the rectangular label and located on both sides (the rectangular label is recognized after the two triangular labels are recognized). Among the two triangular labels, the sides to be measured of the two triangular labels are the second side to be measured of the triangular label on the left side of the rectangular label and the first side to be measured of the triangular label on the right side of the rectangular label, respectively. In particular, when the robot moves to the predicted position point determined in the last executed step C, the vertical line segments from the optical center O of the camera to the sides to be measured of the triangular labels closest to the rectangular label on both sides are line segments OT3 and OT4, respectively. Therefore, the distance from the camera to the two triangular labels is determined to be within the range defined by the distance from the camera to the sides to be measured of the two triangular labels. The lower and upper limits of the distance range defined by the distance to the side to be measured of the triangular label are the lengths of line segments OT3 and OT4, respectively. Therefore, the distance from the camera to the side to be measured of the two triangular labels can be set as the upper and lower limits of the distance from the camera to the side to be measured of the rectangular label. Thus, when the robot has already identified the rectangular label, it has identified the two adjacent target detection points C2 and D2 of the rectangular label, as well as the side to be measured where the camera's optical center O is located and the side to be measured where vertex C2 and D2 are located in the rectangular label. Therefore, the length of the perpendicular line segment from the camera's optical center O to the side to be measured on both sides of the rectangular label can be calculated. This is within the distance range defined by the distance from the camera to the side to be measured of the two triangular labels, further limiting the distance from the camera to the side to be measured of the rectangular label, especially refining it to the distance information of the rectangular label representing the charging interface relative to the camera, correspondingly forming the distance range formed by the charging interface represented by the target graphic label relative to the camera.

[0169] As one example, if any side of the triangular label to be measured is not perpendicular to the target straight line, then any side of the triangular label to be measured will be distorted in the camera, so that the distance between the triangular label and the camera calculated by the robot using the pinhole imaging model will have a distance measurement error. Figure 4 The first side C1T to be tested is not perpendicular to the target line direction, and the line connecting the two adjacent target detection points C1 and D1 is distorted. Therefore, the first side C1T to be tested will be distorted in the camera. Similarly, the second side D1T to be tested, which is not perpendicular to the target line direction, will also be distorted in the camera. If any side of the rectangular label to be tested is perpendicular to the target line direction, then any side of the rectangular label to be tested will not be distorted in the camera; so that the distance between the rectangular label and the camera calculated by the robot using the pinhole imaging model has no distance measurement error. Figure 5The first side C2E to be tested is perpendicular to the target line direction, while the line connecting the two adjacent target detection points C2 and D2 is distorted. Therefore, the first side C2E will not be distorted in the camera. Similarly, the second side D2F, which is perpendicular to the target line direction, will also not be distorted in the camera. Therefore, when the target line direction is parallel to the pinhole plane of the camera but the vertical surface is not parallel to the pinhole plane, a triangular label is used as a label to correct the robot's movement direction. The robot first identifies and locates the triangular label, then identifies and locates the rectangular label, guiding the robot from the positions corresponding to the triangular labels on both sides to the position corresponding to the rectangular label in the middle (the side of the rectangular label perpendicular to the target line direction is undistorted), reducing the ranging error caused by the distortion of the triangular label in the camera. Therefore, in guiding the robot to contact the charging interface from far to near, from identifying the triangular label to identifying the rectangular label, from having ranging error to not having ranging error, it achieves high-precision closing of the distance between the robot and the charging interface represented by the rectangular label and aligning it with the charging interface.

[0170] If the lengths of the sides to be measured where the first detection point of the label pattern is located and the lengths of the sides to be measured where the second detection point of the same label pattern is located are both preset to be less than a preset distortion error value, then the distance between the label pattern and the camera calculated by the robot using the pinhole imaging model is set to have no ranging error. Therefore, step B, calculating the distance from the camera to the rectangular or triangular label, can be performed using the pinhole imaging model, saving computational effort and ensuring ranging accuracy. For example, Figure 4 If the lengths of the first side C1T and the second side D1T are both set to be less than the preset distortion error value, then the distance between the triangular label TC1D1 calculated by the robot using the pinhole imaging model and the camera is set to have no ranging error, and it is considered that neither the first side C1T nor the second side D1T is distorted in the camera.

[0171] Furthermore, if any side of the triangular label to be measured is not perpendicular to the target line, while any side of the rectangular label to be measured is perpendicular to the target line, the robot repeats steps A to D. First, it identifies the triangular labels located on both sides of the rectangular label and calculates the distance between each triangular label and the camera, as well as the deflection angle of the triangular label relative to the camera. Then, it identifies the rectangular label and calculates the distance between the rectangular label and the camera, as well as the deflection angle of the rectangular label relative to the camera. Therefore, in guiding the robot to approach the charging interface from a distance, the robot calculates the distance from the triangular label to the rectangular label using monocular ranging principles, reducing the distance from having ranging errors to eliminating them. This allows for high-precision aligning of the robot with the charging interface represented by the rectangular label.

[0172] It should be noted that if the robot detects that one edge of the label pattern is not parallel to the pinhole plane of the camera, it confirms that this edge, which is not parallel to the pinhole plane of the camera, causes distortion in the camera. Therefore, when the robot detects that the tilt angle of the plane of the object under test is not equal to 0 degrees and is not an integer multiple of 180 degrees, the robot determines that the label pattern in the plane of the object under test is distorted in the camera. If the robot detects that one edge of the label pattern is parallel to the pinhole plane of the camera, it confirms that this edge, which is parallel to the pinhole plane of the camera, does not cause distortion in the camera, but the label pattern in the plane of the object under test may still cause distortion in the camera. This achieves the detection of whether the label pattern in the plane of the object under test is distorted in the camera. Therefore, in the aforementioned embodiment, when the target straight line direction is the horizontal direction passing through the target graphic label, the edge of the label pattern parallel to the horizontal direction is distorted in the camera, and the edge of the label pattern perpendicular to the target straight line direction is not distorted in the camera, corresponding to... Figure 5 The first side to be measured, C2E, and the second side to be measured, D2F, are used to directly calculate the distance between the first and second sides to be measured and the camera without using distortion parameters or distortion formulas commonly used in existing technologies, thus improving calculation efficiency and accuracy.

[0173] Regarding the distortion phenomenon of the label pattern in the camera, it should be noted that, since the height of the intersection point between the principal ray of the label pattern in the plane of the object under test at different tilt angles and the imaging plane of the camera after passing through a monocular camera with a fixed focal length is inconsistent and is not equal to the ideal image height, distortion will occur. Specifically, among the sides of the rectangular label and the triangular label of the object under test at an angle to the pinhole plane, the side that is relatively closer to the optical center of the camera will have a larger imaging size (e.g., imaging width or imaging height) in the imaging plane, and will be taller than the ideal object due to distortion; the side that is relatively farther from the optical center of the camera will have a smaller imaging size (e.g., imaging width or imaging height) in the same imaging plane, and will be shorter than the ideal object due to distortion.

[0174] As an example of calculating ranging error, when the robot identifies two adjacent target detection points on the plane of the object under test, it calculates the product of the distance Ux between the two adjacent target detection points and the sine of the tilt angle of the object under test, Ux*sin(ax). Then, Ux*sin(ax) is set as the ranging error caused by the distortion of the line connecting the two adjacent target detection points in the camera. Furthermore, when the distance between the currently identified target graphic label and the camera, and the deflection angle of the currently identified target graphic label relative to the camera, both meet preset positioning conditions, the ranging error caused by the distortion of the line connecting the two adjacent target detection points in the camera falls within a preset target error range, which includes the value 0. The two adjacent target detection points can originate from previously identified direction correction labels (triangle labels) or previously identified target graphic labels (rectangular labels). When the robot detects that the tilt angle of the plane of the object under test is not equal to 0 degrees, and the tilt angle of the plane of the object under test is not an integer multiple of 180 degrees, the robot determines that the plane of the object under test is not parallel to the pinhole plane of the camera, and determines that the label pattern in the plane of the object under test is distorted in the camera. Therefore, when the charging base plane where the label pattern is located is not parallel to the imaging plane of the camera, the edge of the label pattern is distorted after being captured by the camera. In this embodiment, the size difference (e.g., the difference between the actual image height and the ideal image height) of the edge line where two adjacent target detection points of the label pattern are located before and after distortion is obtained by multiplying the distance Ux between two adjacent target detection points in the plane of the object under test by the sine of the tilt angle of the plane of the object under test. As described in the foregoing embodiments, after the robot determines that the distance between the currently identified target graphic label and the camera meets the preset positioning conditions, the robot adjusts the optical axis of the camera to be perpendicular to the plane of the object under test by rotation. This changes the robot's movement direction to be parallel to the perpendicular direction of the rectangular label at the center of the target graphic label, thus determining that the deflection angle of the currently identified target graphic label relative to the camera meets the preset positioning conditions. Therefore, when both the distance between the currently identified target graphic label and the camera, and the deflection angle of the currently identified target graphic label relative to the camera, meet the preset positioning conditions, the ranging error caused by the distortion of the line connecting two adjacent target detection points in the camera is equal to 0. This indicates that the robot is aligned with the rectangular label at the center of the target graphic label, signifying that the robot has connected to the charging interface. By utilizing the size difference between the edge lines of the adjacent target detection points of the label pattern before and after distortion, the timing of the robot aligning with the rectangular label at the center of the target graphic label is determined. This links the ranging error caused by distortion with the robot's preset positioning conditions, improving the accuracy of the robot contacting and aligning with the rectangular label at the center of the target graphic label.

[0175] In one embodiment, the robot updates the predicted position point by performing step D and moves to the latest predicted position point. This reduces the ranging error caused by the distortion of the label pattern, until the latest determined ranging error caused by the label pattern distortion falls within a pre-set target error range. Furthermore, the latest predicted position point is the location of the charging port on the robot's walking plane, and the robot's movement direction is perpendicular to the plane containing the target graphic label and points towards the target graphic label. The pre-set target error range includes the value 0. This suppresses the error caused by the label pattern distortion, improves the accuracy of distance and angle calculations in step B, and ultimately improves the positioning accuracy of the predicted position point.

[0176] Combination Figure 4 It can be seen that, among two adjacent target detection points C1 and D1, the robot records the plane that passes through the side to be measured where the first detection point C1 is located and is parallel to the pinhole plane Lf of the camera as the first virtual plane Lv1, and the plane that passes through the side to be measured where the second detection point D1 is located and is parallel to the pinhole plane Lf of the camera as the second virtual plane Lv2. Then, the distance between the first virtual plane Lv1 and the second virtual plane Lv2 is set to be equal to the ranging error caused by the distortion of the line connecting the two adjacent target detection points C1 and D1. According to the geometric relationship of triangles, the ranging error is equal to the product of the length U1 of line segment C1D1 and the sine of the tilt angle a1 of the plane where the label pattern is located. When the distance between the first virtual plane Lv1 and the second virtual plane Lv2 is smaller, the ranging error caused by the distortion of the label pattern (including the distortion of the line connecting the two adjacent target detection points C1 and D1) is smaller, and the tilt angle a1 of the plane where the label pattern is located is also smaller.

[0177] Preferably, since the distance between two adjacent target detection points obtained by the robot is so small as to be negligible, it can also be regarded as the ranging error caused by the newly determined label pattern distortion falling within the preset target error range; in fact, it is the distance error caused by the change in the angle between the surface of the object to be tested and the pinhole plane of the camera; therefore, in this embodiment, the ranging error caused by the label pattern distortion is initially reduced by reducing the distance between two adjacent target detection points set on the surface of the object to be tested (specifically the charging base surface), thereby achieving the effect of initially correcting the angle distortion.

[0178] As one embodiment, when the plane of the object to be measured is not parallel to the pinhole plane of the camera, and the robot detects that the side of the rectangular label perpendicular to the target line is parallel to the pinhole plane of the camera, the robot determines that the side of the rectangular label parallel to the horizontal direction is distorted in the camera, while the side to be measured in the rectangular label is not distorted in the camera. The side to be measured in the rectangular label includes the side to be measured at the first detection point and the side to be measured at the second detection point. Since both the side to be measured at the first detection point and the side to be measured at the second detection point are parallel to the pinhole plane of the camera, in step B, when calculating the distance from the camera to the side to be measured at the first detection point of the rectangular label and the distance from the camera to the side to be measured at the second detection point of the rectangular label, the robot determines that the ranging error caused by the distortion of the side to be measured at the first detection point of the rectangular label in the camera is equal to 0, and the ranging error caused by the distortion of the side to be measured at the second detection point of the rectangular label in the camera is equal to 0. Figure 5 It can be seen that the ranging error is caused by the distortion of the line connecting two adjacent target detection points C2 and D2 within a rectangular label. However, the measured side C2E, where the first detection point is located, and the measured side D2F, where the second detection point is located, are not distorted. Therefore, step B, which calculates the distances from the camera to the measured side where the first detection point of the rectangular label is located, and the distances from the camera to the measured side where the second detection point of the rectangular label is located, can be calculated using the pinhole imaging model, saving computational effort and ensuring ranging accuracy.

[0179] Preferably, the charging interface docking method further includes: when the distance between two adjacent target detection points obtained by the robot is less than 0.5*(dx1+dx2), the robot sets the ranging error caused by the distortion of the line connecting the two adjacent target detection points in the camera to be equal to the value 0. Here, dx1+dx2 represents the sum of the distances dx1 from the first side to be measured to the camera and dx2 from the second side to be measured to the camera in the same label pattern, and 0.5*(dx1+dx2) represents half of this sum. If half of this sum is greater than the length of the line connecting the two adjacent target detection points, i.e., greater than the width of the label pattern, then the distance in the label pattern where the line connecting the two adjacent target detection points is located is small enough to be negligible, and the actual imaging width in the imaging plane may not differ much from the ideal imaging width, thus the distortion generated by the label pattern can also be ignored. Alternatively, when the distance between two adjacent target detection points obtained by the robot is less than or equal to 0.5*(dx1+dx2), the smaller the distance between the two adjacent target detection points detected by the robot, the smaller the ranging error caused by the distortion of the line connecting the two adjacent target detection points. If the distance between the two adjacent target detection points obtained by the robot is much less than 0.5*(dx1+dx2), the robot can even ignore the ranging error caused by the label pattern distortion. In summary, before executing the charging interface docking method, the label pattern on the plane of the object to be measured can be set to a relatively small size, thereby minimizing the ranging error caused by distortion when the plane of the object to be measured is not parallel to the imaging plane of the camera, i.e., suppressing the distance error caused by angle changes.

[0180] Based on the above embodiments, if the two vertices of the edge to be tested where the first detection point is located are updated to the two adjacent target detection points, then the edges originally distributed along the target straight line direction in the same triangular label are set as the edge to be tested where one of the updated target detection points is located, and the edge to be tested where the original second detection point is located is set as the edge to be tested where another updated target detection point is located, so that the target straight line direction is updated to be parallel to the edge to be tested where the original first detection point is located; then the distance from the camera to the edge to be tested where one of the updated target detection points is located is set to equal dx1, and the distance from the camera to the edge to be tested where another updated target detection point is located is set to equal dx2; when the distance between the two adjacent target detection points after the update is less than 0.5*(dx1+dx2), the robot sets the ranging error caused by the distortion of the line connecting the two adjacent target detection points in the camera to equal the value 0. Thus, during the execution of the charging interface docking method, the label patterns of the two adjacent target detection points with a distance of less than 0.5*(dx1+dx2) are specifically selected for distance and angle calculation, improving the accuracy of the planar positioning calculation of the object to be tested.

[0181] Combination Figure 4 It can be seen that, in a triangular label, the ranging error caused by the distortion of the line connecting two adjacent target detection points C1 and D1 in the camera is equal to the product of the length U1 of line segment C1D1 and the sine of the tilt angle a1 of the plane containing the label pattern, according to the geometric relationship of triangles. The tilt angle a1 of the plane containing the label pattern is represented by the angle between the plane Lr1 of the object to be measured and the first virtual plane Lv1. The first virtual plane Lv1 is the plane passing through the side TC1 of the target detection point C1 and parallel to the pinhole plane Lf of the camera. The first virtual plane Lv1 and the second virtual plane Lv2 are virtual planes that pass through the edge TD1 where the target detection point D1 is located and are parallel to the pinhole plane Lf of the camera. The product of the length U1 of line segment C1D1 and the sine of the tilt angle a1 of the plane where the label pattern is located is also equal to the distance between the first virtual plane Lv1 and the second virtual plane Lv2. The smaller the tilt angle a1 of the plane where the label pattern is located, and / or the smaller the length U1 of line segment C1D1 is set, the smaller the distance between the first virtual plane Lv1 and the second virtual plane Lv2, and the smaller the ranging error caused by the distortion of the label pattern.

[0182] If the two vertices of the edge C1T where the first detection point C1 is located are updated to the two adjacent target detection points, that is, the two adjacent target detection points are updated to the first detection point C1 and the second detection point T, then the edge C1D1 that was originally distributed along the target line direction in the same triangle label is set as the edge to be tested where the first detection point C1 is located, and the edge D1T where the original second detection point D1 is located is set as the edge to be tested where the updated target detection point T is located, so that the target line direction is updated to be parallel to the edge C1T where the original first detection point is located; then the distance from the camera to the edge to be tested where the updated target detection point C1 is located is set to be equal to dx1, and the distance from the camera to the edge to be tested where the updated target detection point T is located is set to be equal to dx2; when the distance between the updated two adjacent target detection points C1 and T is less than 0.5*(dx1+dx2), the robot sets the ranging error caused by the distortion of the line C1T connecting the updated two adjacent target detection points in the camera to be equal to the value 0.

[0183] Similarly, when the updated two adjacent target detection points are D1 and T respectively, and the distance between the two adjacent target detection points D1 and T is less than 0.5*(dx1+dx2), the robot sets the ranging error caused by the distortion of the line D1T connecting the updated two adjacent target detection points in the camera to a value of 0. Therefore, it can be determined that the preset distortion error value disclosed in the aforementioned embodiment is equal to 0.5*(dx1+dx2).

[0184] In summary, during the execution of the charging interface docking method, the label patterns containing two adjacent target detection points with a distance of less than 0.5*(dx1+dx2) are specifically selected for distance and angle calculation, thereby improving the accuracy of planar positioning calculation of the object under test.

[0185] It should be noted that the pinhole plane of the camera is parallel to the imaging plane of the camera; the optical center of the camera is located in the pinhole plane, and the distance from the optical center to the imaging plane is equal to the focal length of the camera. Specifically, the incident light from the target detection point corresponding to each triangular label intersects the optical center of the camera; the target detection points corresponding to each triangular label are distributed along a straight line in the plane of the object to be measured, and are identified sequentially during the repeated execution of step A by the robot, and then used in step B to calculate the ranging error caused by the distortion of the label pattern. The ranging error here includes the ranging error caused by the distortion of the line connecting two adjacent target detection points in the camera. Of course, only two adjacent target detection points corresponding to the same label pattern are needed to calculate this ranging error.

[0186] As one embodiment, in step A, the preprocessing of the image captured by the robot's camera includes: first, the robot converts the captured image to grayscale to obtain a binarized image; then, the binarized image is Gaussian smoothed to remove image noise, resulting in a smoothed image; then, edge detection is performed on the smoothed image to obtain a target edge map, preferably using the Canny operator. The robot can extract closed curves from the target edge map, and these closed curves can form the closed shape disclosed in the aforementioned embodiment. The target edge map is then dilated to obtain the preprocessed image, which allows the robot to identify the vertices and graphic attributes of the label pattern from the preprocessed image. Based on the graphic attributes of the label pattern, it is possible to distinguish... Figure 3 The preprocessed orientation correction labels and target graphic labels are included. Figure 3 The pre-processed label pattern on the left is a regular polygonal graphic arranged with a pre-processed rectangular label (a black square surrounded by a white closed edge line) as the center of symmetry. This regular polygonal graphic is composed of multiple identical black squares surrounded by white closed edge lines, which is the pre-processed target graphic label. Figure 3The preprocessed label pattern on the right consists of a preprocessed triangular label (a black triangle surrounded by a white closed edge line), which is the preprocessed orientation correction label. Before and after preprocessing, the positional relationship between the target graphic label and each orientation correction label remains unchanged, and the shape and size of each label pattern change only slightly. The only difference is that the preprocessed edge lines are smoother and sharper, enabling clear imaging on the camera's imaging plane. This overcomes the problems of image blurring and noise interference within specific lighting conditions. Therefore, a single monocular camera with a fixed focal length can achieve clear imaging, capturing all vertices of the label pattern at once, thereby improving the accuracy of calculating the length of the edges connecting the vertices.

[0187] Preferably, in step A, the method for searching for the vertices of the label pattern in the preprocessed image includes: extracting a closed shape in the preprocessed image so that the closed shape represents the label pattern that the robot needs to search for; wherein the closed shape is generated in the edge detection; then extracting the vertices of the label pattern and the graphic attributes of the label pattern based on the closed shape, and then filtering the extracted vertices of the label pattern and the graphic attributes of the label pattern to suppress noise in the label pattern; then setting the filtered vertices as the vertices of the label pattern searched by the robot in the preprocessed image, and setting the filtered graphic attributes as the graphic attributes of the label pattern searched by the robot in the preprocessed image, so as to distinguish labels of different shapes. In this embodiment, the filtered graphic attributes are reflected in the white lines in step 3 as clear and sharp edge imaging.

[0188] Preferably, the closed shape is formed by multiple line segments connected end-to-end, or by multiple pixels connected sequentially. Pixels at the corners of the closed shape are identified as vertices. The graphic attributes of the label pattern are the edge features of the closed shape, including the number of sides and vertices. In the closed shape, the robot identifies the line connecting two adjacent vertices as a side forming the closed shape. When the robot searches for the graphic attributes and vertices of the label pattern in the preprocessed image, it distinguishes different shapes of label patterns by detecting the number of sides forming the closed shape. Therefore, in the process of detecting the number of sides of a closed shape in the acquired image, when the robot detects three sides, it identifies the currently detected closed shape as a triangular label and determines it as a direction-corrected label; when the robot detects four sides, it identifies the currently detected closed shape as a rectangular label, thus distinguishing different shapes of labels based on the graphic attributes of the label pattern.

[0189] This application also discloses a charging system, which includes a robot and the charging device disclosed in the foregoing embodiments. The robot is configured to execute the tag recognition-based charging interface docking method disclosed in the foregoing embodiments, so that the robot docks with the charging interface and charges. The robot is equipped with a monocular camera for collecting multiple tags set on the surface of the charging device, including collecting multiple tags of different sizes and shapes. The specific shape and size characteristics can be referred to the embodiments corresponding to the foregoing charging device, which will not be repeated here.

[0190] Compared with existing technologies, this charging system controls the robot to use the vertices of the identified label pattern to calculate the distance and angle information of the label relative to the same camera, thereby predicting the relative positional relationship between the charging interface and the camera. This provides accurate positioning information for the robot to navigate back to the charging device for docking and charging. In the process of iteratively processing the distance between the label pattern and the camera, as well as the deflection angle of the label pattern relative to the camera, the robot's movement direction is corrected to the orientation of the charging interface represented by the target graphic label. This improves the visual positioning accuracy of the camera and the accuracy of docking and charging as the robot approaches the charging interface.

[0191] The charging device includes a charging interface for the robot. The surface of the charging device features a target graphic label and multiple orientation correction labels. The positions of these labels can be captured by an external camera and displayed visually on the camera's imaging plane. In some embodiments, they are grouped into a set of label patterns or presented as QR codes on the surface of the charging device. The target graphic label indicates the charging interface. Multiple orientation correction labels are distributed on both sides of the target graphic label to guide the robot from the orientation correction labels to the target graphic label. Preferably, the multiple orientation correction labels are evenly distributed on both sides of a target graphic label, and each orientation correction label is of the same type as the target graphic label in the charging device, but of a different type. The multiple orientation correction labels can be placed at equal intervals near the charging interface, and the charging interface itself also has a corresponding target graphic label, forming orientation labels to guide the robot back to charging. When the robot systematically traverses the various orientation correction labels in a single direction, it can be distinguished whether the robot is approaching or moving away from the charging interface, thus correcting the robot's movement direction. The docking structure of the robot's charging electrodes and the charging interface is adapted to each other. Therefore, in the process of iteratively processing the distance between the label pattern and the camera, as well as the deflection angle of the label pattern relative to the camera, the robot's movement direction is corrected to the orientation of the charging interface represented by the target graphic label, so that the visual positioning accuracy of the camera and the accuracy of docking and charging are improved when the robot approaches the charging interface.

[0192] The vertical direction of the plane (surface of the charging device) where the target graphic label is located indicates the orientation of the charging interface. Specifically, the charging device uses multiple orientation correction labels and a target graphic label on its surface to design the position information of the charging interface for the robot. This allows the robot to accurately dock with the charging interface after being guided or corrected by the other label patterns, overcoming the influence of the camera's ranging error. Especially when the surface of the charging device under test is not perpendicular to the optical axis of the monocular camera, the robot gradually approaches the charging interface by relying on the guidance direction formed by the specific arrangement of the target graphic label and the orientation correction label, which helps improve the ranging accuracy of the monocular camera installed on the robot for the charging device.

[0193] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A charging device, wherein the charging device is provided with a charging interface for charging a robot, characterized in that, The surface of the charging device is provided with a target graphic label and multiple orientation correction labels; The target graphic label is used to represent the label of the charging interface of the charging device. Multiple directional correction labels are distributed on both sides of the target graphic label to guide the robot to move from the directional correction labels on both sides to the target graphic label. The vertical direction of the plane where the target graphic label is located indicates the orientation of the charging interface, which corresponds to the direction in which the robot docks with the charging interface. A target graphic label is a regular polygonal graphic arranged with a rectangular label as its center of symmetry. Within the target graphic label, there are no rectangular labels distributed in the neighborhood of its center of symmetry, so that all rectangular labels distributed around its center of symmetry do not exceed one circle. Within the target graphic label, the number of rectangular labels distributed in the neighborhood on both sides of its center of symmetry is different. The charging interface includes charging electrodes for the charging device, which include a positive electrode and a negative electrode, respectively positioned directly below the target graphic label.

2. The charging device according to claim 1, characterized in that, The orientation correction label and the target graphic label are set on the upright surface of the charging device along the target straight line direction; The target straight line direction is set as the extension direction of the surface of the charging device on the horizontal ground, and the target straight line direction is set as parallel to the horizontal ground; when the charging device is placed on the horizontal ground, the vertical surface of the charging device is set at an angle to the horizontal ground.

3. The charging device according to claim 2, characterized in that, The orientation correction labels on one side of the target graphic label are arranged differently from those on the other side of the target graphic label. The directional correction labels on each side of the target graphic label are evenly spaced on the surface of the charging device; The charging interface is located below the target graphic label to accommodate the height of the robot's charging electrodes.

4. The charging device according to claim 2, characterized in that, A direction correction label consists of a triangle label, with the two vertices of the triangle label forming one side of the triangle label, and the number of sides forming the triangle label is 3. A target graphic label is composed of multiple identical rectangular labels. The vertical edge of a rectangular label is formed by connecting two adjacent vertices in the vertical direction, and the horizontal edge of a rectangular label is formed by connecting two adjacent vertices in the horizontal direction. Both the vertical edge and the horizontal edge of a rectangular label are considered as sides of the label, and the number of sides that make up the rectangular label is 4. The target straight line direction is parallel to the base of the triangular label, and the target straight line direction is parallel to the horizontal edge of the rectangular label.

5. The charging device according to claim 4, characterized in that, The target graphic label contains three rows of rectangular labels. Each row has three rectangular labels passing through the center of symmetry. One rectangular label is located at the center of symmetry of the target graphic label and is parallel to the central axis of the charging device. The other two rectangular labels are located on either side of the center of symmetry of the target graphic label. The row passing through the center of symmetry of the target graphic label is called the middle row. The row above the middle row has two rectangular labels, which are placed in the same column as the rectangular label at the center of symmetry and the rectangular label on one side of it, respectively. The row below the middle row has two rectangular labels, which are placed in the same column as the rectangular label at the center of symmetry and the rectangular label on the other side of it, respectively. The directional correction labels set on one side of the target graphic label and the directional correction labels set on the other side of the target graphic label are centrally symmetrical about the center of symmetry of the target graphic label.

6. The charging device according to claim 4, characterized in that, Within the vertical surface, the coverage area of ​​a triangular label is greater than the coverage area of ​​a rectangular label; None of the sides of the triangular label are perpendicular to the target line direction, while each rectangular label has two parallel sides that are perpendicular to the target line direction, so that multiple triangular labels are distributed on both sides of the rectangular label.

7. A charging interface docking method based on tag recognition, characterized in that, The charging interface docking method is used to control the robot to position and dock with the charging interface of the charging device according to any one of claims 1 to 6; Charging interface connection methods include: Step A: The robot preprocesses the images captured by its camera, and then searches for the graphic attributes and vertices of the label pattern in the preprocessed images, wherein the label pattern is a target graphic label or a direction correction label. Step B: Based on the graphic attributes and vertices of the label pattern, calculate the distance between the label pattern and the camera, as well as the deflection angle of the label pattern relative to the camera, using the monocular ranging principle. Step C: Set the predicted position point based on the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera; Step D: The robot moves to the predicted position point, and then uses the camera to capture an image of the label pattern. Then, steps A to D are repeated until the latest predicted position point is the position point occupied by the charging interface in the robot's walking plane. The robot's movement direction is parallel to the vertical direction of the plane where the target graphic label is located so that the robot can dock with the charging interface.

8. The charging interface docking method according to claim 7, characterized in that, Step C specifically includes: determining whether the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera, meet the preset positioning conditions. If yes, no new predicted position point is set; otherwise, a new predicted position point is set. The preset positioning conditions include the latest predicted position point being the position point occupied by the charging interface in the robot's walking plane, and the robot's movement direction being the perpendicular direction to the plane where the target graphic label is located and pointing towards the target graphic label.

9. The charging interface docking method according to claim 8, characterized in that, In step A, the robot identifies the orientation correction label based on the graphic attributes of the label pattern, so that the currently identified orientation correction label represents the label pattern, and is used to repeat steps A to D. The robot identifies the target graphic label at the last predicted location point, so that the currently identified target graphic label represents the label pattern, and is used to execute step D.

10. The charging interface docking method according to claim 9, characterized in that, Before recognizing the target graphic label, during the process of the robot performing steps A to D, it sequentially traverses each of the recognized directional correction labels to guide the robot to move from the recognized directional correction labels on both sides to the unrecognized area in the middle. The robot identifies a rectangular label at the center of the target graphic label at the last determined predicted position point, and determines that the distance between the rectangular label and the camera meets the preset positioning conditions, thus confirming the existence of the target graphic label in the unidentified area in the middle, and determining that there are multiple orientation correction labels distributed on both sides of the target graphic label. The target graphic label is composed of multiple rectangular labels. Moreover, the robot adjusts its pose so that the robot's movement direction is parallel to the vertical direction of the plane where the target graphic label is located and points towards the target graphic label.

11. The charging interface docking method according to claim 10, characterized in that, When the robot recognizes the orientation correction label, it does not recognize the rectangular label, and therefore does not recognize the target graphic label. At this point, the robot is at the first predicted position. When the robot recognizes the rectangular label, it does not recognize the orientation correction label. At this point, the robot is at the second predicted position. The distance between the first predicted location point and the charging interface is greater than the distance between the second predicted location point and the charging interface; the direction correction label does not include rectangular labels. In this case, the size of one orientation correction label on the surface of the charging device is larger than the size of any rectangular label included within the target graphic label on the surface of the charging device.

12. The charging interface docking method according to claim 10, characterized in that, In step A, the robot identifies the orientation correction label and / or the target graphic label from multiple label patterns at once based on the graphic attributes of the label pattern, so as to obtain the vertices of the identified label pattern and the graphic attributes of the identified label pattern. Whenever multiple orientation correction labels are identified, in step B, based on the graphic attributes and vertices of each orientation correction label, the distance between each orientation correction label and the camera, as well as the deflection angle of each orientation correction label relative to the camera, are calculated using the monocular ranging principle. Then, the distance between each identified directional correction label and the camera is traversed sequentially. When the robot determines that the distance between the two directional correction labels with different placement patterns located on both sides of the middle position and the closest distance is not the two smallest distances among the identified directional correction labels and the camera, it determines that the distance between each identified label pattern and the camera does not meet the preset positioning conditions.

13. The charging interface docking method according to claim 12, characterized in that, In step C, the robot sets a predicted position point in front of the two labels with different orientations that are located on both sides of the middle position and are closest to each other. Then, the robot adjusts its pose and moves to the predicted position point based on the deflection angle of the currently set predicted position point relative to the camera, so as to shorten the distance between the robot and the target graphic label. The robot repeats steps A to C until it determines that the distance between the two directional correction labels, which are located on both sides of the middle position and are closest to each other, and the camera is the two sets of distances with the smallest values ​​among the distances between the identified directional correction labels and the camera. Then, the robot moves to the currently set predicted position point and identifies the target graphic label in step A. Before the robot identifies the target graphic label in step A, the distance between each currently identified label pattern and the camera is not allowed to meet the preset positioning condition.

14. The charging interface docking method according to claim 13, characterized in that, After the robot identifies the target graphic label in step A, it then identifies the individual rectangular labels that make up the target graphic label. Then, in step B, based on the graphic attributes and vertices of each rectangular label, the distance between each rectangular label and the camera, as well as the deflection angle of each rectangular label relative to the camera, are calculated using the monocular ranging principle. Then, iterate through the distances between each identified rectangular label and the camera; When the robot determines in step C that the distance between the rectangular label at the center of the target graphic label and the camera is not the smallest among the distances between the identified rectangular labels and the camera, it determines that the distance between the currently identified target graphic label and the camera does not meet the preset positioning conditions; then in step D, the robot moves towards the rectangular label at the center of the target graphic label, until the distance between the rectangular label at the center of the target graphic label and the camera is the smallest among the distances between the identified rectangular labels and the camera, and determines that the distance between the currently identified target graphic label and the camera meets the preset positioning conditions, wherein the currently moved position is the latest determined predicted position. Then, at the newly determined predicted position point, the robot's movement direction is adjusted to be parallel to the vertical direction of the rectangular label at the center position of the target graphic label, and the distance between the currently identified target graphic label and the camera, as well as the deflection angle of the currently identified target graphic label relative to the camera, are determined to meet the preset positioning conditions.

15. The charging interface docking method according to claim 7, characterized in that, The robot recognizes an orientation correction label as consisting of a triangular label and determines the side length of the triangular label by its vertex; the arrangement of the orientation correction labels on one side of the target graphic label is different from the arrangement of the orientation correction labels on the other side of the target graphic label. The directional correction labels on each side of the target graphic label are evenly spaced along a straight line on the surface of the charging device. The robot identifies a target graphic label as being composed of multiple identical rectangular labels arranged together, and determines the side length of the rectangular labels by their vertices. A target graphic label is a regular polygonal graphic arranged with a rectangular label as its center. Within the target graphic label, the number of rectangular labels distributed in the two neighboring areas on both sides of its center position is different, so as to distinguish the two sides of the target graphic label.

16. The charging interface docking method according to claim 15, characterized in that, Methods for identifying orientation correction labels and / or target graphic labels from multiple label patterns at once based on the graphic attributes of label patterns include: When the robot searches for the graphic attributes and vertices of the label pattern in the preprocessed image, it detects the number of edges that form a closed shape. The graphic attributes of the label pattern are the edge features of the closed shape, which include the number of edges and vertices of the closed shape. In the edge lines of the closed shape, the robot identifies the line connecting two adjacent vertices as an edge that forms a closed shape. When the robot detects that there are 3 sides forming a closed shape, it identifies the currently detected closed shape as a triangle label and determines the orientation correction label. When the robot detects that there are 4 sides forming a closed shape and there are two sets of sides that are perpendicular to each other, the currently detected closed shape is identified as a rectangular label. Each set has two parallel sides. If the number of rectangular labels detected by the robot in the same frame is the total number of rectangular labels required to form a target graphic label, and the cumulatively detected rectangular labels are symmetrically arranged with one of the rectangular labels as the center position, and the number of rectangular labels distributed in the neighborhood on both sides of the center position is different, then a target graphic label is identified. The target straight line is parallel to the base of the triangular label, and the target straight line is parallel to the horizontal edge of the rectangular label.

17. The charging interface docking method according to claim 16, characterized in that, In the label pattern, the robot records the edge that forms a certain angle with the target straight line as the edge to be tested in the label pattern; the distance between the label pattern and the camera in step B includes the distance from the edge to be tested where the first detection point is located to the camera and the distance from the edge to be tested where the second detection point is located to the camera. The monocular ranging principle described in step B includes the pinhole imaging model; Methods for calculating the distance between a label pattern and a camera using a pinhole imaging model include: The lens focal length f of the camera, the side length w of the side to be measured, and the pixel width p formed by the side to be measured in the imaging plane of the camera are obtained in advance. The distance between the edge to be measured and the camera is calculated using the following formula: If one endpoint of the edge to be tested is the first detection point, then the edge to be tested is the edge to be tested where the first detection point is located, and then d is set to be equal to the distance dx1 from the camera to the edge to be tested where the first detection point is located; If one endpoint of the edge to be tested is the second detection point, then the edge to be tested is the edge to be tested where the second detection point is located, and then d is set to be equal to the distance dx2 from the camera to the edge to be tested where the second detection point is located; Wherein, the plane of the object to be tested is the surface of the charging device where the label pattern is located; when the plane of the object to be tested is not parallel to the pinhole plane of the camera, the intersection line of the plane of the object to be tested and the pinhole plane of the camera is set perpendicular to the straight line direction of the target.

18. The charging interface docking method according to claim 17, characterized in that, In step B, the method for calculating the distance between the label pattern and the camera, and the deflection angle of the label pattern relative to the camera, using the monocular ranging principle includes: The robot sets the target straight line direction as the extension direction of the plane of the object under test on the horizontal plane, and sets the target straight line direction to be parallel to the robot's walking plane; The robot sets two vertices of a label pattern distributed along the target straight line as two adjacent target detection points in the plane of the object to be tested; the distance Ux between the two adjacent target detection points in the plane of the object to be tested is obtained in advance; the plane of the object to be tested represents the surface of the charging device where the label pattern is located; Among the two adjacent target detection points, the robot records one of the target detection points as the first detection point and uses the pinhole imaging model to calculate the distance dx1 from the edge to be tested where the first detection point is located to the camera; the robot records the other target detection point as the second detection point and uses the pinhole imaging model to calculate the distance dx2 from the edge to be tested where the second detection point is located to the camera. Based on the side-angle relationships of a triangle, the deflection angle of the label pattern relative to the camera is calculated using the following formula: ； ； The angle between the perpendicular segment from the camera to the side to be tested where the first detection point is located and the pinhole plane of the camera in the opposite direction of the target straight line is denoted as bx11, and the angle between the perpendicular segment from the camera to the side to be tested where the second detection point is located and the pinhole plane of the camera in the target straight line is denoted as bx21. The robot sets the angle between the plane of the object to be tested and the pinhole plane of the camera in the target straight line direction as the tilt angle of the plane of the object to be tested, where ax is the tilt angle of the plane of the object to be tested; the tilt angle of the plane where the label pattern is located is the tilt angle of the plane of the object to be tested. The deflection angle of the label pattern relative to the camera includes the angle bx12 between the perpendicular segment of the camera to the side to be tested where the first detection point is located and the optical axis, and the angle bx22 between the perpendicular segment of the camera to the side to be tested where the second detection point is located and the optical axis.

19. The charging interface docking method according to claim 18, characterized in that, When the label pattern is represented as a triangular label, the two adjacent target detection points are the two vertices of the base of the triangular label. The target detection points corresponding to each triangular label are distributed along the target straight line in the plane of the object to be measured. The line connecting the two vertices of a triangular label that are distributed along the first angle with the target straight line is called the side to be measured where the first detection point is located. The line connecting the two vertices of the same triangular label that are distributed along the second angle with the target straight line is called the side to be measured where the second detection point is located. The sum of the second angle and the first angle is equal to 180 degrees. When the label pattern includes a rectangular label, the two adjacent target detection points are two vertices of the side of the rectangular label that is parallel to the target straight line direction, and the two sides of the rectangular label that are perpendicular to the target straight line direction are the test side where the first detection point is located and the test side where the second detection point is located, respectively.

20. The charging interface docking method according to claim 19, characterized in that, If any side of the triangular label to be measured is not perpendicular to the direction of the target line, then any side of the triangular label to be measured will be distorted in the camera, causing a distance measurement error in the distance between the triangular label and the camera calculated by the robot using the pinhole imaging model. If any side of the rectangular label to be measured is perpendicular to the direction of the target line, then any side of the rectangular label to be measured will not be distorted in the camera. If the length of the side to be measured where the first detection point of the label pattern is located and the length of the side to be measured where the second detection point of the same label pattern is located are both preset to be less than the preset distortion error value, then the distance between the label pattern and the camera calculated by the robot using the pinhole imaging model is set to have no distance measurement error.

21. The charging interface docking method according to claim 20, characterized in that, If any side of the triangular label to be tested is not perpendicular to the target straight line, and any side of the rectangular label to be tested is perpendicular to the target straight line, the robot repeats steps A to D to first identify each triangular label located on both sides of the rectangular label and calculate the distance between each triangular label and the camera, as well as the deflection angle of the triangular label relative to the camera. Then, it identifies the rectangular label and calculates the distance between the rectangular label and the camera, as well as the deflection angle of the rectangular label relative to the camera.

22. The charging interface docking method according to claim 17, characterized in that, In step C, the robot selects the region formed by the angles formed by the perpendicular segments of the perpendicular lines from the camera to the sides of the identified triangular labels with different arrangements on both sides of the unidentified area, and then sets the predicted position point within the region formed by the selected angle with the smallest angle. Each time step C is executed, the angle with the smallest selected angle by the robot is updated so that the predicted position point is updated, thereby guiding the robot to move from the identified triangular labels on both sides to the unidentified area in the middle. The distance between a triangular tag and the camera includes the distance from the camera to the side to be measured where the first detection point of the triangular tag is located, and the distance from the camera to the side to be measured where the second detection point of the triangular tag is located, which constitutes a set of distances for a triangular tag.

23. The charging interface docking method according to claim 22, characterized in that, After the robot identifies the rectangular label in the target graphic label, during step D, as it moves towards the rectangular label closer to the center of the target graphic label, if the robot is detected to be to the left of the rectangular label at the center of the target graphic label based on the current deflection angle of the identified rectangular label relative to the camera, then the robot moves to the right to the new predicted position point; or, if the robot is detected to be to the right of the rectangular label at the center of the target graphic label based on the current deflection angle of the identified rectangular label relative to the camera, then the robot moves to the left to the new predicted position point; until the robot is located at the newly determined predicted position point, indicating it is near the center of the target graphic label. The distance between the rectangular label at the center position and the camera is the smallest among all the distances between the identified rectangular labels and the camera, and it is determined that the distance between the currently identified target graphic label and the camera meets the preset positioning conditions. While determining that the distance between the currently identified target graphic label and the camera meets the preset positioning conditions, the robot adjusts the optical axis of the camera to be perpendicular to the plane of the object under test by rotating, so that the robot's movement direction becomes parallel to the vertical direction of the rectangular label at the center position of the target graphic label and points towards the rectangular label, thereby determining that the deflection angle of the currently identified target graphic label relative to the camera meets the preset positioning conditions. The distance between a rectangular label and a camera includes the distance from the camera to the side to be measured where the first detection point of the rectangular label is located, and the distance from the camera to the side to be measured where the second detection point of the rectangular label is located, which constitutes a set of distances for a rectangular label. The distance between the target graphic label and the camera includes the distances between all the rectangular labels required to make up the target graphic label and the camera.

24. The charging interface docking method according to claim 19, characterized in that, When the robot identifies two adjacent target detection points on the plane of the object under test, it calculates the product Ux of the distance Ux between the two adjacent target detection points on the plane of the object under test and the sine of the tilt angle of the plane of the object under test. sin(ax), then Ux sin(ax) is set as the ranging error caused by the distortion of the line connecting two adjacent target detection points in the camera; If the robot detects that there is an edge in the label pattern that is not parallel to the pinhole plane of the camera, it confirms that the edge that is not parallel to the pinhole plane of the camera is distorted in the camera, and thus determines that the label pattern is not parallel to the pinhole plane of the camera and that the label pattern is distorted in the camera. If the robot detects that there is an edge in the label pattern that is parallel to the pinhole plane of the camera, it confirms that the edge parallel to the pinhole plane of the camera will not cause distortion in the camera.

25. The charging interface docking method according to claim 24, characterized in that, The robot updates the predicted position point by performing step D and moves to the latest predicted position point, so as to reduce the ranging error caused by the distortion of the acquired label pattern. Until the latest predicted position point is the position point occupied by the charging interface in the robot's walking plane, and the robot's movement direction is perpendicular to the plane where the target graphic label is located and points towards the target graphic label, the ranging error caused by the distortion of the line connecting two adjacent target detection points in the camera falls within the preset target error range, where the preset target error range includes the value 0.

26. The charging interface docking method according to claim 25, characterized in that, The distance between two adjacent target detection points obtained by the robot is less than 0.

5. When (dx1+dx2), the robot sets the ranging error caused by the distortion of the line connecting two adjacent target detection points in the camera to be equal to the value 0.

27. A charging system, characterized in that, The charging system includes a robot and a charging device. The robot is configured to perform the tag-based charging interface docking method as described in claim 7, so that the robot docks with the charging interface and performs charging. The robot is equipped with a monocular camera for collecting data on multiple tags set on the surface of the charging device.

Images