Robot recharging positioning method and device, foot type robot and storage medium

Abstract

The embodiment of the invention provides a robot recharging positioning method, a robot recharging positioning device, a foot type robot and a storage medium, which are used for realizing accurate positioning and stable butt joint in an autonomous recharging process of the robot. The robot recharging positioning method comprises the following steps: acquiring image data acquired by a robot in a process of moving to a charging pile; identifying positioning identification information in the image data; when at least two pieces of positioning identification information are identified, relative external parameters between different pieces of positioning identification information are determined based on corresponding position information of the at least two pieces of positioning identification information in the image data; the at least two pieces of positioning identification information comprise at least one piece of first positioning identification information and at least one piece of second positioning identification information; when at least one piece of first positioning identification information is identified and the second positioning identification information is not identified, determining target pose information of the at least one piece of second positioning identification information relative to the robot based on corresponding position information of the first positioning identification information in the image data and the relative external parameters; and generating recharging path information according to the target pose information.

Description

Technical Field

[0001] This disclosure relates to the field of robotics, and in particular to a robot recharging and positioning method, apparatus, legged robot, and storage medium. Background Technology

[0002] Autonomous recharging is a core component ensuring the continuous outdoor operation of legged robots. The key lies in accurately locating the charging station and navigating to the charging interface to complete automatic docking and ensure the robot's continuous operation capability.

[0003] Currently, some wheeled and tracked mobile devices, based on the stable characteristics of their translational movement, can achieve recharging and positioning by recognizing a single identifier on a charging station. However, because legged robots employ a multi-jointed, multi-degree-of-freedom limb structure, their movement is affected by joint linkages and terrain adaptation, making the robot prone to swaying. This makes it difficult for them to stably recognize the single identifier on the charging station, leading to positioning deviations and charging docking failures during recharging.

[0004] Therefore, there is an urgent need for a recharging and positioning method adapted to legged robots to achieve accurate positioning and stable docking during their autonomous recharging process. Summary of the Invention

[0005] This disclosure provides a robot recharging and positioning method, device, legged robot, and storage medium to achieve accurate positioning and stable docking during the robot's autonomous recharging process.

[0006] In a first aspect, a robot recharging and positioning method is provided, comprising: acquiring image data collected by the robot during its movement toward a charging station; identifying positioning identifier information in the image data; when at least two positioning identifiers are identified, determining relative extrinsic parameters between different positioning identifiers based on the position information corresponding to the at least two positioning identifiers in the image data; the at least two positioning identifiers include at least one first positioning identifier and at least one second positioning identifier; when at least one first positioning identifier is identified but no second positioning identifier is identified, determining the target pose information of at least one second positioning identifier relative to the robot based on the position information corresponding to the first positioning identifier in the image data and the relative extrinsic parameters; and generating recharging path information based on the target pose information.

[0007] The robot recharging and positioning method of this disclosure collects image data during the robot's movement toward the charging station. If at least two positioning markers are simultaneously identified from the image data (for example, multiple positioning markers can generally be identified simultaneously when the robot is far from the charging station), the relative extrinsic parameters between the different markers are calibrated. If the second positioning marker (such as the charging base marker) becomes invisible due to robot body shaking or close proximity, the target pose of the second positioning marker can be deduced based on the identified first positioning marker, thus achieving continuous positioning in shaking or close-range scenarios.

[0008] In one embodiment, identifying the positioning identifier information in the image data includes: identifying the positioning identifier information in the image data according to a preset period, or after each acquisition of image data collected by the robot; determining the relative extrinsic parameters between different positioning identifier information includes: updating the relative extrinsic parameters between the different positioning identifier information based on the position information corresponding to the at least two positioning identifier information in the image data each time at least two positioning identifier information is identified; determining the target pose information of at least one second positioning identifier information relative to the robot includes: after the second positioning identifier information cannot be identified, after at least one first positioning identifier information is identified, updating the target pose information of at least one second positioning identifier information relative to the robot based on the position information corresponding to the first positioning identifier information in the image data and the relative extrinsic parameters.

[0009] The above-described implementation method, by updating relative extrinsic parameters in real time or periodically, can automatically compensate for minor shifts in the position of positioning markers caused by vibrations or uneven ground. Compared to pre-calibrated fixed extrinsic parameters, this method better ensures positioning accuracy. Furthermore, updating the target pose of the second positioning marker each time the first positioning marker is identified ensures that the target pose is synchronized with the actual position during robot movement, avoiding docking errors caused by pose lag and improving the dynamic adaptability of the recharge path. In addition, this continuous updating mechanism of extrinsic parameters and target pose enables the robot to cope with dynamic interference such as changes in ambient light and slight occlusion, avoiding the accumulation of single identification errors, further reducing the risk of recharge failure, and enhancing positioning robustness.

[0010] In one embodiment, determining the relative extrinsic parameters between different positioning identifiers based on the position information corresponding to the at least two positioning identifiers in the image data includes: determining a first extrinsic parameter of at least one first positioning identifier relative to the camera coordinate system and a second extrinsic parameter of at least one second positioning identifier relative to the camera coordinate system based on the pixel coordinates corresponding to the at least two positioning identifiers in the image data, the camera's intrinsic parameters, and the three-dimensional coordinates of the at least two positioning identifiers in their own coordinate systems; determining the relative extrinsic parameters between the first positioning identifier and the second positioning identifier based on the first extrinsic parameter and the second extrinsic parameter; determining the target pose information of at least one second positioning identifier relative to the robot based on the position information corresponding to the first positioning identifier in the image data and the relative extrinsic parameter includes: determining a third extrinsic parameter of the first positioning identifier relative to the camera coordinate system based on the pixel coordinates of the first positioning identifier in the image data, the camera's intrinsic parameters, and the three-dimensional coordinates of the first positioning identifier in its own coordinate system; determining the target pose information of the second positioning identifier relative to the robot based on the third extrinsic parameter and the relative extrinsic parameter.

[0011] In one embodiment, determining at least one first extrinsic parameter of the first positioning identifier information relative to the camera coordinate system and at least one second extrinsic parameter of the second positioning identifier information relative to the camera coordinate system includes: based on the pixel coordinates of the first positioning identifier information in the image data, the intrinsic parameters of the camera, and the three-dimensional coordinates of the first positioning identifier information in its own coordinate system, solving for the first initial extrinsic parameter of the first positioning identifier information relative to the camera coordinate system using a pose estimation algorithm, and performing nonlinear optimization processing on the first initial extrinsic parameter to obtain the first extrinsic parameter of the first positioning identifier information relative to the camera coordinate system; based on the pixel coordinates of the second positioning identifier information in the image data, the intrinsic parameters of the camera, and the three-dimensional coordinates of the second positioning identifier information in its own coordinate system, solving for the second initial extrinsic parameter of the second positioning identifier information relative to the camera coordinate system using a pose estimation algorithm, and performing nonlinear optimization processing on the second initial extrinsic parameter to obtain the second extrinsic parameter of the second positioning identifier information relative to the camera coordinate system.

[0012] In one embodiment, the nonlinear optimization of the first initial extrinsic parameters to obtain the first extrinsic parameters of the first positioning identifier information relative to the camera coordinate system includes: constructing a first multiple projection error term based on the first initial extrinsic parameters, and iteratively optimizing the first multiple projection error term until the error change is less than a preset error threshold or a preset maximum number of iterations is reached, thereby obtaining the first extrinsic parameters of the first positioning identifier information relative to the camera coordinate system; the nonlinear optimization of the second initial extrinsic parameters to obtain the second extrinsic parameters of the second positioning identifier information relative to the camera coordinate system includes: constructing a second multiple projection error term based on the second initial extrinsic parameters, and iteratively optimizing the second multiple projection error term until the error change is less than the preset error threshold or the preset maximum number of iterations is reached, thereby obtaining the second extrinsic parameters of the second positioning identifier information relative to the camera coordinate system; wherein, when iteratively optimizing the first multiple projection error term and the second multiple projection error term, manifold constraints are applied.

[0013] In one implementation, constructing the first projection error term includes: based on the pixel coordinates of the acquired first positioning identifier information in the image data, the camera's intrinsic parameters, the three-dimensional coordinates of the first positioning identifier information in its own coordinate system, and the camera projection function, using the extrinsic parameters of the first positioning identifier information relative to the camera coordinate system as optimization variables, constructing the sum of squared residuals between the pixel coordinates of the first positioning identifier information and the predicted coordinates obtained by projection through the camera projection function, to obtain the first projection error term; constructing the second projection error term includes: based on the pixel coordinates of the acquired second positioning identifier information in the image data, the camera's intrinsic parameters, the three-dimensional coordinates of the second positioning identifier information in its own coordinate system, and the camera projection function, using the extrinsic parameters of the second positioning identifier information relative to the camera coordinate system as optimization variables, constructing the sum of squared residuals between the pixel coordinates of the second positioning identifier information and the predicted coordinates obtained by projection through the camera projection function, to obtain the second projection error term.

[0014] The above implementation method is the underlying implementation of target pose determination in this disclosure. Through step-by-step solution and nonlinear optimization, initial errors and random noise can be offset, ensuring accurate and reliable calculation of extrinsic parameters between the positioning marker and the camera. By quantifying the projection deviation through the reprojection error term, the core optimization objective is focused, and manifold constraints are applied to avoid invalid solutions, thus balancing positioning accuracy and computational efficiency (limiting the number of iterations).

[0015] In one embodiment, the method further includes: detecting the physical state of the at least two positioning identifiers; and when any of the positioning identifiers is detected to be occluded, raised, or contaminated, reducing the weight of the occluded area, raised area, or contaminated area in the corresponding reprojection error term.

[0016] Here, by detecting the occlusion, lifting, and contamination of the markers and reducing the weight of the affected areas, we can avoid the overall positioning accuracy from decreasing due to poor local conditions, ensure that good positioning accuracy can still be maintained under non-ideal conditions, and reduce the recharging failure rate caused by environmental interference.

[0017] In one embodiment, determining the relative extrinsic parameters between the first positioning identifier information and the second positioning identifier information based on the first extrinsic parameter and the second extrinsic parameter includes: after acquiring multiple frames of image data that simultaneously contain the first positioning identifier information and the second positioning identifier information, and calculating the single-frame relative extrinsic parameters between the first positioning identifier information and the second positioning identifier information for each frame of image data, performing multi-frame optimization on the single-frame relative extrinsic parameters of the multiple frames to obtain stable relative extrinsic parameters between the first positioning identifier information and the second positioning identifier information.

[0018] Here, by jointly optimizing the relative extrinsic parameters of multiple frames and single frames, the influence of noise that may exist in a single frame image (such as recognition deviation caused by camera shake or sudden changes in illumination) on the relative extrinsic parameters can be reduced, resulting in a more stable and reliable relative relationship between the labels, and further improving the robustness of localization.

[0019] In one implementation, the step of performing multi-frame optimization on the single-frame relative extrinsic parameters of multiple frames to obtain stable relative extrinsic parameters between the positioning identifier information and the second positioning identifier information includes: obtaining the single-frame relative extrinsic parameters of the most recent N frames within a sliding window; constructing the relative extrinsic parameters to be optimized between the first positioning identifier information and the second positioning identifier information and the sum of squared residuals between the single-frame relative extrinsic parameters of each frame within the sliding window as a joint nonlinear least squares optimization objective function; and obtaining the stable relative extrinsic parameters by minimizing the joint nonlinear least squares optimization objective function; where N is a positive integer greater than 1.

[0020] By employing a sliding window optimization strategy, recent valid data can be aggregated to ensure that extrinsic parameters can track changes in marker position in real time. This allows the relative extrinsic parameter updates to match the robot's movement rhythm, avoiding positioning errors caused by extrinsic parameter lag. Furthermore, by combining nonlinear least squares optimization, the cumulative error of multi-frame data is minimized. Compared to single-frame optimization or simple averaging, this approach can more efficiently remove outliers and converge to the optimal solution, controlling computational complexity while maintaining accuracy.

[0021] In one embodiment, determining the target pose information of the second positioning identifier information relative to the robot based on the third extrinsic parameter and the relative extrinsic parameter includes: determining a fourth extrinsic parameter of the second positioning identifier information relative to the camera coordinate system based on the third extrinsic parameter and the relative extrinsic parameter; and determining the target pose information corresponding to the second positioning identifier information in the robot body coordinate system based on the camera extrinsic parameter of the camera relative to the robot body coordinate system and the fourth extrinsic parameter.

[0022] In one embodiment, the first positioning identifier is located on the supporting component of the charging pile, and the second positioning identifier is located on the charging interface bearing component; the step of generating return path information based on the target pose information includes: generating return path information for the robot to reach the charging interface from the current position based on the target pose information, and controlling the robot to move to the charging interface position based on the return path information.

[0023] In the above implementation scenario, when the robot is close to the charging pile, if the second positioning mark on the charging interface support component cannot be observed, the target pose of the second positioning mark relative to the robot can be deduced based on the relative extrinsic parameters calibrated when the robot is far from the charging pile. Then, the robot can be controlled to reach the position of the second positioning mark, which is the charging interface position, so as to accurately guide the robot to perform charging docking.

[0024] Secondly, a robot autonomous recharging and positioning device is provided, the device comprising: The system includes: an acquisition module for acquiring image data collected by the robot during its movement toward a charging station; an identification module for identifying positioning identifier information in the image data; a first determination module for determining relative extrinsic parameters between different positioning identifiers based on the corresponding position information of the at least two positioning identifiers in the image data when at least two positioning identifiers are identified; the at least two positioning identifiers include at least one first positioning identifier and at least one second positioning identifier; a second determination module for determining the target pose information of at least one second positioning identifier relative to the robot based on the corresponding position information of the first positioning identifier in the image data and the relative extrinsic parameters when at least one first positioning identifier is identified but no second positioning identifier is identified; and a navigation module for generating recharge path information based on the target pose information.

[0025] Thirdly, a legged robot is provided, comprising: a robot body with multiple mechanical legs; a camera mounted on the robot body for image acquisition during robot movement; a controller disposed within the robot body for acquiring image data acquired during the robot's movement toward a charging station; identifying positioning identifier information in the image data; when at least two positioning identifiers are identified, determining relative extrinsic parameters between different positioning identifiers based on the corresponding position information of the at least two positioning identifiers in the image data; the at least two positioning identifiers including at least one first positioning identifier and at least one second positioning identifier; when at least one first positioning identifier is identified but no second positioning identifier is identified, determining target pose information of at least one second positioning identifier relative to the robot based on the corresponding position information of the first positioning identifier in the image data and the relative extrinsic parameters; generating recharge path information based on the target pose information; and a charging interface disposed on the robot body for docking with charging contacts on the charging station to receive charging current.

[0026] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, performs the steps of the robot recharging and positioning method described in any of the preceding claims.

[0027] The beneficial effects of the aforementioned autonomous robot recharging and positioning device, legged robot, and storage medium are described in the preceding method description and will not be repeated here.

[0028] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure.

[0029] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0031] Figure 1A flowchart of a robot recharging and positioning method provided in this embodiment of the disclosure; Figure 2 This is a schematic diagram of a charging pile with dual QR codes provided in an embodiment of the present disclosure; Figure 3 This is a schematic diagram of a robot recharging and positioning device provided in an embodiment of the present disclosure; Figure 4 This is a schematic diagram of the structure of a legged robot 400 provided in an embodiment of the present disclosure; Figure 5 This is a schematic diagram of the structure of a controller 500 provided in an embodiment of the present disclosure. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0033] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0034] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0035] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein.

[0036] Research has found that for some translational mobile devices, stable recharging positioning can be achieved by recognizing a single identifier on the charging station (usually located on the charging station pillar). Before the robot performs recharging positioning, the extrinsic parameters of the single identifier on the charging station pillar relative to the charging base are typically calibrated. This method relies on manual calibration; for example, a worker needs to manually move the robot to the charging base for charging. During charging, the calibration process is initiated to obtain the extrinsic parameters of the QR code on the charging station pillar relative to the charging base, making it not a plug-and-play solution. Furthermore, over long-term use, slight shifts in the physical positional relationship between the charging base and the QR code on the charging station pillar can lead to inaccurate extrinsic parameters, resulting in inaccurate charging base poses derived from these parameters. Moreover, because legged robots are prone to shaking, this single-identifier positioning method is susceptible to instability.

[0037] Based on this, the present disclosure provides an autonomous recharging and positioning method that can dynamically adapt to changes in extrinsic parameters. Multiple positioning markers are set on the charging pile. As the robot moves towards the charging pile, it collects image data. Using image data captured at a distance, including information from at least two positioning markers, the relative extrinsic parameters and proximity distance between different positioning markers are determined. When only a portion of the multiple positioning markers can be identified (e.g., positioning markers near the charging port cannot be identified), the pose information of the unidentified second positioning marker relative to the robot is derived based on the identified first positioning marker information and the relative extrinsic parameters. Then, the robot navigates the recharging path based on this pose information. This embodiment of the present disclosure calibrates the relative extrinsic parameters between different positioning markers during robot movement, which can dynamically adapt to changes in environment and state, improving the accuracy and robustness of extrinsic parameter calibration. Subsequently, if the second positioning marker (such as the charging base marker) becomes invisible due to robot body shaking or close proximity, the target pose of the second positioning marker can be deduced based on the identified first positioning marker, achieving continuous positioning in shaking (effectively adaptable to recharging positioning of multi-joint, multi-degree-of-freedom legged robots) or close-range scenarios. Thus, this embodiment of the present disclosure can effectively improve the success rate of robot autonomous recharging.

[0038] The recharge positioning method according to the embodiments of this disclosure will be described in further detail below.

[0039] like Figure 1 As shown, this disclosure provides a robot recharging and positioning method, including: S101: Acquire image data collected by the robot during its movement toward the charging station; identify the positioning identification information in the image data.

[0040] Here, as the robot moves toward the charging station, a camera mounted on the robot takes a picture of the location marker on the charging station. The location marker can be a barcode, a QR code, or any other marker that can be used for location.

[0041] Multiple positioning markers can be pre-set on the charging pile. These positioning markers can be located in different positions on the charging pile. For example, some positioning markers can be set in easily observable positions such as the supporting components of the charging pile, while others can be set near the charging interface.

[0042] For example, the multiple positioning markers include at least one first positioning marker and at least one second positioning marker; the at least one second positioning marker may be located near the charging interface and is a positioning marker whose target pose needs to be determined when generating the recharge path; the at least one first positioning marker may be located at any other location on the charging pile, and the different first positioning markers may be in different locations. Optionally, the first positioning marker may be located in an easily observable position on the charging pile. For example, the first positioning marker may be located on the supporting structure of the charging pile (such as the charging pile column), and the second positioning marker may be located on the charging interface bearing structure (such as the charging base).

[0043] For example, consider a charging pile that includes a charging pile column and a charging base, with a QR code used as the location identifier. Figure 2 As shown in the exemplary embodiment of this disclosure, the charging pile 200 includes a charging pile column 21 and a charging base 22. A first QR code (Tag A) is provided on the charging pile column 21, and a second QR code (Tag B) is provided on the charging base 22. While traveling to the charging pile, the robot continuously captures images of the charging pile using its onboard camera. The first QR code (Tag A) fixedly provided on the charging pile column 21 can be used for continuous observation at both near and far distances. The second QR code (Tag B) fixedly provided on the charging base 22 is directly associated with the charging interface and may not be visible when the robot is close.

[0044] S102: When at least two positioning identifiers are identified, the relative extrinsic parameters between the different positioning identifiers are determined based on the position information corresponding to the at least two positioning identifiers in the image data; the at least two positioning identifiers include at least one first positioning identifier and at least one second positioning identifier.

[0045] Here, when the robot is far from the charging station, multiple positioning markers at different positions on the charging station can be observed within the camera's field of view. At this time, the camera can capture an image that simultaneously contains at least one first positioning marker and at least one second positioning marker. Based on the feature data of the positioning markers in the image, the relative extrinsic parameters between different positioning markers can be calculated, that is, the relationship between the position and attitude of different positioning markers in space.

[0046] In one embodiment, determining the relative extrinsic parameters between different positioning identifiers based on the position information corresponding to the at least two positioning identifiers in the image data includes: determining a first extrinsic parameter of at least one first positioning identifier relative to the camera coordinate system and a second extrinsic parameter of at least one second positioning identifier relative to the camera coordinate system based on the pixel coordinates corresponding to the at least two positioning identifiers in the image data, the camera's intrinsic parameters, and the three-dimensional coordinates of the at least two positioning identifiers in their own coordinate systems; and determining the relative extrinsic parameters between the first positioning identifier and the second positioning identifier based on the first extrinsic parameter and the second extrinsic parameter.

[0047] Here, taking the location marker as a QR code, and setting a first QR code (located in an easily observable position) and a second QR code (located near the charging interface) on the charging pile as an example, we can determine the first extrinsic parameter of the first QR code relative to the camera coordinate system and the second extrinsic parameter of the second QR code relative to the camera coordinate system based on the corner pixel coordinates of the first and second QR codes in the image, the camera's intrinsic parameters, and the three-dimensional coordinates of the corner points of the first and second QR codes in the QR code coordinate system, respectively. Then, we can determine the relative extrinsic parameter between the first and second QR codes based on the first and second extrinsic parameters.

[0048] Here, the camera's intrinsic parameters are its own inherent parameters, including focal length (fx, fy), principal point coordinates (cx, cy), and distortion coefficients (optional, used to correct lens distortion). These can be used to establish a mathematical relationship between 3D spatial points and 2D pixels. By establishing a local coordinate system with each QR code itself as the origin (the QR code plane is the xy plane, and the z-axis (also called the depth axis) is perpendicular to the QR code plane), the 3D coordinates of its four corner points in this local coordinate system can be obtained.

[0049] External reference (first external reference T) cam A. Second external parameter T cam B) is the pose transformation matrix describing the transformation from the calibration marker's coordinate system (such as the QR code coordinate system) to the camera coordinate system. It includes two key pieces of information: rotation angle and translation vector. The rotation angle reflects the calibration marker's attitude angle relative to the camera, and the translation vector reflects the calibration marker's spatial position relative to the camera. Using extrinsic parameters, the 3D coordinates of the calibration marker (such as a QR code corner point) in its own coordinate system can be converted to 3D coordinates in the camera coordinate system. Taking a QR code as the positioning marker as an example, based on the aforementioned corner point pixel coordinates, the camera's intrinsic parameters, and the 3D coordinates of the QR code corner points in their respective QR code coordinate systems, the extrinsic parameter information can be estimated based on pose estimation.

[0050] The relative extrinsic parameter can be the extrinsic parameter of any positioning identifier relative to the coordinate systems of other positioning identifiers, that is, reflecting the relative pose of any positioning identifier in the coordinate systems of other positioning identifiers. Taking different positioning identifiers including the first QR code and the second QR code as an example, the relative extrinsic parameter can be the extrinsic parameter of the first QR code relative to the coordinate system of the second QR code, or it can be the extrinsic parameter of the second QR code relative to the coordinate system of the first QR code. When the relative extrinsic parameter is the extrinsic parameter of the first QR code relative to the second QR code coordinate system (labeling the pose of the first QR code relative to the second QR code), when determining the relative extrinsic parameter between the first QR code and the second QR code based on the first and second extrinsic parameters, a matrix inversion operation can be performed on the second extrinsic parameter to obtain the inverse transformation matrix from the camera coordinate system to the second QR code coordinate system. Multiplying the inverse transformation matrix with the first extrinsic parameter yields the single-frame relative extrinsic parameter of the first QR code in the second QR code coordinate system. Similarly, when the relative extrinsic parameter is the extrinsic parameter of the second QR code relative to the first QR code coordinate system, when determining the relative extrinsic parameter between the first QR code and the second QR code based on the first and second extrinsic parameters, a matrix inversion operation can be performed on the first extrinsic parameter to obtain the inverse transformation matrix from the camera coordinate system to the first QR code coordinate system. Multiplying the inverse transformation matrix with the second extrinsic parameter yields the single-frame relative extrinsic parameter of the second QR code in the first QR code coordinate system.

[0051] In some embodiments, taking the different positioning identifiers including a first QR code and a second QR code as an example, the three-dimensional corner coordinates of the first QR code in its own coordinate system, the pixel corner coordinates in the image, and the camera intrinsic parameters can be used as one set of inputs; the three types of data of the second QR code can be used as another set of inputs. Running a pose estimation algorithm (such as solvePnP) on each set of inputs yields initial values ​​for the extrinsic parameters (first initial extrinsic parameter, second initial extrinsic parameter). The core of the pose estimation algorithm is to solve for the optimal pose transformation by minimizing the deviation between the predicted pixel coordinates obtained by projecting the three-dimensional point through the extrinsic and intrinsic parameters and the actual detected pixel coordinates. Then, a reprojection error term is constructed based on the initial extrinsic parameters, and iterative optimization is performed using a nonlinear optimization algorithm, such as the Levenberg-Marquardt (LM) algorithm, to finally obtain the accurate first extrinsic parameter (T). cam A) and second external parameter (T) cam B).

[0052] For example, determining at least one first extrinsic parameter of the first positioning identifier information relative to the camera coordinate system and at least one second extrinsic parameter of the second positioning identifier information relative to the camera coordinate system includes: based on the pixel coordinates of the first positioning identifier information in the image data, the intrinsic parameters of the camera, and the three-dimensional coordinates of the first positioning identifier information in its own coordinate system, solving for the first initial extrinsic parameter of the first positioning identifier information relative to the camera coordinate system using a pose estimation algorithm, and performing nonlinear optimization processing on the first initial extrinsic parameter to obtain the first extrinsic parameter of the first positioning identifier information relative to the camera coordinate system; based on the pixel coordinates of the second positioning identifier information in the image data, the intrinsic parameters of the camera, and the three-dimensional coordinates of the second positioning identifier information in its own coordinate system, solving for the second initial extrinsic parameter of the second positioning identifier information relative to the camera coordinate system using a pose estimation algorithm, and performing nonlinear optimization processing on the second initial extrinsic parameter to obtain the second extrinsic parameter of the second positioning identifier information relative to the camera coordinate system.

[0053] Taking the positioning identifier, which includes a first QR code and a second QR code, as an example, based on the corner pixel coordinates of the first QR code in the image, the camera's intrinsic parameters, and the three-dimensional coordinates of the corner of the first QR code in the QR code coordinate system, a pose estimation algorithm (such as the PnP algorithm) is used to solve for the first initial extrinsic parameters of the first QR code relative to the camera coordinate system. Nonlinear optimization processing is then performed on the first initial extrinsic parameters to obtain the first extrinsic parameters of the first QR code relative to the camera coordinate system. Similarly, based on the corner pixel coordinates of the second QR code in the image, the camera's intrinsic parameters, and the three-dimensional coordinates of the corner of the second QR code in the QR code coordinate system, a pose estimation algorithm is used to solve for the second initial extrinsic parameters of the second QR code relative to the camera coordinate system. Nonlinear optimization processing is then performed on the second initial extrinsic parameters to obtain the second extrinsic parameters of the second QR code relative to the camera coordinate system.

[0054] Here, for the first and second positioning markers, their pixel coordinates in the image (which can be the pixel coordinates of representative points on the calibration marker, such as corner pixel coordinates for a QR code), known camera intrinsics, and their 3D coordinates in their own coordinate system (or corner 3D coordinates for a QR code) are used as input data. The PnP algorithm can quickly obtain the initial extrinsic parameters (first and second initial extrinsic parameters) of each positioning marker relative to the camera coordinate system. This provides a reasonable starting point for subsequent optimization and avoids getting trapped in local optima. For example, based on the 3D coordinates of the QR code corner, the corresponding 2D pixel coordinates, and the camera intrinsics, the inverse problem of projecting the 3D spatial points of the QR code corner onto the camera intrinsics to obtain the 2D pixel points can be deduced. This pose transformation relationship is the initial extrinsic parameter containing the rotation matrix (describing the QR code's pose angle relative to the camera) and the translation vector (describing the QR code's spatial position relative to the camera).

[0055] Then, nonlinear optimization was performed on the two initial extrinsic parameters to correct errors in the initial solution (such as image noise, corner detection deviation, etc.), and finally, high-precision first and second extrinsic parameters were obtained.

[0056] In one implementation, nonlinear optimization of the first initial extrinsic parameters to obtain the first extrinsic parameters of the first positioning identifier information relative to the camera coordinate system includes: constructing a first projection error term based on the first initial extrinsic parameters, and iteratively optimizing the first projection error term (e.g., using the Levenberg-Marquardt algorithm) until the error change is less than a preset error threshold or reaches a preset maximum number of iterations (e.g., the preset error threshold is 1e-6, and the preset maximum number of iterations is 50), to obtain the first extrinsic parameters of the first positioning identifier information relative to the camera coordinate system; correspondingly, nonlinear optimization of the second initial extrinsic parameters to obtain the second extrinsic parameters of the second positioning identifier information relative to the camera coordinate system includes: constructing a second projection error term based on the second initial extrinsic parameters, and iteratively optimizing the second projection error term until the error change is less than the preset error threshold or reaches the preset maximum number of iterations, to obtain the second extrinsic parameters of the second positioning identifier information relative to the camera coordinate system; wherein, when iteratively optimizing the first and second projection error terms, manifold constraints are applied.

[0057] In one embodiment, constructing the first projection error term may include: based on the pixel coordinates of the first positioning identifier information in the image data (or corner pixel coordinates if it is a QR code), the intrinsic parameters of the camera, the three-dimensional coordinates of the first positioning identifier information in its own coordinate system (or corner three-dimensional coordinates if it is a QR code), and the camera projection function, using the extrinsic parameters of the first positioning identifier information relative to the camera coordinate system as optimization variables, constructing the sum of squared residuals between the pixel coordinates of the first positioning identifier information (or corner pixel coordinates if it is a QR code) and the predicted coordinates obtained by projection through the camera projection function, to obtain the first projection error term; Accordingly, the construction of the second projection error term may include: based on the pixel coordinates of the acquired second positioning identifier information in the image data, the intrinsic parameters of the camera, the three-dimensional coordinates of the second positioning identifier information in its own coordinate system, and the camera projection function, using the extrinsic parameters of the second positioning identifier information relative to the camera coordinate system as optimization variables, constructing the sum of squared residuals between the pixel coordinates of the second positioning identifier information and the predicted coordinates obtained by projection through the camera projection function, and obtaining the second projection error term.

[0058] In the above implementation, the extrinsic parameter is used as the optimization target. The difference between the actual pixel coordinates of the quantized calibration mark (such as the corner point of a QR code) and the predicted pixel coordinates derived from the extrinsic parameter is used to form an error term that can be iteratively optimized, providing a clear optimization target for subsequent nonlinear optimization.

[0059] Specifically, by using the camera projection function π (which describes the projection logic from a 3D point to a 2D image), combined with the camera intrinsic parameter K (which describes the camera imaging rules) and the current extrinsic parameter T (which determines the projection result from the 3D coordinates Pi to the 2D pixel coordinates), the 3D coordinates of representative positions on the calibration marker (such as the 3D coordinates Pi of the QR code corners (since the QR code size is known, the 3D coordinates of the QR code corners in their own coordinate system can be directly obtained, such as (0,0,0), (a,0,0), etc.)) are projected into the theoretically predicted pixel coordinates (π(K, T, Pi)). Then, the difference (residual) between the actual observed pixel coordinates ui (such as the QR code corner pixel coordinates detected from the image) and the predicted pixel coordinates is calculated. The sum of squares of the residuals corresponding to the left and right representative positions on the positioning marker (all corners of the QR code) is then calculated to finally form the reprojection error term E(T), as shown in the following formula:

[0060] The magnitude of the error term E(T) directly reflects the accuracy of the current extrinsic parameter T; the smaller the error, the closer the projection position derived from the extrinsic parameter matches the actual observation, and the higher the accuracy of the extrinsic parameter.

[0061] After obtaining the initial extrinsic parameters through pose estimation algorithms, the reprojection error term can be iteratively optimized using nonlinear least squares algorithms (such as the Levenberg-Marquardt algorithm). Combined with explicit termination conditions and physical constraints, systematic errors in the initial extrinsic parameters (such as image noise and corner detection bias) can be eliminated, ultimately yielding high-precision extrinsic parameters. Here, the first / second initial extrinsic parameters obtained by the PnP algorithm are used as the starting point for iteration, avoiding convergence difficulties or local optima problems caused by starting optimization with random values, thus balancing optimization efficiency and effectiveness.

[0062] In addition, the rotation matrix in the extrinsic parameters must satisfy the physical properties of being orthogonal and having a determinant of 1. In this embodiment, by applying SO (3) manifold constraints, the optimization process can be restricted to the legal space of the rotation matrix, thereby avoiding the occurrence of mathematically convergent but physically infeasible rotation matrices (such as matrices with a determinant of 1) during the optimization process. This ensures that the final extrinsic parameters (first extrinsic parameter / second extrinsic parameter) conform to the actual physical scenario, avoids subsequent pose transformation errors due to the failure of the rotation matrix, and further ensures the reliability of the positioning accuracy.

[0063] To address the issue that location markers are susceptible to physical abnormalities such as occlusion, warping, or contamination in real-world usage scenarios, leading to deviations in the detection of representative location points (such as QR code corner points) and thus interfering with the accuracy of reprojection error calculation, this disclosure provides a strategy for dynamically adjusting the weights of error terms in optional embodiments. For example, in one embodiment, the physical state of at least two location marker information is detected. When any of the location marker information is detected to be occluded, warped, or contaminated, the weight of the occluded, warped, or contaminated area in the corresponding reprojection error term is reduced.

[0064] Specifically, based on the area ratio of the occluded area, raised area, or polluted area, the weighting coefficient of the corresponding representative location point (such as the corner point in the occluded area) can be calculated, and the weighting coefficient can be multiplied by the reprojection error of the corresponding representative location point to obtain the weighted reprojection error term.

[0065] Here, the calculation of the reprojection error term relies on the residual between the pixel coordinates and the predicted coordinates of representative location points (such as QR code corner points). If the representative location points are inaccurately detected due to occlusion, warping, or contamination (e.g., pixel coordinate offset, inability to identify), their corresponding residuals will contain spurious errors. If these are included in the optimization with normal weights, they will mislead the extrinsic parameter estimation results. Therefore, reducing the weights of these abnormal representative location points (e.g., adjusting the normal weight 1 to 0.2~0.5) can reduce the impact of abnormal data on the optimization objective, making the reprojection error term more accurately reflect the actual deviation of the extrinsic parameters.

[0066] In one implementation, to address the potential reliability risks posed by factors such as image noise and corner detection bias in extrinsic parameter estimation, the uncertainty of pose estimation can be quantified using a covariance matrix, and a feedback correction mechanism can be established to further ensure the accuracy and reliability of extrinsic parameter estimation. Specifically, after obtaining the first extrinsic parameter from the first positioning marker to the camera coordinate system and the second extrinsic parameter from the second positioning marker to the camera coordinate system, the covariance matrix of the first and second extrinsic parameters is calculated. The uncertainty of pose estimation is evaluated based on the covariance matrix. When the uncertainty exceeds a preset uncertainty threshold, the number of sampling frames is increased, and pose estimation is performed again.

[0067] Here, the covariance matrix is ​​used to quantify the uncertainty of pose estimation. It includes the variances of the rotation matrix and translation vector parameters in the first and second extrinsic parameters, as well as the correlations between these parameters. The magnitude of the values ​​in the matrix directly reflects the degree of fluctuation in the estimation results of the corresponding parameters. The larger the variance, the more significant the interference from noise, detection bias, etc., and the lower the reliability of the extrinsic parameter estimation; conversely, the smaller the variance, the more stable the extrinsic parameters. When the uncertainty exceeds the limit, pose estimation is performed again by increasing the number of sampling frames. Essentially, this utilizes observation data from more image frames to offset random errors. Multiple frames of data can filter out instantaneous noise, corner false detections, and other accidental interference from single-frame images, improving the statistical stability of the extrinsic parameter estimation.

[0068] In one embodiment, to address the problem that the relative extrinsic parameters calculated from a single frame image are easily affected by factors such as instantaneous noise, illumination fluctuations, and deviations in the detection of representative location points, resulting in insufficient accuracy and poor stability, this disclosure provides a multi-frame optimization strategy: after acquiring multiple frames of image data that simultaneously contain the first positioning identifier information and the second positioning identifier information, and calculating the single-frame relative extrinsic parameters between the first positioning identifier information and the second positioning identifier information for each frame of image data, the single-frame relative extrinsic parameters of the multiple frames are optimized to obtain stable relative extrinsic parameters between the first positioning identifier information and the second positioning identifier information.

[0069] Specifically, the process of optimizing the single-frame relative extrinsic parameters of multiple frames may include: obtaining the single-frame relative extrinsic parameters of the most recent N frames within a sliding window; constructing the relative extrinsic parameters to be optimized between the first positioning identifier information and the second positioning identifier information and the sum of squared residuals between the single-frame relative extrinsic parameters of each frame within the sliding window as a joint nonlinear least squares optimization objective function; and obtaining the stable relative extrinsic parameters by minimizing the joint nonlinear least squares optimization objective function; where N is a positive integer greater than 1.

[0070] Here, we assume the relative extrinsic parameter to be optimized is T. BA_stable (i.e., the stable relative extrinsic parameter to be obtained, such as describing the pose of the first QR code TagA relative to the second QR code TagB), the single-frame relative extrinsic parameter of the k-th frame within the sliding window is T. B A(k) (k=1,2,...,N, each frame is passed through) (Calculated).

[0071] With T B A_stable and each frame T B The sum of squared residuals of A(k) is the optimization objective, i.e. (The residual represents the deviation between the relative extrinsic parameters of a single frame and the relative extrinsic parameters to be optimized; the sum of squares can amplify the impact of outliers and enhance convergence.) This ensures that the final stable relative extrinsic parameters are as close as possible to the statistical average level of the relative extrinsic parameters of each single frame, while filtering out outlier frames with excessive deviations. A nonlinear least squares solution method is used to iteratively optimize the objective function. During the optimization process, random errors (such as corner detection deviations and deviations caused by instantaneous illumination interference) and outliers (such as incorrect extrinsic parameters caused by sudden occlusion in a single frame) in the relative extrinsic parameters are automatically suppressed. When the optimization iteration meets the convergence condition (such as the sum of squares of the residuals being less than a preset threshold or reaching the maximum number of iterations), the output T is... B A_stable refers to the stable relative extrinsic parameter. This result is not a single-frame relative extrinsic parameter of a particular frame, but rather the optimal fusion result of multiple frames of valid data, combining both accuracy and stability.

[0072] for The detailed implementation can be achieved through the following formula:

[0073] here, Let represent the pose deviation matrix between the relative extrinsic parameters of the k-th frame and the extrinsic parameters to be optimized (the pose difference between the two is obtained directly by eliminating the intermediate coordinate system through matrix inverse multiplication). log(•) is the logarithmic mapping from the Lie group to the Lie algebra; This is the 6-dimensional vector residual after transforming the pose deviation matrix (3 dimensions correspond to rotation deviation, and 3 dimensions correspond to translation deviation). The above formula converts the deviation of the pose matrix into a computable vector residual. Through logarithmic mapping, the pose deviation matrix in Lie group space is transformed into a 6-dimensional vector in Lie algebra space. The joint nonlinear least squares optimization objective function constructed in this way is the sum of squares of these vector residuals (i.e., ).

[0074] The above explanation uses the relative extrinsic parameter to represent the pose of the first positioning identifier information (such as the first QR code TagA) relative to the second positioning identifier information (such as the second QR code TagB). Similarly, the relative extrinsic parameter can also be the pose of the second positioning identifier information (such as the second QR code TagB) relative to the first positioning identifier information (such as the first QR code TagA). In specific implementation, it is only necessary to interchange the relevant information of the first positioning identifier information and the second positioning identifier information.

[0075] In one implementation, to further improve the reliability of positioning results and address the issue of accuracy decay of stable relative extrinsic parameters caused by factors such as minor displacement of calibration markers (e.g., due to ground vibration or loosening of adhesive), mechanical vibration, and environmental changes, a dynamic update mechanism involving periodic acquisition of new data and recalibration can be used to ensure that the relative extrinsic parameters always conform to the actual pose relationship. Specifically, the stable relative extrinsic parameters are periodically updated and calibrated using newly acquired image frames that simultaneously contain the first and second positioning markers, and the updated stable relative extrinsic parameters replace the original stable relative extrinsic parameters stored in the robot system.

[0076] Here, new image frames containing both the first and second positioning identifiers can be acquired according to a preset cycle (e.g., the first startup each day, or after N uses of the recharge function). This ensures that the data reflects the true positional relationship between the first and second positioning identifiers. Following the aforementioned process of single-frame relative extrinsic parameter calculation and sliding window multi-frame optimization, new stable relative extrinsic parameters are recalculated based on the newly acquired image frames. These newly obtained stable relative extrinsic parameters directly replace the original extrinsic parameters stored in the robot system, completing the calibration update without manual intervention.

[0077] In some embodiments, when using a depth camera to photograph the QR code on the charging pile, depth information corresponding to the photographed image can be obtained simultaneously. The depth information can be used to help verify the accuracy of the extrinsic parameter estimation.

[0078] Here, unlike ordinary 2D cameras that can only acquire the pixel coordinates of the positioning marker, depth cameras can simultaneously acquire 2D pixel images and depth information, such as each representative location point of the positioning marker (e.g., each corner of a QR code) and the actual physical distance from the area to the camera lens (z-axis coordinate in 3D space). Based on this, after obtaining the extrinsic parameter T from the positioning marker to the camera using the above method, the 3D coordinates (X_cam, Y_cam, Z_cam) of the representative location point in the camera coordinate system can be derived from the extrinsic parameter T and the 3D coordinates Pi of the representative location point in the positioning marker's own coordinate system. Here, Z_cam is the theoretical depth value calculated using the extrinsic parameter. The measured depth value is compared with the theoretical depth value. If the deviation is within a threshold, it indicates that the extrinsic parameter estimation is accurate and can be directly used for subsequent relative extrinsic parameter calculations. If the deviation exceeds the threshold, it indicates that the extrinsic parameter estimation may be affected by image noise, corner detection errors, insufficient algorithm optimization, etc., and a correction mechanism needs to be triggered (such as re-performing nonlinear optimization, increasing the number of sampling frames, and checking the physical state of the positioning marker) until the depth value derived from the extrinsic parameter matches the measured value. This method uses a depth camera to simultaneously acquire images of representative locations and corresponding depth data, and uses three-dimensional spatial distance information to assist in verifying the extrinsic parameter estimation results, which can further improve the reliability and accuracy of extrinsic parameter verification.

[0079] S103: When at least one of the first positioning identifiers is identified and no second positioning identifier is identified, the target pose information of at least one second positioning identifier relative to the robot is determined based on the position information corresponding to the first positioning identifier in the image data and the relative extrinsic parameters.

[0080] Taking the example that the first positioning mark is located on an easily observable support component (such as the charging pile column) on the charging pile, and the second positioning mark is located on a charging interface support component (such as the charging base) that is close to the charging interface, when the robot approaches the charging interface position on the charging pile (such as the charging base), the second positioning mark on the charging base is easily obscured or exceeds the camera's field of view. At this time, it may only be possible to observe the first positioning mark on the charging pile column through the image, and the second positioning mark needs to be located indirectly.

[0081] In some embodiments, determining the target pose information of at least one second positioning identifier relative to the robot based on the position information corresponding to the first positioning identifier in the image data and the relative extrinsic parameter may include: determining a third extrinsic parameter of the first positioning identifier relative to the camera coordinate system based on the pixel coordinates of the first positioning identifier in the image data, the intrinsic parameters of the camera, and the three-dimensional coordinates of the first positioning identifier in its own coordinate system; and determining the target pose information of the second positioning identifier relative to the robot based on the third extrinsic parameter and the relative extrinsic parameter.

[0082] Here, the method for determining the third extrinsic parameter can refer to the methods for determining the first and second extrinsic parameters mentioned above. For example, based on the pixel coordinates of the representative position point of the first positioning marker in the image, the intrinsic parameters of the camera, and the three-dimensional coordinates of the representative position point of the first positioning marker in the positioning marker's own coordinate system, the third initial extrinsic parameter from the first positioning marker to the camera coordinate system can be solved by a pose estimation algorithm. By performing nonlinear optimization on the third initial extrinsic parameter, the third extrinsic parameter from the first positioning marker to the camera coordinate system can be obtained.

[0083] For example, when determining the target pose of the second positioning marker in the camera coordinate system based on the third extrinsic parameter and the relative extrinsic parameter, if the relative extrinsic parameter represents the pose of the first positioning marker relative to the second positioning marker, that is, the pose of the first positioning marker in the coordinate system of the second positioning marker, then matrix inversion can be performed on the relative extrinsic parameter to obtain the inverse relative extrinsic parameter of the second positioning marker in the coordinate system of the first positioning marker. Then, matrix multiplication of the third extrinsic parameter and the inverse relative extrinsic parameter can be performed to obtain the target pose of the second positioning marker in the camera coordinate system. Alternatively, if the relative extrinsic parameter represents the pose of the second positioning marker relative to the first positioning marker, that is, the pose of the second positioning marker in the coordinate system of the first positioning marker, then matrix multiplication of the third extrinsic parameter and the relative extrinsic parameter can be performed directly to obtain the target pose of the second positioning marker in the camera coordinate system.

[0084] In some embodiments, to avoid deviations in the recharge path caused by changes in the robot's own posture, environmental interference, or dynamic shifts in extrinsic parameters during robot movement, this embodiment of the present disclosure can update the relative extrinsic parameters and / or the third extrinsic parameters in real time or periodically during the process of controlling the robot to move along the recharge path to the location of the charging interface. Based on the updated third extrinsic parameters and the relative extrinsic parameters, the current target pose of the second positioning marker in the camera coordinate system is recalculated. The recharge path is then dynamically adjusted based on the current target pose until the robot reaches the location of the charging interface support component (e.g., the charging base) and completes the charging interface docking. In this way, it can be ensured that the recharge path accurately matches the actual position of the charging interface support component throughout the entire process from the robot's approach to the completion of docking, effectively avoiding docking deviations caused by fixed parameters and significantly improving the success rate of autonomous recharge.

[0085] For example, according to a preset period, or after each acquisition of image data collected by the robot, the positioning identifier information in the image data is identified; when at least two positioning identifier information are identified each time, the relative extrinsic parameters between the different positioning identifier information are updated based on the position information corresponding to the at least two positioning identifier information in the image data; when the second positioning identifier information cannot be identified, after at least one first positioning identifier information is identified each time, the target pose information of at least one second positioning identifier information relative to the robot is updated based on the position information corresponding to the first positioning identifier information in the image data and the relative extrinsic parameters.

[0086] Furthermore, dynamic obstacles can be detected in real time during robot movement. When a dynamic obstacle is detected obstructing the charging interface support component, the third extrinsic parameter can be continuously updated through observation of the first QR code during obstacle avoidance. Based on the updated third extrinsic parameter and the relative extrinsic parameter, the corrected target pose of the second positioning marker in the camera coordinate system is recalculated. The return path is then corrected in real time according to the corrected target pose, resulting in navigation commands after obstacle avoidance compensation. This achieves seamless coordination between obstacle avoidance and positioning, ensuring that the robot avoids obstacles while continuously updating parameters and correcting the path to prevent docking deviation with the charging interface support component after obstacle avoidance, thus ensuring that the return charging accuracy is not affected.

[0087] S104: Generate recharge path information based on the target pose information.

[0088] Here, based on the target pose information, the robot can generate a return path information from its current position to the charging interface (i.e., the charging interface support component, such as the charging base), and control the robot to move to the charging interface position according to the return path information.

[0089] In specific implementation, when determining the target pose information of the second positioning identifier relative to the robot based on the third extrinsic parameter and the relative extrinsic parameter, a fourth extrinsic parameter of the second positioning identifier relative to the camera coordinate system can be determined based on the third extrinsic parameter and the relative extrinsic parameter. Based on the camera extrinsic parameter of the camera relative to the robot body coordinate system and the fourth extrinsic parameter, the target pose information corresponding to the second positioning identifier in the robot body coordinate system is determined (the base pose matrix is ​​calculated by multiplying the camera extrinsic parameter matrix and the target pose matrix). For example, if the second positioning identifier is located on the charging base, the base pose of the charging base in the robot body coordinate system is determined. Based on the base pose, a return path for the robot from its current position to the charging base is planned, and the robot is controlled to move along the return path to the charging base.

[0090] Here, combining the known camera extrinsic parameters of the camera relative to the robot's body coordinate system, the pose of the second positioning marker in the camera coordinate system (represented by the fourth extrinsic parameter) is converted into the target pose of the second positioning marker in the robot's body coordinate system. Based on the converted target pose and the robot's current position, an optimal navigation path (avoiding static obstacles and adapting to the moving posture) is planned from the current point to the location of the second positioning marker. As the robot moves along the path, relevant parameters (such as the third extrinsic parameter) can be updated in real time, and the path can be dynamically adjusted to compensate for errors. If dynamic obstacles are encountered, the target pose is corrected while avoiding the obstacles to ensure that the path does not deviate. Finally, the robot is controlled to accurately reach the location of the charging interface component (such as the charging base), complete the alignment and connection of the charging interface, and finally achieve autonomous recharging.

[0091] Here, after the robot reaches the target pose indication position along the recharge path, the relative deviation between the robot's charging interface and the charging contact on the base is controlled within centimeters. Subsequently, multiple sensors such as vision, magnetic sensing, and infrared can be fused to correct the robot's translation, rotation, and pitch deviations, ensuring that the posture of the charging interface and the contact is perfectly matched.

[0092] After completing autonomous recharging, the changes in the first extrinsic parameter, the second extrinsic parameter, the third extrinsic parameter, and the relative extrinsic parameter during this recharging process can be recorded, and the error distribution of pose estimation can be statistically analyzed to optimize the parameter settings for subsequent recharging processes.

[0093] like Figure 3 As shown, another embodiment of this disclosure provides a robot recharging and positioning device 300 corresponding to the above-described robot recharging and positioning method, which can be deployed on the robot and includes: The acquisition module 31 is used to acquire image data collected by the robot during its movement towards the charging station; the identification module 32 is used to identify the positioning identifier information in the image data; the first determination module 33 is used to determine the relative extrinsic parameters between different positioning identifier information based on the position information corresponding to the at least two positioning identifier information in the image data when at least two positioning identifier information are identified; the at least two positioning identifier information includes at least one first positioning identifier information and at least one second positioning identifier information; the second determination module 34 is used to determine the target pose information of at least one second positioning identifier information relative to the robot based on the position information corresponding to the first positioning identifier information in the image data and the relative extrinsic parameters when at least one first positioning identifier information is identified and no second positioning identifier information is identified; the navigation module 35 is used to generate recharge path information according to the target pose information.

[0094] The specific implementation of each of the above modules can be found in the description of the methods mentioned above, and will not be repeated here.

[0095] Using the methods and devices of this disclosure, image data is acquired during the robot's movement toward a charging station. The relative extrinsic parameters between different markers are calibrated. If the second positioning marker becomes invisible due to robot movement or close proximity, the target pose of the second positioning marker can be deduced based on the identified first positioning marker, achieving continuous positioning in moving or close-range scenarios. Since the relative extrinsic parameters are newly calibrated during the robot's movement toward the charging station, rather than being pre-calibrated and fixed, the accuracy of the relative extrinsic parameters is significantly improved. Some optional implementations of this disclosure also enhance robustness through multiple design features to cover various complex scenarios. For example, when a positioning marker is partially obscured, raised, or contaminated, positioning accuracy can be maintained using another positioning marker. Simultaneously, the error term weights of the corner points in the obscured or contaminated areas can be dynamically adjusted to avoid the influence of abnormal data. Furthermore, depth information from a depth camera can be used to assist in verifying the accuracy of extrinsic parameter estimation. The covariance matrix can assess pose estimation uncertainty, and when uncertainty exceeds the limit, the number of sampling frames is automatically increased for correction, resisting environmental noise interference. Additionally, when a dynamic obstacle is detected obstructing the charging dock, extrinsic parameters can be continuously updated and the navigation path corrected during obstacle avoidance to prevent docking deviations caused by obstacle avoidance. If the charging dock is accidentally touched, the pose can be corrected through a single dual-code rapid observation, eliminating the need for re-navigation.

[0096] In summary, the embodiments disclosed herein provide a flexible and reliable solution for autonomous recharging of robots in different scenarios, significantly improving the success rate, stability and practicality of autonomous recharging.

[0097] like Figure 4As shown in the illustration, this disclosure also provides a legged robot 400, comprising: a robot body 41 with multiple mechanical legs; a camera 42 mounted on the robot body 41 for image acquisition during robot movement; a controller 43 disposed within the robot body 41 for acquiring image data collected during robot movement toward a charging station; identifying positioning identifier information in the image data; determining relative extrinsic parameters between different positioning identifiers based on the corresponding position information of the at least two positioning identifiers in the image data when at least two positioning identifiers are identified; the at least two positioning identifiers include at least one first positioning identifier and at least one second positioning identifier; when at least one first positioning identifier is identified but no second positioning identifier is identified, determining the target pose information of at least one second positioning identifier relative to the robot based on the corresponding position information of the first positioning identifier in the image data and the relative extrinsic parameters; generating recharge path information based on the target pose information; and a charging interface 44 disposed on the robot body 41 for docking with charging contacts on the charging station to receive charging current.

[0098] Reference Figure 5 The diagram shown is a schematic representation of a controller 500 according to an exemplary embodiment of this disclosure. The controller 500 can be deployed on a legged robot and includes: The processor 510, memory 520, and bus 530 are included. The memory 520 is used to store execution instructions and includes main memory 521 and external memory 522. The main memory 521, also known as internal memory, is used to temporarily store the operation data in the processor 510 and the data exchanged with external memory 522 such as hard disk. The processor 510 exchanges data with external memory 522 through main memory 521.

[0099] In this embodiment, the memory 520 is specifically used to store application code that executes the scheme of this disclosure, and its execution is controlled by the processor 510. That is, when the electronic device 500 is running, the processor 510 communicates with the memory 520 through the bus 530, or the processor 510 communicates with the memory 520 through other means, so that the processor 510 executes the application code stored in the memory 520, thereby executing the steps of the robot recharging and positioning method described in any of the foregoing embodiments. The memory 520 may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The processor 510 may be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0100] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the robot recharging and positioning method described in any of the above embodiments. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. Available media can be magnetic media, such as hard disks, floppy disks, and magnetic tapes; optical media, such as DVD-ROM, DVD-RAM, DVD-RW, DVD+RW, CD-ROM, CD-RW, CD-RW, and MO (magneto-optical) storage media; and semiconductor storage media, such as flash memory, EEPROM, Dynamic Random Access Memory (DRAM), and Static Random Access Memory (SRAM).

[0101] The computer program can be written in various computer programming languages, including but not limited to C, C++, Python, and custom messages and services under the ROS framework. When the computer program is executed by the processor, it implements the various steps of the robot recharging and positioning method in the embodiments of this disclosure.

[0102] This disclosure also provides a computer program product storing a computer program. When executed by a processor, this computer program performs the steps of the robot recharging and positioning method provided in any of the above embodiments of this disclosure. For details, please refer to the above method embodiments, which will not be repeated here. The computer program product can be implemented using hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied as a computer storage medium, which can be a volatile or non-volatile computer-readable storage medium. In another optional embodiment, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0103] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices and apparatuses described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0104] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, the functional units in the various embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0105] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A robot recharging and positioning method, characterized in that, include: Acquire image data as the robot moves toward the charging station; Identify the location marker information in the image data; When at least two location identifiers are identified, the relative extrinsic parameters between the different location identifiers are determined based on the location information corresponding to the at least two location identifiers in the image data. The at least two location identification information includes at least one first location identification information and at least one second location identification information; When at least one of the first positioning identifiers is identified and no second positioning identifier is identified, the target pose information of at least one second positioning identifier relative to the robot is determined based on the position information corresponding to the first positioning identifier in the image data and the relative extrinsic parameters. Recharge path information is generated based on the target pose information.

2. The method according to claim 1, characterized in that, The identification of location marker information in the image data includes: According to a preset cycle, or after each acquisition of image data collected by the robot, the positioning identification information in the image data is identified; The determination of the relative extrinsic parameters between different location identifier information includes: Each time at least two location identifiers are identified, the relative extrinsic parameters between the different location identifiers are updated based on the location information corresponding to the at least two location identifiers in the image data. Determining at least one of the second positioning identifier information relative to the target pose information of the robot includes: After failing to identify the second positioning identifier information, each time at least one of the first positioning identifier information is identified, the target pose information of at least one of the second positioning identifier information relative to the robot is updated based on the position information corresponding to the first positioning identifier information in the image data and the relative extrinsic parameters.

3. The method according to claim 1, characterized in that, The step of determining the relative extrinsic parameters between different positioning identifiers based on the location information corresponding to the at least two positioning identifiers in the image data includes: Based on the pixel coordinates corresponding to the at least two positioning identifiers in the image data, the camera's intrinsic parameters, and the three-dimensional coordinates of the at least two positioning identifiers in their own coordinate systems, a first extrinsic parameter of at least one first positioning identifier relative to the camera coordinate system and a second extrinsic parameter of at least one second positioning identifier relative to the camera coordinate system are determined. The relative extrinsic parameters between the first positioning identifier information and the second positioning identifier information are determined based on the first extrinsic parameter and the second extrinsic parameter. The step of determining the target pose information of at least one of the second positioning identifiers relative to the robot based on the position information corresponding to the first positioning identifier in the image data and the relative extrinsic parameters includes: Based on the pixel coordinates of the first positioning identifier in the image data, the intrinsic parameters of the camera, and the three-dimensional coordinates of the first positioning identifier in its own coordinate system, the third extrinsic parameter of the first positioning identifier relative to the camera coordinate system is determined. Based on the third extrinsic parameter and the relative extrinsic parameter, the target pose information of the second positioning identifier relative to the robot is determined.

4. The method according to claim 3, characterized in that, Determining at least one first extrinsic parameter of the first positioning identifier information relative to the camera coordinate system and at least one second extrinsic parameter of the second positioning identifier information relative to the camera coordinate system includes: Based on the pixel coordinates of the first positioning identifier in the image data, the camera's intrinsic parameters, and the three-dimensional coordinates of the first positioning identifier in its own coordinate system, the first initial extrinsic parameters of the first positioning identifier relative to the camera coordinate system are solved by a pose estimation algorithm. The first initial extrinsic parameters are then subjected to nonlinear optimization processing to obtain the first extrinsic parameters of the first positioning identifier relative to the camera coordinate system. Based on the pixel coordinates of the second positioning identifier in the image data, the camera's intrinsic parameters, and the three-dimensional coordinates of the second positioning identifier in its own coordinate system, the second initial extrinsic parameters of the second positioning identifier relative to the camera coordinate system are solved by a pose estimation algorithm. The second initial extrinsic parameters are then subjected to nonlinear optimization processing to obtain the second extrinsic parameters of the second positioning identifier relative to the camera coordinate system.

5. The method according to claim 4, characterized in that, The nonlinear optimization process performed on the first initial extrinsic parameters to obtain the first extrinsic parameters of the first positioning identifier information relative to the camera coordinate system includes: Based on the first initial extrinsic parameters, a first projection error term is constructed, and the first projection error term is iteratively optimized until the error change is less than a preset error threshold or reaches a preset maximum number of iterations, thereby obtaining the first extrinsic parameters of the first positioning identifier information relative to the camera coordinate system. The nonlinear optimization of the second initial extrinsic parameters to obtain the second extrinsic parameters of the second positioning identifier information relative to the camera coordinate system includes: Based on the second initial extrinsic parameters, a second projection error term is constructed, and the second projection error term is iteratively optimized until the error change is less than the preset error threshold or the preset maximum number of iterations is reached, thereby obtaining the second extrinsic parameters of the second positioning identifier information relative to the camera coordinate system. When iteratively optimizing the first and second projection error terms, manifold constraints are applied.

6. The method according to claim 5, characterized in that, The construction of the first projection error term includes: Based on the pixel coordinates of the first positioning identifier information in the image data, the camera's intrinsic parameters, the three-dimensional coordinates of the first positioning identifier information in its own coordinate system, and the camera projection function, the extrinsic parameters of the first positioning identifier information relative to the camera coordinate system are used as optimization variables to construct the sum of squared residuals between the pixel coordinates of the first positioning identifier information and the predicted coordinates obtained by projection through the camera projection function, thus obtaining the first reprojection error term. The construction of the second projection error term includes: Based on the pixel coordinates of the second positioning identifier information in the image data, the camera's intrinsic parameters, the three-dimensional coordinates of the second positioning identifier information in its own coordinate system, and the camera projection function, the extrinsic parameters of the second positioning identifier information relative to the camera coordinate system are used as optimization variables. The sum of squared residuals between the pixel coordinates of the second positioning identifier information and the predicted coordinates obtained by projection through the camera projection function is constructed to obtain the second reprojection error term.

7. The method according to claim 6, characterized in that, The method further includes: The physical state of the at least two positioning identifiers is detected. When any of the positioning identifiers is detected to be occluded, raised, or contaminated, the weight of the occluded area, raised area, or contaminated area in the corresponding reprojection error term is reduced.

8. The method according to claim 3, characterized in that, Determining the relative extrinsic parameters between the first positioning identifier information and the second positioning identifier information based on the first extrinsic parameter and the second extrinsic parameter includes: After acquiring multiple frames of image data that simultaneously contain the first positioning identifier information and the second positioning identifier information, and calculating the single-frame relative extrinsic parameters between the first positioning identifier information and the second positioning identifier information for each frame of image data, the single-frame relative extrinsic parameters of the multiple frames are optimized to obtain stable relative extrinsic parameters between the first positioning identifier information and the second positioning identifier information.

9. The method according to claim 8, characterized in that, The step of performing multi-frame optimization on the single-frame relative extrinsic parameters of multiple frames to obtain stable relative extrinsic parameters between the positioning identifier information and the second positioning identifier information includes: Obtain the single-frame relative extrinsic parameters of the most recent N frames within the sliding window, construct the relative extrinsic parameters to be optimized between the first positioning identifier information and the second positioning identifier information, and the sum of squared residuals between the single-frame relative extrinsic parameters of each frame within the sliding window as a joint nonlinear least squares optimization objective function. By minimizing the joint nonlinear least squares optimization objective function, the stable relative extrinsic parameters are obtained; N is a positive integer greater than 1.

10. The method according to claim 3, characterized in that, Determining the target pose information of the second positioning identifier relative to the robot based on the third extrinsic parameter and the relative extrinsic parameter includes: Based on the third extrinsic parameter and the relative extrinsic parameter, determine the fourth extrinsic parameter of the second positioning identifier information relative to the camera coordinate system; Based on the camera extrinsic parameters relative to the robot's body coordinate system and the fourth extrinsic parameter, the target pose information corresponding to the second positioning identifier information in the robot's body coordinate system is determined.

11. The method according to claim 1, characterized in that, The first positioning mark is located on the supporting component of the charging pile, and the second positioning mark is located on the charging interface bearing component; The step of generating recharge path information based on the target pose information includes: Based on the target pose information, generate the return path information for the robot to reach the charging interface from its current position, and control the robot to move to the charging interface position according to the return path information.

12. A positioning device for autonomous robot recharging, characterized in that, The device includes: The acquisition module is used to acquire image data collected by the robot as it moves toward the charging station; The identification module is used to identify the location identification information in the image data; The first determining module is used to determine the relative extrinsic parameters between different positioning identifiers based on the position information corresponding to the at least two positioning identifiers in the image data when at least two positioning identifiers are identified; the at least two positioning identifiers include at least one first positioning identifier and at least one second positioning identifier. The second determining module is used to determine the target pose information of at least one second positioning identifier relative to the robot based on the position information corresponding to the first positioning identifier in the image data and the relative extrinsic parameters when at least one first positioning identifier is identified and no second positioning identifier is identified. The navigation module is used to generate recharge path information based on the target pose information.

13. A legged robot, characterized in that, include: The robot body has multiple mechanical legs. A camera, mounted on the robot body, is used to acquire images during the movement of the legged robot. A controller, housed within the robot body, is used to acquire image data collected during the robot's movement toward a charging station; identify positioning marker information in the image data; when at least two positioning markers are identified, determine relative extrinsic parameters between the different positioning markers based on the corresponding position information of the at least two positioning markers in the image data; the at least two positioning markers include at least one first positioning marker and at least one second positioning marker; when at least one first positioning marker is identified but no second positioning marker is identified, determine the target pose information of at least one second positioning marker relative to the robot based on the corresponding position information of the first positioning marker in the image data and the relative extrinsic parameters; and generate recharge path information based on the target pose information. A charging interface is provided on the robot body for connecting to the charging contacts on the charging station to receive charging current.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the robot recharging and positioning method as described in any one of claims 1 to 10.

Images