Robot positioning method based on multi-view weighted fusion

Abstract

The invention discloses a robot positioning method based on multi-view weighted fusion, and belongs to the field of robot positioning, and the method comprises the steps: 1, constructing a mobile robot positioning system; 2, single-view-angle data collection is carried out, an image containing a positioning label and the pose of the mechanical arm are obtained, and an observation result is calculated; 3, planning and collecting multiple groups of multi-view data, and calculating a preliminary estimation pose matrix; 4, calculating a preliminary fusion pose matrix as a reference; 5, calculating an average residual vector and an information matrix in the Lie algebra space; 6, performing weighted fusion on the average residual vector by using the information matrix, and calculating a fusion pose matrix of the positioning label relative to the mechanical arm base; 7, obtaining an operation pose; 8, in the actual operation stage, operation pose deviation generated by chassis displacement is corrected through multi-view-angle positioning; according to the method, the error distribution difference of the preliminarily estimated poses under different visual angles is utilized, and the multi-visual-angle observation results are weighted and fused through the information matrix, so that the positioning precision can be improved.

Description

Technical Field

[0001] This invention relates to the field of robot localization, and more specifically to a robot localization method based on multi-view weighted fusion. Background Technology

[0002] With the continuous advancement of automation technology, mobile robots integrating chassis and robotic arms are increasingly widely used in industrial manufacturing, laboratory automation, and other fields. Compared to fixed robotic arms, mobile robots possess better environmental adaptability and flexible path planning capabilities, enabling them to autonomously navigate and perform tasks in complex and changing environments. However, limited by non-ideal factors such as the kinematic accuracy of the mobile robot chassis and the ground environment, there is a centimeter-level deviation between the actual parking position and the target position. Currently, visual positioning technology based on visual positioning tags is being widely used to correct this parking position deviation, ensuring that the robotic arm can stably and reliably perform precision operations.

[0003] Existing methods typically acquire images of the positioning tag and the robot arm pose from a single viewpoint, estimate the tag's pose in the camera coordinate system using the PnP algorithm, and then map the pose to the robot arm's base coordinate system for single-view localization. However, due to calibration errors, sensor random noise, and the inherent limitations of monocular vision's depth axis observation accuracy, single-view localization results suffer from significant errors. To improve accuracy, some existing solutions introduce multi-view fusion strategies, integrating observation data from multiple views using decoupling and averaging (i.e., averaging the translation and rotation components separately) or minimizing reprojection errors. However, existing multi-view fusion methods assume isotropic observation noise and ignore the correlation between the pose estimation error distribution and viewpoint selection. This equal-weight fusion logic fails to reflect the uncertainties in each axis of the initial pose estimation from different views, making optimal fusion impossible and limiting the improvement of localization accuracy. Summary of the Invention

[0004] The present invention addresses the shortcomings of the existing technology by proposing a robot localization method based on multi-view weighted fusion. This method aims to improve the accuracy of robot localization by utilizing the differences in the error distribution of the initial pose estimation under different views and by weighted fusion of multi-view observation results through an information matrix.

[0005] To achieve the above objectives, the present invention provides the following technical solution: The robot localization method based on multi-view weighted fusion of the present invention is characterized by the following steps: Step 1: Construct a mobile robot positioning system, including: a mobile robot, a positioning tag, and an object to be worked on; the mobile robot includes: a robot chassis, a robotic arm, and a camera fixed to the end effector of the robotic arm; wherein, the robot chassis is located at an initial position P; the positioning tag contains... The feature point, the first The three-dimensional spatial positions of the feature points are denoted as follows: , ; The camera and robotic arm were calibrated to obtain the internal parameters of the calibrated camera. Distortion parameters after camera calibration Hand-eye matrix after hand-eye calibration ; Step 2: Control the robotic arm to drive the camera to perform single-view data acquisition on the positioning tag, and obtain an image pose pair, including: the captured image of the positioning tag. And filming robotic arm position at time ; and according to and The pose of the positioning tag is estimated to obtain a preliminary estimated pose matrix of the positioning tag relative to the robot arm base. ; Step 3, based on planning A single shooting angle; thereby controlling the robotic arm to drive the camera to perform [positioning] on the positioning tag. A group of multi-view data acquisitions were obtained. Image pose pairs; among which... The number of groups for multi-view positioning. The number of viewpoints for multi-view positioning; according to In the image pose pair, the first Group 1 The positioning tag images and robotic arm poses acquired from the first viewpoint are used to estimate the pose of the positioning tag, resulting in the first... Group 1 Preliminary estimated pose matrix of the localization tag relative to the robotic arm base from one viewpoint ; Step 4, for Preliminary fusion is performed to obtain the preliminary fused pose matrix of the positioning tag relative to the robotic arm base. ; Step 5, based on as well as Calculate the first Group 1 Pose residual vector from each viewpoint Therefore, based on The first result was obtained using statistical analysis. The average residual vector of each viewpoint and the Information matrix from a single perspective ; Step 6, according to right Weighted fusion is performed to obtain the fused pose residual vector. ; and according to and Calculate the fused pose matrix of the positioning tag relative to the robotic arm base. ; Step 7: Guide the robotic arm to traverse the preset paths using the teach mode. The first working pose is used to perform operations on the object to be worked on; when the robotic arm reaches the first position in the base coordinate system... Each working position At that time, the first position in the positioning tag coordinate system is calculated using equation (12). Each working position , : (12) In equation (12), the superscript Indicates the inverse matrix operation; Step 8: When the robot chassis moves to another position and then returns to the initial position P, record the actual stopping position of the robot chassis as P'. Single-view data acquisition and pose estimation are performed on the positioning tag at point P' to obtain the preliminary estimated pose matrix of the positioning tag relative to the robotic arm base when the robot chassis is at point P'. ; At point P', multi-view acquisition, pose estimation, preliminary fusion, average residual vector calculation, and pose fusion are performed on the positioning tag to obtain the fused pose matrix of the positioning tag relative to the robotic arm base when the robot chassis is at point P'. ; Using equation (13), calculate the first time base coordinate system of the chassis at point P'. Correction of the working position Thus obtain A corrected working pose is used to control the robotic arm to move and complete the task on the object to be worked on: (13).

[0006] The robot localization method of the present invention is also characterized in that pose estimation is performed in step 2 according to the following process: Extracting based on image grayscale features and geometric constraints middle The position of each feature point is obtained. The pixel coordinates of each feature point; and combined with and as well as The PnP algorithm is used to solve for the estimated pose matrix of the localization tag relative to the camera. Therefore, the preliminary estimated pose matrix of the positioning tag relative to the robotic arm base is calculated using equation (1). : (1).

[0007] Furthermore, step 3 includes: Step 3.1: Select the location tag to be photographed in the camera coordinate system. Individual pose ,in, Indicates the first position in the camera coordinate system The pose of the positioning tag is captured from multiple angles. ; Step 3.2: Calculate the first step using equation (2). The target pose of the robotic arm from multiple perspectives Thus, the first Group The target pose of the robotic arm from multiple perspectives; (2) In equation (2), the superscript Indicates the inverse matrix operation; Step 3.3, in the... Group 1 When acquiring data from a single perspective, control the robotic arm to move to... And use a camera to capture images of the location tags. Record the robotic arm's pose when capturing images. , ; Step 3.4: Repeat the process in step 3.3. Positioning is performed from multiple perspectives, and a set of multi-view data is obtained; this process is then repeated. Group multi-view data acquisition, and obtained A pair of image poses; Step 3.5, for the first Group 1 Pose estimation is performed on the localization label from the nth viewpoint, and the nth position is calculated using equation (1). Group 1 Preliminary estimated pose matrix of the localization tag relative to the robotic arm base from one viewpoint .

[0008] Furthermore, step 4 includes: Step 4.1: Obtain the preliminary fusion translation vector using equation (3). : (3) In equation (3), for The translation part; Step 4.2: Calculate the quaternion-derived matrix using equation (4). ; and calculate The eigenvector corresponding to the largest eigenvalue is used as the initial quaternion fusion vector. ; (4) In equation (4), for The quaternion representation of the rotated part, with superscript Indicates the transpose operation; Step 4.3, and Combined into a preliminary fusion pose matrix .

[0009] Furthermore, step 5 includes: Step 5.1: Calculate the first step using equation (5). Group 1 Pose residual matrix from each perspective : (5) Step 5.2: Use equation (6) to obtain the first... Group 1 Pose residual vector from each viewpoint : (6) In equation (6), Represents the Lie algebra-logarithmic mapping; Step 5.3: Calculate the first step using equations (7) and (8) respectively. The average residual vector of each viewpoint and the Covariance matrix of each perspective : (7) (8) Step 5.4: Use equation (9) to obtain the first... Information matrix from a single perspective : (9).

[0010] Furthermore, step 6 includes: Step 6.1: Using the information matrix of each perspective as weights, and applying equation (10) to... Average residual vector from each perspective Weighted fusion is performed to obtain the fused pose residual vector. : (10) Step 6.2: Obtain the fused pose matrix using equation (11). : (11) In equation (11), This represents the Lie algebra exponent mapping.

[0011] The present invention provides an electronic device, including a memory and a processor, characterized in that the memory is used to store a program supporting the processor in performing the method described therein, and the processor is configured to execute the program stored in the memory.

[0012] The present invention provides a computer-readable storage medium on which a computer program is stored, characterized in that the computer program is executed by a processor to perform the steps of the method described thereon.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention acquires image pose pairs from multiple perspectives and performs statistical analysis. Compared with single-view positioning, it can effectively suppress random measurement errors from a single observation, thereby improving the accuracy of the positioning results.

[0014] 2. This invention uses the preliminary estimated pose matrix obtained from single-view observation to guide the calculation of the target pose of the robotic arm during multi-view observation. It aims to maintain a high degree of consistency in the shooting pose before and after the chassis moves, effectively avoiding the inaccuracy of error distribution caused by large changes in the viewing angle, thereby ensuring the accuracy of the multi-view positioning results.

[0015] 3. This invention maps the pose residual matrix to the Lie algebra space and performs weighted fusion of pose residual vectors from different viewpoints in the tangent space. The introduction of Lie algebra can locally linearize the nonlinear pose manifold and support the application of different weighting factors to each axis of the pose, thereby realizing refined weighting of the preliminary estimated pose matrix.

[0016] 4. This invention utilizes the differences in the error distribution of the estimated pose matrix under multiple perspectives by weighted fusion. Compared with the traditional equal-weighted fusion method, the weighted fusion method can improve the accuracy of multi-view positioning and effectively correct the docking position deviation of the chassis, thereby ensuring that the robotic arm can complete high-precision operations on the object to be operated during the actual operation stage. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the method of the present invention; Figure 2 This is a schematic diagram of the scene layout for the multi-view localization task of the mobile robot in an embodiment of the present invention; Figure 3 This is a standard deviation distribution diagram of the pose estimation quaternions qx, qy, and qz components under different shooting angles in an embodiment of the present invention. Detailed Implementation

[0018] The technical solution of the present invention will now be further described in conjunction with specific embodiments and the accompanying drawings.

[0019] like Figure 1 As shown in this embodiment, a robot localization method based on multi-view weighted fusion includes the following steps: Step 1, as follows Figure 2 As shown, a mobile robot positioning system is constructed, including: a mobile robot, a positioning tag, and an object to be worked on; the mobile robot includes: a robot chassis, a robotic arm, and a camera fixed on the end effector of the robotic arm. In this embodiment, the robotic arm is a Universal Robots UR5e model; the camera is a Realsense D435i model.

[0020] The robot chassis is located at the initial position P; the positioning tag contains The feature point, the first The three-dimensional spatial positions of the feature points are denoted as follows: , In this embodiment, the positioning tag is a dot array positioning tag, with a total of The nth feature point, its nth The feature point is located at the th feature point The center of a circle.

[0021] The camera and robotic arm were calibrated to obtain the internal parameters of the calibrated camera. Distortion parameters after camera calibration Hand-eye matrix after hand-eye calibration In this embodiment, the specific calibration process is as follows: The camera is calibrated using the Zhang Zhengyou calibration method to obtain the camera's intrinsic parameters. and distortion parameters ,in for The intrinsic parameter matrix, The vector contains radial and tangential distortion parameters; hand-eye calibration is performed using the Tsai-Lenz algorithm to obtain the hand-eye matrix. Hand-eye matrix for The homogeneous transformation matrix.

[0022] Step 2: Control the robotic arm to drive the camera to collect single-view data from the positioning tag, and obtain... and Then, pose estimation is performed to obtain a preliminary estimated pose matrix of the positioning tag relative to the robot arm base. .

[0023] This invention utilizes a preliminary estimated pose matrix to guide the calculation of the target pose of the robotic arm during multi-view observation, aiming to maintain the consistency of the observation perspective before and after chassis movement. By ensuring that the shooting pose of the multi-view data acquisition in the actual operation stage (step 8) is highly consistent with the shooting pose in the modeling stage (steps 3 to 6), it effectively avoids the inaccuracy of error distribution caused by large changes in perspective, thereby ensuring the accuracy of the multi-view positioning results.

[0024] Step 2.1: Control the robotic arm to move above the positioning tag, use a camera to acquire single-view data of the positioning tag, and obtain an image pose pair, including the captured image of the positioning tag. and filming robotic arm position at time , for The homogeneous transformation matrix.

[0025] Step 2.2: Extract based on image grayscale features and geometric constraints. middle The position of each feature point is obtained. The pixel coordinates of each feature point; and combined with and as well as The PnP algorithm is used to solve for the estimated pose matrix of the localization tag relative to the camera. Therefore, the preliminary estimated pose matrix of the positioning tag relative to the robotic arm base is calculated using equation (1). : (1) The result and All The homogeneous transformation matrix.

[0026] Step 3, based on planning One shooting angle; controlling the robotic arm to drive the camera to locate the tag. A group of multi-view data acquisitions were obtained. Image pose pairs; according to In the image pose pair, the first Group 1 The positioning tag images and robotic arm poses acquired from the first viewpoint are used to estimate the pose of the positioning tag, resulting in the first... Group 1 Preliminary estimated pose matrix of the localization tag relative to the robotic arm base from one viewpoint .

[0027] Due to calibration errors, sensor random noise, and the inherent limitations of monocular vision in depth axis observation accuracy, single-view localization results exhibit significant random fluctuations. This invention employs a multi-view strategy, through integration... By statistically analyzing the observation results from each independent perspective, the random measurement error of a single observation can be effectively suppressed, thereby improving the accuracy of the positioning results.

[0028] In the data acquisition process of this invention, by repeated execution A multi-view data acquisition group was used to construct a multi-view positioning sample set, aiming to provide statistical support for the uncertainty assessment of pose estimation at each viewpoint in step 5; in this embodiment, the number of multi-view positioning groups was selected. To fit the error model for each perspective, in the actual operation stage of step 8, in order to simplify the positioning data acquisition process and improve real-time performance, only a single multi-view positioning is performed, that is, the number of multi-view positioning groups when the chassis is at point P' is selected. .

[0029] Step 3.1: Select the location tag to be photographed in the camera coordinate system. Individual pose ,in, Indicates the first position in the camera coordinate system The pose of the positioning tag from each shooting angle. In this embodiment, the number of viewpoints The selected viewpoints include one directly above viewpoint and four obliquely above viewpoints (with pitch angles of 60°-75°, distributed around the perimeter).

[0030] Step 3.2: Calculate the first step using equation (2). The target pose of the robotic arm from multiple perspectives Thus, the first Group The target pose of the robotic arm from multiple perspectives; (2) In equation (2), the superscript This represents the inverse matrix operation.

[0031] Step 3.3, during the process of... Group 1 When acquiring data from a single perspective, control the robotic arm to move to... And use a camera to capture images of the location tags. Record the robotic arm's pose when capturing images. This forms an image pose pair. ; Step 3.4: Repeat the process in step 3.3. Positioning from multiple perspectives, as a set of multi-view data collection; repeated... Multi-view data acquisition was conducted to obtain... Image pose pairs.

[0032] Step 3.5: Following the pose estimation method in Step 2.2, perform the following steps on the first... Group 1 The pose of the localization label is estimated from the first viewpoint, and then the first position is calculated using Equation (1). Group 1 Preliminary estimated pose matrix of the localization tag relative to the robotic arm base from one viewpoint .

[0033] Step 4, for Preliminary fusion is performed to obtain the preliminary fused pose matrix of the positioning tag relative to the robotic arm base. ; Step 4.1: Obtain the preliminary fusion translation vector using equation (3). : (3) In equation (3), for The translation part.

[0034] Step 4.2: Use the Markley method to fuse quaternions: Calculate the quaternion-derived matrix using equation (4). ; and calculate The eigenvector corresponding to the largest eigenvalue is used as the initial quaternion fusion vector. ; (4) In equation (4), for The quaternion representation of the rotated part is a 4-dimensional column vector; superscript This indicates the transpose operation.

[0035] Step 4.3, and Combined into a preliminary fusion pose matrix The result for The homogeneous transformation matrix.

[0036] Step 5, based on as well as Calculate the first Group 1 Pose residual vector from each viewpoint Therefore, based on The first result was obtained using statistical analysis. The average residual vector of each viewpoint and the Information matrix from a single perspective .

[0037] Since the observation distance and direction differ from different viewpoints, the preliminary estimated pose matrix for each viewpoint is as follows. The error distribution is non-uniform along each axis of translation and rotation. Therefore, differentiated weights should be assigned based on the contribution of each viewpoint to each component of the pose.

[0038] Figure 3 This demonstrates the variation of the standard deviation of the pose estimation quaternions qx, qy, and qz with the observation angle. The shooting angle is defined as follows: a coordinate system is established with the center of the positioning tag as the origin, its x and y axes parallel to the tag plane, and its z axis perpendicular to the tag plane and pointing outwards. Twenty-five poses of the positioning tag are selected for shooting. The x and y coordinates of the translation portion of the shooting pose vary in 5cm increments within the range of -10cm to 10cm, while the z-axis coordinate is fixed at 25cm. The rotation portion of the shooting pose is adjusted based on the translation portion to ensure that the camera's optical axis always points towards the center of the positioning tag.

[0039] Following step 3, perform multiple sets of multi-view localization to obtain a preliminary estimated pose matrix. The number of multi-view positioning groups in this experimental section Number of selected viewpoints ; will the first Group 1 From a perspective The quaternion representation of the rotation part is denoted as the pose estimation quaternion. ,statistics From a perspective The standard deviations of the first three components qx, qy, and qz are shown. The results indicate that when viewed from directly above, the standard deviations of the quaternion's qx and qy components are larger, while the standard deviation of the qz component is smaller, reflecting that the system has high uncertainty in estimating small rotations around the x and y axes, while the estimation accuracy for small rotations around the z axis is higher; when viewed from an oblique angle above, the accuracy distribution shows the opposite trend.

[0040] Based on the above findings, this invention models the error correlation of each viewpoint and axis using an information matrix. Utilizing the differences in error distribution of pose estimation results from different viewpoints, it weights and fuses multi-view observations using the information matrix, thereby improving the overall accuracy of robot localization. Furthermore, considering... The pose estimation results from each viewpoint are highly independent. To simplify the localization data acquisition process, this model assumes that the observation noise from each viewpoint is uncorrelated and does not perform additional modeling for the statistical correlation across viewpoints.

[0041] In the multi-view weighted fusion process of this invention, Lie groups and Lie algebras are introduced as a mathematical framework. The pose information of the robotic arm is composed of translation and rotation, belonging to a special Euclidean group. The corresponding parameter space exhibits non-Euclidean manifold characteristics. In such a nonlinear space, the arithmetic mean method cannot be directly used to achieve a weighted average along each axis. In contrast, the Lie algebra space... As a linear tangent space at the unit element, it allows for linear weighted fusion of small transformations. Therefore, this invention maps the pose residual matrix to the Lie algebra space to achieve weighted fusion under manifold constraints, thereby significantly improving the accuracy of multi-view localization.

[0042] Step 5.1: Calculate the first step using equation (5). Group 1 Pose residual matrix from each perspective : (5) The result for The homogeneous transformation matrix.

[0043] Step 5.2: Use equation (6) to obtain the first... Group 1 Pose residual vector from each viewpoint : (6) In equation (6), Indicates the special Euclidean group To its corresponding Lie algebra The logarithmic mapping; the resulting pose residual vector It is a six-dimensional column vector containing translation and rotation components.

[0044] Step 5.3: Calculate the first step using equations (7) and (8) respectively. The average residual vector of each viewpoint and the Covariance matrix of each perspective : (7) (8) Step 5.4: Use equation (9) to obtain the first... Information matrix from a single perspective : (9) Each covariance matrix obtained and each information matrix All dimensions are .

[0045] Step 6, according to right Weighted fusion is performed to obtain the fused pose residual vector. ; and according to and Calculate the fused pose matrix of the positioning tag relative to the robotic arm base. .

[0046] Step 6.1: Information matrix from various perspectives As the weights, use equation (10) to... Average residual vector from each perspective Weighted fusion is performed to obtain the fused pose residual vector. : (10) The result obtained in equation (10) It is a six-dimensional column vector.

[0047] Step 6.2: Obtain the fused pose matrix using equation (11). : (11) In equation (11), Lie algebras representing special Euclidean groups To special Euclidean groups The exponential mapping; the result for The homogeneous transformation matrix.

[0048] Step 7: Guide the robotic arm to traverse the preset paths using the teach mode. The first working pose is used to perform operations on the object to be worked on; when the robotic arm reaches the first... When the robot arm is in the first working pose, its pose at that time is recorded as the first pose in the base coordinate system. Each working position The for The homogeneous transformation matrix is ​​obtained, and the first homogeneous transformation matrix in the positioning tag coordinate system is calculated using equation (12). Each working position , : (12) In equation (12), the superscript This represents the inverse matrix operation.

[0049] Step 8: During the actual operation phase, control the chassis to return to P from other positions, and record the actual parking position of the chassis as P'; perform single-view and multi-view positioning processes again to obtain the fused pose matrix of the chassis at P'. This allows for the correction of the work position.

[0050] Step 8.1: When the robot chassis moves to another position and then returns to the initial position P, record the actual stopping position of the robot chassis as P'. Due to limitations imposed by non-ideal factors such as the kinematic accuracy of the mobile robot chassis and the ground environment, there is approximately a centimeter-level deviation between the actual docking position P' and the position P of the chassis. The multi-view weighted fusion method proposed in this invention can effectively correct this deviation, ensuring that the robotic arm can still perform high-precision operations on the object under work even with chassis docking position deviations.

[0051] Step 8.2: Following step 2, perform single-view acquisition of the positioning tag at point P' to obtain the positioning tag image. and robotic arm pose Then, pose estimation is performed to obtain the preliminary estimated pose matrix of the positioning tag relative to the robotic arm base when the robot chassis is at point P'. ; Step 8.3: Following steps 3 to 6, perform multi-view acquisition, pose estimation, preliminary fusion, average residual vector calculation, and pose fusion on the localization tag at point P' to obtain the fused pose matrix of the localization tag relative to the robotic arm base when the robot chassis is at point P'. .

[0052] In this embodiment, the multi-view positioning process of the chassis at point P' is as follows: First, according to steps 3.3 and 3.4, a set of multi-view data is collected, that is, the number of multi-view positioning sets when the chassis is at point P'. The number of viewpoints in multi-view positioning is still [number missing]. According to step 3.5, when the chassis is at point P', the calculation is performed. Preliminary estimated pose matrix of the localization tag relative to the robotic arm base from one viewpoint ,in Subsequently, the preliminary fused pose matrix of the chassis at point P' is calculated according to step 4. Based on this, and according to steps 5.1 to 5.3, based on as well as Calculate the first time when the chassis is at point P' The average residual vector of each viewpoint Finally, following step 6, according to... right Weighted fusion is performed to obtain the fused pose matrix of the positioning tag relative to the robotic arm base when the chassis is at point P'. .

[0053] Step 8.4: Calculate the time base coordinate system of the chassis at point P' using equation (13). Correction of the working position Thus obtain A corrected working pose is used to control the robotic arm to move and complete the task on the object to be worked on: (13).

[0054] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the methods described above, and the processor is configured to execute the program stored in the memory.

[0055] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above method.

Claims

1. A robot localization method based on multi-view weighted fusion, characterized in that, Includes the following steps: Step 1: Construct a mobile robot positioning system, including: a mobile robot, a positioning tag, and an object to be worked on; the mobile robot includes: a robot chassis, a robotic arm, and a camera fixed to the end effector of the robotic arm; wherein, the robot chassis is located at an initial position P; the positioning tag contains... The feature point, the first The three-dimensional spatial positions of the feature points are denoted as follows: , ; The camera and robotic arm were calibrated to obtain the internal parameters of the calibrated camera. Distortion parameters after camera calibration Hand-eye matrix after hand-eye calibration ; Step 2: Control the robotic arm to drive the camera to perform single-view data acquisition on the positioning tag, and obtain an image pose pair, including: the captured image of the positioning tag. And filming robotic arm position at time ; and according to and The pose of the positioning tag is estimated to obtain a preliminary estimated pose matrix of the positioning tag relative to the robot arm base. ; Step 3, based on planning A single shooting angle; thereby controlling the robotic arm to drive the camera to perform [positioning] on the positioning tag. Multiple perspective data acquisitions were conducted to obtain... Image pose pairs; among which... The number of groups for multi-view positioning. The number of viewpoints for multi-view positioning; according to In the image pose pair, the first Group 1 The positioning tag images and robotic arm poses acquired from the first viewpoint are used to estimate the pose of the positioning tag, resulting in the first... Group 1 Preliminary estimated pose matrix of the localization tag relative to the robotic arm base from one viewpoint ; Step 4, for Preliminary fusion is performed to obtain the preliminary fused pose matrix of the positioning tag relative to the robotic arm base. ; Step 5, based on as well as Calculate the first Group 1 Pose residual vector from each viewpoint Therefore, based on The first result was obtained using statistical analysis. The average residual vector of each viewpoint and the Information matrix from a single perspective ; Step 6, according to right Weighted fusion is performed to obtain the fused pose residual vector. ; and according to and Calculate the fused pose matrix of the positioning tag relative to the robotic arm base. ; Step 7: Guide the robotic arm to traverse the preset paths using the teach mode. The first working pose is used to perform operations on the object to be worked on; when the robotic arm reaches the first position in the base coordinate system... Each working position At that time, the first position in the positioning tag coordinate system is calculated using equation (12). Each working position , : (12) In equation (12), the superscript Indicates the inverse matrix operation; Step 8: When the robot chassis moves to another position and then returns to the initial position P, record the actual stopping position of the robot chassis as P'. Single-view data acquisition and pose estimation are performed on the positioning tag at point P' to obtain the preliminary estimated pose matrix of the positioning tag relative to the robotic arm base when the robot chassis is at point P'. ; At point P', multi-view acquisition, pose estimation, preliminary fusion, average residual vector calculation, and pose fusion are performed on the positioning tag to obtain the fused pose matrix of the positioning tag relative to the robotic arm base when the robot chassis is at point P'. ; Using equation (13), calculate the first time base coordinate system of the chassis at point P'. Correction of the working position Thus obtain A corrected working pose is used to control the robotic arm to move and complete the task on the object to be worked on: (13)。 2. The robot localization method according to claim 1, characterized in that, In step 2, pose estimation is performed according to the following process: Extracting based on image grayscale features and geometric constraints middle The position of each feature point is obtained. The pixel coordinates of each feature point; and combined with and as well as The PnP algorithm is used to solve for the estimated pose matrix of the localization tag relative to the camera. Therefore, the preliminary estimated pose matrix of the positioning tag relative to the robotic arm base is calculated using equation (1). : (1)。 3. The robot positioning method according to claim 2, characterized in that, Step 3 includes: Step 3.1: Select the location tag to be photographed in the camera coordinate system. Individual pose ,in, Indicates the first position in the camera coordinate system The pose of the positioning tag is captured from multiple angles. ; Step 3.2: Calculate the first step using equation (2). The target pose of the robotic arm from multiple perspectives Thus, the first Group The target pose of the robotic arm from multiple perspectives; (2) In equation (2), the superscript Indicates the inverse matrix operation; Step 3.3, in the... Group 1 When acquiring data from a single perspective, control the robotic arm to move to... And use a camera to capture images of the location tags. Record the robotic arm's pose when capturing images. , ; Step 3.4: Repeat the process in step 3.

3. Positioning is performed from multiple perspectives, and a set of multi-view data is obtained; this process is then repeated. Group multi-view data acquisition, and obtained A pair of image poses; Step 3.5, for the first Group 1 Pose estimation is performed on the localization label from the nth viewpoint, and the nth position is calculated using equation (1). Group 1 Preliminary estimated pose matrix of the localization tag relative to the robotic arm base from one viewpoint .

4. The robot positioning method according to claim 3, characterized in that, Step 4 includes: Step 4.1: Obtain the preliminary fusion translation vector using equation (3). : (3) In equation (3), for The translation part; Step 4.2: Calculate the quaternion-derived matrix using equation (4). ; and calculate The eigenvector corresponding to the largest eigenvalue is used as the initial quaternion fusion vector. ; (4) In equation (4), for The quaternion representation of the rotated part, with superscript Indicates the transpose operation; Step 4.3, and Combined into a preliminary fusion pose matrix .

5. The robot localization method according to claim 4, characterized in that, Step 5 includes: Step 5.1: Calculate the first step using equation (5). Group 1 Pose residual matrix from each perspective : (5) Step 5.2: Use equation (6) to obtain the first... Group 1 Pose residual vector from each viewpoint : (6) In equation (6), Represents the Lie algebra-logarithmic mapping; Step 5.3: Calculate the first step using equations (7) and (8) respectively. The average residual vector of each viewpoint and the Covariance matrix of each perspective : (7) (8) Step 5.4: Use equation (9) to obtain the first... Information matrix from a single perspective : (9)。 6. The robot localization method according to claim 5, characterized in that, Step 6 includes: Step 6.1: Using the information matrix of each perspective as weights, and applying equation (10) to... Average residual vector from each perspective Weighted fusion is performed to obtain the fused pose residual vector. : (10) Step 6.2: Obtain the fused pose matrix using equation (11). : (11) In equation (11), This represents the Lie algebra exponent mapping.

7. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports a processor in executing the method of any one of claims 1-6, the processor being configured to execute the program stored in the memory.

8. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program is executed by a processor to perform the steps of the method according to any one of claims 1-6.

Images