Method, device and medium for estimating pose of on-off button of electrical cabinet based on machine vision

Abstract

The application discloses a kind of based on machine vision's electrical cabinet opening and closing button pose estimation method, equipment and medium, comprising:1 carries out button detection to image, and extracts button ROI area with target detection network;2 carries out edge extraction to ROI area, obtains edge point set;3 based on circular perspective projection ellipse geometric relation, utilize edge point set to restore the initial pose of button under camera coordinate system;4 according to initial pose, establish space circle model, and project it to image plane, obtain projection profile point set;5 based on projection profile point set, establish the two-way probability correlation error model between observation edge and projection profile;6 iteration driving dynamic LM algorithm is used to iteration optimization to model, to update button pose;7 the pose of optimized button is converted to mechanical arm base coordinate system in combination with hand-eye calibration relation, controls mechanical arm to complete pressing action.The application can improve the precision, stability and robustness of electrical cabinet button pose estimation under complex industrial environment.

Description

Technical Field

[0001] This invention relates to the fields of positioning detection and industrial robot control, specifically to a method, device, and medium for estimating the pose of electrical cabinet opening and closing buttons based on machine vision. Background Technology

[0002] With the development of smart grid and substation automation technologies, the use of robotic arms equipped with vision sensors to replace manual operation of electrical cabinets (such as pressing circuit breaker buttons) has become an important trend in industrial applications. In actual operation, accurately estimating the three-dimensional pose of the circular buttons on the electrical cabinet panel is crucial to ensuring precise pressing operations by the robotic arm's end effector. However, the surface of electrical cabinets is mostly made of metal and the ambient lighting is complex, making it prone to specular reflection. In addition, wear and oil on the button surface often lead to partial occlusion or breakage of the button edges in the acquired images. Traditional visual pose estimation algorithms often use the nearest neighbor Chamfer distance when constructing reprojection errors. This method is prone to gradient discontinuity when dealing with incomplete contours containing a lot of noise and breaks, causing the matching to get stuck in a local optimum. On the other hand, the conventional Levenberg-Marquardt (LM) optimization algorithm usually adopts a fixed adjustment strategy for damping parameters when solving nonlinear pose optimization problems. This approach not only suffers from slow convergence and a high tendency to diverge when facing complex working conditions and nonlinear error spaces, but also struggles to meet the high precision and stability requirements of robotic arms for real-time control. Therefore, existing visual pose estimation methods all have certain limitations in practical and complex industrial applications. Summary of the Invention

[0003] This invention addresses the shortcomings of existing technologies by proposing a machine vision-based method, device, and medium for estimating the pose of electrical cabinet opening and closing buttons. The aim is to improve the accuracy and efficiency of target object pose estimation and enhance robustness to edge defects and image noise, thereby improving the accuracy and stability of electrical cabinet button pose estimation in complex industrial environments.

[0004] To achieve the above-mentioned objectives, the present invention adopts the following technical solution: The present invention provides a machine vision-based method for estimating the pose of electrical cabinet opening and closing buttons, characterized by the following steps: S1: Obtain the original image containing the opening and closing buttons, and use the YOLOv11 model to detect the buttons in the original image to obtain the ROI region image of the buttons. Then, mark the upper left corner coordinates of the ROI region in the original image as... The ROI region is a pressable circular area with color features in the center of the button. S2: Perform edge extraction on the ROI region image to obtain the edge point set of the pressable circular region. ;in, Indicates the first The coordinates of the edge points in the ROI region image. This represents the total number of edge points; S3: Yes Perform ellipse fitting to obtain fitting coefficients, which are then used to calculate the geometric center of the ellipse. Long axis short axis and principal axis direction angle Thus, the plane normal vector of the button is obtained. and depth Based on the radius of the pressable circular area and the camera's intrinsic parameter matrix This yields the translation vector of the pressable circular area in the camera coordinate system. With rotation matrix This allows us to obtain the initial pose of the button in the camera coordinate system. ; S4: The button pose is iteratively optimized using the dynamic Levenberg-Marquardt algorithm driven by iteration to obtain the optimal button pose in the camera coordinate system. ; S5: Use equation (27) to optimize the button's pose. Transform from the camera coordinate system to the robot arm base coordinate system to obtain the button pose in the base coordinate system. : (27) In equation (27), This represents the transformation matrix of the robotic arm's end-effector coordinate system relative to the base coordinate system; The transformation matrix represents the camera coordinate system relative to the robotic arm's end effector coordinate system; S6: Based on the button pose in the base coordinate system The control system uses the end effector of the robotic arm to position and press the circuit breaker buttons.

[0005] The method for estimating the pose of the opening and closing buttons of an electrical cabinet based on machine vision, as described in this invention, is also characterized in that step S4 includes: S4.1: Define the current iteration number as and initialize Initialize the first The button pose in the next iteration is ; Initialize the first Damping parameters in the next iteration and adjustment coefficient ; S4.2: According to and radius Determine the first The spatial circle model under the iteration is projected onto the image plane to obtain the projected ellipse. Uniform sampling is then performed along its contour to obtain the... The projection point set in the next iteration ;in, Indicates the first The first iteration on the projected ellipse Coordinates of the projection points The number of projection points on the contour; S4.3: According to and The distance between them, calculate the first The bidirectional probabilistic association weights under the nth iteration are thus obtained. Error function of bidirectional Soft Chamfer reprojection in the next iteration ; S4.4: Based on the first The bidirectional probabilistic association weights under the nth iteration are obtained. Lie algebra increment of the optimal pose in the next iteration ; S4.5: Using equation (24) to obtain the first... Button pose in the next iteration : (twenty four) In equation (24), Indicates the use of the Wedge operator Will The resulting 4×4 matrix after mapping; S4.6: The first Button pose in the next iteration After being projected onto the image plane, the first... The projected ellipse under the next iteration is uniformly sampled along its contour to obtain the first iteration. The projection point set in the next iteration Therefore, based on and Get the first Error function value of bidirectional Soft Chamfer reprojection in the next iteration ; S4.7: Calculate the first using equation (25) The objective function gain ratio in the next iteration : (25) In equation (25), This represents the predicted reprojection error output by the local quadratic approximation model when the Lie algebra increment is zero. Indicates application Then, the predicted reprojection error output by the local quadratic approximation model; S4.8: When season and control the first Damping parameters in the next iteration and adjustment coefficient Increase synchronously, thus obtaining the first Damping parameters in the next iteration and adjustment coefficient Otherwise, keep Unchanged, and controlling the first Damping parameters in the next iteration Decrease, adjustment coefficient Reset or remain within the preset range to obtain the first Damping parameters in the next iteration and adjustment coefficient ; S4.9: If Once the maximum number of iterations is reached, stop the calculation and proceed with the current pose. As the optimized button's best pose output, let it be denoted as Otherwise, let Assign to Return to step S4.2 and continue iterative execution.

[0006] Furthermore, S4.3 includes: S4.3.1: Using equation (14) to obtain and Square Euclidean distance between : (14) S4.3.2: Calculate the first... using equations (15) and (16) respectively. The edge point and the first The projection point at the th ... Positive probability association weights in the next iteration Weights associated with inverse probability : (15) (16) In equations (15) and (16), For the first The edge point and the first The projection point at the th ... Squared Euclidean distance in the next iteration For the first The edge point and the first The projection point at the th ... Squared Euclidean distance in the next iteration This is a temperature parameter used to control the smoothness of the probability correlation distribution; S4.3.3: Using equation (17) to obtain : (17) In equation (17), This is the two-way error balance coefficient.

[0007] Furthermore, S4.4 includes: S4.4.1: Using equation (18) to obtain the first... The th iteration scalar residuals corresponding to each edge point Thus, the dimension is reprojection residual column vector : (18) S4.4.2: Calculate the first... In the next iteration, the error function with respect to the Lie algebra perturbation variable Jacobian matrix ; S4.4.3: Calculate the first... The column vector of the objective function gradient in the next iteration ;in, Indicates transpose; S4.4.4: Calculate the first using equation (22) Driving dynamic weights in the next iteration : (twenty two) In equation (22), The constant parameter is set; S4.4.5: Calculate the first using equation (23) Lie algebra increment of pose in the next iteration : (twenty three) In equation (23), Indicates the first Damping parameters in the next iteration This represents the matrix inversion operation. This indicates the norm operation.

[0008] The present invention provides an electronic device, characterized in that it includes: a memory for storing a computer program; and a processor for executing the computer program to implement the machine vision-based electrical cabinet opening and closing button pose estimation method as described above.

[0009] The present invention provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the machine vision-based electrical cabinet opening and closing button pose estimation method.

[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention uses a bidirectional Soft Chamfer model based on probabilistic correlation to construct the reprojection error, which effectively overcomes the gradient discontinuity problem caused by edge breakage and local occlusion in traditional rigid matching, and greatly improves the robustness of the system in complex industrial environments.

[0011] 2. This invention introduces adaptive fusion weights into the Levenberg-Marquardt optimization framework, jointly updating the LM direction and the normalized gradient descent direction to achieve dynamic adjustment of the optimization direction. In actual substations or industrial scenarios, electrical cabinet panels are often subject to complex physical interferences such as strong specular reflection, shadows, oil stains, and button surface wear, resulting in a highly non-convex reprojection error field with multiple local extrema. This invention, by dynamically adjusting the optimization direction, enables the algorithm to quickly avoid local extrema and prevent iterative divergence when initial deviations are large or image noise causes local edge loss. This ensures that the algorithm maintains high convergence stability and pose estimation accuracy even under extremely harsh field conditions, providing a solid guarantee for the subsequent safe, stable, and non-deviation-prone pressing operations of the robotic arm, and reducing the safety hazards of operation failure or robotic arm collisions caused by pose estimation algorithm divergence.

[0012] 3. This invention performs pose increment optimization in the SE(3) Lie algebra space and completes rigid body transformation updates through exponential mapping. In actual robotic arm calibration and operation scenarios where the "eye is on the hand," due to the relatively long arm length, even small rotational errors of the camera are physically amplified, resulting in severe millimeter-level or even centimeter-level deviations in the end-effector pressing position. This invention is based on... The unconstrained optimization parameter update method naturally guarantees the orthogonality of the rotation matrix, avoiding the additional nonlinear normalization error introduced by quaternion or rotation matrix distortion under traditional Euclidean parameter updates. This improves the physical consistency and numerical stability of rotation and translation parameters in three-dimensional space during multiple iterations, ensuring that even in complex three-dimensional poses such as multi-angle and large tilt angles of the electrical cabinet, the robotic arm end effector can accurately conform to the button plane and smoothly advance its stroke along the calculated normal direction, thus improving the success rate of the robot's automatic opening and closing operations. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the overall system structure involved in the present invention; Figure 2 This is a flowchart of the pose estimation method of the present invention; The following numbers are labeled in the diagram: 1. Monocular camera; 2. Gripper; 3. Robotic arm; 4. Robotic arm base; 5. Human-machine interface unit; 6. Control unit; 7. Electrical cabinet; 8. Opening and closing buttons. Detailed Implementation

[0014] In this embodiment, a machine vision-based method for estimating the pose of electrical cabinet opening and closing buttons is applied to applications such as... Figure 1 The electrical cabinet opening / closing button pose estimation device shown includes: 1. a monocular camera; 2. a gripper; 3. a robotic arm; 4. a robotic arm base; 5. a human-machine interface unit; 6. a control unit; 7. an electrical cabinet; and 8. an opening / closing button. The monocular camera 1 is mounted at the end of the robotic arm 3 in a "eye-on-hand" manner, with the optical axis of the monocular camera 1 aligned with the Z-axis of the end of the robotic arm 3. The robotic arm 3 is a 6-DOF robotic arm, carrying the monocular camera 1 to the image acquisition position to perform image acquisition tasks. The gripper 2 is mounted on the flange at the end of the robotic arm 3. The human-machine interface unit 5 is used to monitor the system's operating status. The control unit 6 is used to send commands and process data. The opening / closing button 8, to be operated, is located on the panel of the electrical cabinet 7.

[0015] In this embodiment, as Figure 2 As shown, a machine vision-based method for estimating the pose of the opening and closing buttons of an electrical cabinet includes the following steps: S1: Obtain the original image containing the opening and closing buttons, and use the YOLOv11 model to detect the buttons in the original image to obtain the ROI region image of the buttons. Then, mark the top-left corner of the ROI region in the original image as... The ROI area is the pressable circular area with color features in the center of the button.

[0016] In this embodiment, the following system preparation work needs to be completed before proceeding to S1, including: (A) The button detection model was trained as follows: A robotic arm carrying a monocular camera was used to acquire 60 images of the electrical cabinet panel under different viewing angles and lighting conditions. The labelme tool was used to label the circular on / off button areas in the dataset. 48 images were randomly selected from the dataset as the training set, and the remaining 12 images were used as the test set. The images and their corresponding labels from the training set were input into the YOLOv11 object detection algorithm for training. After the model was trained, it was applied to the test set, and the parameters were adjusted based on the results to obtain a detection model that could stably output the detection model of the button's center ROI region.

[0017] (B) The monocular camera is calibrated using the checkerboard calibration method to obtain the camera's focal length and optical center coordinates. The monocular camera's intrinsic parameter matrix is ​​then obtained using equation (1). The expression is: (1) In equation (1), and camera image coordinate system axis, Normalized focal length on the axis, and Indicates the optical center of the camera.

[0018] (C) The Tsai method is used to perform hand-eye calibration on the robotic arm-camera system. Equation (2) is used to solve for the transformation matrix of the camera coordinate system relative to the coordinate system of the robotic arm end effector (i.e., the center point TCP of the gripper tool). : (2) In equation (2), Let represent the rotation matrix and translation vector of the camera coordinate system relative to the coordinate system of the robotic arm's end effector, respectively.

[0019] S2: Preprocessing and feature extraction of the ROI region image. Specifically, firstly, the ROI image is converted to grayscale and denoised using Gaussian filtering. Then, the Canny edge detection algorithm is used to extract features, obtaining the set of edge pixels of the button's pressable circular area. ,in Indicates the first The coordinates of each edge pixel in the ROI image This represents the total number of edge points.

[0020] S3: Yes An ellipse fitting is performed, thereby constructing an elliptic algebraic equation with the objective of minimizing the sum of algebraic distances using equation (3). To obtain 6 fitting coefficients .

[0021] (3) By analyzing the geometric relationship of the algebraic equation of the ellipse, the abscissa of the geometric center of the ellipse can be calculated using equation (4). The ordinate of the geometric center of the ellipse can be calculated using equation (5). : (4) (5) Calculate the major axis using equation (6) The minor axis is calculated using equation (7). The principal axis direction angle is calculated using equation (8). : (6) (7) (8) Let the angle between the button plane and the image plane be . ,and The plane normal vector of the button is calculated using equation (9). : (9) In equation (9), to eliminate the ambiguity of the direction of the normal vector, the normal vector must satisfy: At that time, the direction of the normal vector is corrected: in: This indicates the direction of the camera's optical axis, ensuring the normal vector points from the button's center outwards. The camera's focal length is used. Given radius and long axis Geometric relationships determine depth : Subsequently, the translation vector of the pressable circular region in the camera coordinate system is calculated using equation (10). : (10) In equation (10), This is the intrinsic parameter matrix of the camera. Simultaneously, the normal vector is used... The transformation yields the rotation matrix of the pressable circular area in the camera coordinate system. Thus, the initial pose of the button in the camera coordinate system, as shown in equation (11), is obtained. : (11) S4: Iterative optimization of the button pose is performed based on the iteratively driven dynamic Levenberg-Marquardt algorithm to obtain the optimal button pose in the camera coordinate system. ; S4.1: Define the current iteration number as and initialize Initialize the first The button pose in the next iteration is ; Initialize the first Damping parameters in the next iteration and adjustment coefficient ; S4.2: According to and radius Determine the first The spatial circle model under the next iteration. First, the spatial circle model is established in the local coordinate system of the button using equation (12): (12) In equation (12), ,in This represents the number of circumferential sampling points used for uniform sampling along the projected ellipse contour.

[0022] Transform spatial points to camera coordinate system using the current pose. And combined with the camera intrinsic parameter matrix Perspective projection is performed using equation (13): (13) In equation (13), the coordinates of the projection point in the image plane are obtained after normalization. This constitutes the first The projection point set in the next iteration ,in, Indicates the first The first iteration on the projected ellipse Coordinates of the projection points This represents the number of projection points on the contour.

[0023] S4.3: According to and The distance between them, calculate the first The bidirectional probabilistic association weights under the nth iteration are thus obtained. Error function of bidirectional Soft Chamfer reprojection in the next iteration Because industrial sites are often affected by local occlusion and fractures, to avoid the gradient discontinuity and getting trapped in local optima caused by the traditional nearest neighbor Chamfer distance, this embodiment adopts a double Soft Chamfer distance model based on probability correlation, which is implemented in the following steps: S4.3.1: Using equation (14) to obtain and Square Euclidean distance between : (14) S4.3.2: Introducing temperature parameters Using equations (15) and (16) respectively, calculate the first... The edge point and the first The projection point at the th ... Positive probability association weights in the next iteration Weights associated with inverse probability : (15) (16) In equations (15) and (16), For the first The edge point and the first The projection point at the th ... Squared Euclidean distance in the next iteration For the first The edge point and the first The projection point at the th ... Squared Euclidean distance in the next iteration The temperature parameter is used to control the smoothness of the probability correlation distribution. The exponential operation transforms the squared Euclidean distance into probability weights between 0 and 1. The exponential decay makes the weights of noise points that are farther away infinitely close to 0, realizing a soft correspondence between observed edge points and projected contour points, thereby improving the algorithm's robustness to occlusion and noise.

[0024] S4.3.3: Construct the first using equation (17) Reprojection error function under the next iteration : (17) In equation (17), This is the two-way error balance coefficient. Dynamically defined as... , This represents the ratio of the number of observed edge points to the number of sampled points of the projected contour, thus balancing the contribution of bidirectional reprojection error.

[0025] S4.4: Based on bidirectional probabilistic association weights, obtain the first... Lie algebra increment of the optimal pose in the next iteration ; To ensure the orthogonality constraint of the rotation matrix during optimization, this embodiment uses the current pose. Mapped to Lie algebra space Perform parameterization. It is a translation vector. Let it be a rotation vector. Use Lie algebra to perturb the variables. Apply a local perturbation to the current pose and solve for the i-th Lie algebra increment of the optimal pose in the next iteration . It is calculated using the following steps: S4.4.1: Using equation (18) to obtain the first... The th iteration scalar residuals corresponding to each edge point Thus, the dimension is reprojection residual column vector : (18) S4.4.2: Calculate the first... In the next iteration, the error function with respect to the Lie algebra perturbation variable Jacobian matrix ;in, For dimension is The partial derivative matrix.

[0026] Jacobian matrix The Behavioral scalar residuals Lie algebra perturbation variables The partial derivatives can be directly calculated using the chain rule as follows: (19) In equation (19), For the first The coordinates of the edge points For the first Coordinates of the projection points Let be the derivative matrix of the projection point coordinates with respect to the Lie algebraic perturbation variables, defined as: (20) In equation (20), These are the three-dimensional coordinates of the current projection point in the camera coordinate system. for The skew-symmetric matrix is ​​represented as: (twenty one) By iterating through all points in the edge point set and arranging the partial derivatives row-wise, a complete Jacobian matrix can be constructed. .

[0027] S4.4.3: Calculate the first... The column vector of the objective function gradient in the next iteration ;in, Indicates transpose; S4.4.4: Calculate the first using equation (22) Driving dynamic weights in the next iteration : (twenty two) In equation (22), The constant parameter is set; as the number of iterations increases... The increase, As it gradually approaches 1, the optimization direction in the later stages tends to be more biased towards the high-precision second-order LM direction.

[0028] S4.4.5: Calculate the Lie algebra increment of the pose using equation (23). : (twenty three) In equation (23), Indicates the first Damping parameters in the next iteration This represents the matrix inversion operation. This indicates the norm operation. This invention jointly updates the LM search direction and the normalized gradient descent direction, utilizing weights. It enables dynamic adjustment of the optimization direction, which helps to quickly avoid local extrema of the non-convex error surface and prevent iterative divergence when the initial deviation is large or the edge is locally missing.

[0029] S4.5: Using equation (24) to obtain the first... Button pose in the next iteration : (twenty four) In equation (24), Indicates the use of the Wedge operator Increment of Lie algebra The mapped 4×4 matrix, and .based on The unconstrained optimization update method naturally guarantees the orthogonality of the rotation matrix from the mathematical and physical level, avoiding the nonlinear normalization error introduced by quaternion or rotation matrix distortion under traditional Euclidean parameter updates. It provides extremely high numerical stability when facing complex working conditions with large tilt angles and multiple angles of electrical cabinets.

[0030] S4.6: Using the first Button pose in the next iteration Perform spatial circular projection, that is, project the spatial circle onto the image plane to obtain the projected ellipse, and then uniformly sample along its contour to obtain the first ellipse. The projection point set in the next iteration Using edge point sets and Calculate the first Error function value of bidirectional Soft Chamfer reprojection in the next iteration .

[0031] S4.7: Using Equation (25), calculate the ratio of the actual decrease in bidirectional Soft Chamfer reprojection error to the model-predicted decrease. The objective function gain ratio in the next iteration This is used to evaluate the accuracy of the approximate model's predictions. (25) In equation (25), the local quadratic approximation model The error function of the bidirectional Soft Chamfer reprojection is a local quadratic Taylor approximation function at the current pose iteration point, and its independent variable is defined. For the universal perturbation increment in the Lie algebra space, the approximate function is... The expanded expression is as follows: (26) This represents the predicted reprojection error output by the local quadratic approximation model when the Lie algebra increment is zero. Indicates application Then, the reprojection error prediction value output by the local quadratic approximation model.

[0032] S4.8: In this embodiment, preferably, the damping range is set to... The adjustment coefficient range is :when season and control the first Damping parameters in the next iteration and adjustment coefficient Increase synchronously according to the incremental strategy: Thus, the first Damping parameters in the next iteration and adjustment coefficient Otherwise, retain the result calculated in step S4.5. and control the first Damping parameters in the next iteration Decrease using a decreasing strategy: adjustment coefficient Reset or remain within the preset range to obtain the first Damping parameters in the next iteration and adjustment coefficient .

[0033] S4.9: If Once the maximum number of iterations is reached, stop the calculation and proceed with the current pose. As the optimized button's best pose output, let it be denoted as Otherwise, let Assign to Return to step S4.2 and continue iterative execution.

[0034] S5: Use equation (27) to optimize the button's pose. Transform from the camera coordinate system to the robot arm base coordinate system to obtain the button pose in the base coordinate system. : (27) In equation (27), The transformation matrix represents the coordinate system of the robotic arm's end effector relative to the base coordinate system (obtained directly from the robot's forward kinematics joint angles). The transformation matrix represents the camera coordinate system relative to the end effector coordinate system of the robotic arm (obtained by hand-eye calibration of the robotic arm-camera system using the Tsai method).

[0035] S6: Based on the button pose in the base coordinate system The control unit 6 generates the corresponding forward and inverse kinematic trajectories of the robot, controlling the robotic arm 3 to precisely move the end effector gripper 2 to the preparatory position in the direction of the button's normal. Subsequently, the control unit 6 issues a command to control the gripper 2 to close (using the closed end of the gripper to form a stable pressing contact), and advances linearly along the calculated normal vector direction, safely, stably, and without deviation, completing the positioning and pressing operation of the electrical cabinet's opening and closing buttons. This significantly reduces the risk of collisions with equipment caused by pose estimation divergence and improves the system's success rate in harsh field conditions.

[0036] In this embodiment, an electronic device includes: a memory for storing a computer program that performs the above-described method; and a processor for executing the program stored in the memory.

[0037] In this embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the above-described method.

Claims

1. A method for estimating the pose of opening and closing buttons in an electrical cabinet based on machine vision, characterized in that, Includes the following steps: S1: Obtain the original image containing the opening and closing buttons, and use the YOLOv11 model to detect the buttons in the original image to obtain the ROI region image of the buttons. Then, mark the upper left corner of the ROI region in the original image as... The ROI region is a pressable circular area with color features in the center of the button. S2: Perform edge extraction on the ROI region image to obtain the edge point set of the pressable circular region. ;in, Indicates the first The coordinates of the edge points in the ROI region image. This represents the total number of edge points; S3: Yes Perform ellipse fitting to obtain fitting coefficients, which are then used to calculate the geometric center of the ellipse. Long axis short axis and principal axis direction angle Thus, the plane normal vector of the button is obtained. and depth Based on the radius of the pressable circular area and the camera's intrinsic parameter matrix This yields the translation vector of the pressable circular area in the camera coordinate system. With rotation matrix This allows us to obtain the initial pose of the button in the camera coordinate system. ; S4: The button pose is iteratively optimized using the iteratively driven dynamic Levenberg-Marquardt algorithm to obtain the optimal button pose in the camera coordinate system. ; S5: Use equation (27) to optimize the button's pose. Transform from the camera coordinate system to the robot arm base coordinate system to obtain the button pose in the base coordinate system. : (27) In equation (27), This represents the transformation matrix of the robotic arm's end-effector coordinate system relative to the base coordinate system; The transformation matrix represents the camera coordinate system relative to the robotic arm's end effector coordinate system; S6: Based on the button pose in the base coordinate system The control system uses the end effector of the robotic arm to position and press the circuit breaker buttons.

2. The method for estimating the pose of the opening and closing buttons of an electrical cabinet based on machine vision according to claim 1, characterized in that, S4 include: S4.1: Define the current iteration number as and initialize Initialize the first The button pose in the next iteration is ; Initialize the first Damping parameters in the next iteration and adjustment coefficient ; S4.2: According to and radius Determine the first The spatial circle model under the iteration is projected onto the image plane to obtain the projected ellipse. Uniform sampling is then performed along its contour to obtain the... The projection point set in the next iteration ;in, Indicates the first The first iteration on the projected ellipse Coordinates of the projection points The number of projection points on the contour; S4.3: According to and The distance between them, calculate the first The bidirectional probabilistic association weights under the nth iteration are thus obtained. Error function of bidirectional Soft Chamfer reprojection in the next iteration ; S4.4: Based on the first The bidirectional probabilistic association weights under the nth iteration are obtained. Lie algebra increment of the optimal pose in the next iteration ; S4.5: Using equation (24) to obtain the first... Button pose in the next iteration : (24) In equation (24), Indicates the use of the Wedge operator Will The resulting 4×4 matrix after mapping; S4.6: The first Button pose in the next iteration After being projected onto the image plane, the first... The projected ellipse under the next iteration is uniformly sampled along its contour to obtain the first iteration. The projection point set in the next iteration Based on and Get the first Error function value of bidirectional Soft Chamfer reprojection in the next iteration ; S4.7: Calculate the first using equation (25) Gain ratio of objective function in the next iteration : (25) In equation (25), This represents the predicted reprojection error output by the local quadratic approximation model when the Lie algebra increment is zero. Indicates application Then, the predicted reprojection error output by the local quadratic approximation model; S4.8: When season and control the first Damping parameters in the next iteration and adjustment coefficient Increase synchronously, thus obtaining the first Damping parameters in the next iteration and adjustment coefficient Otherwise, keep Unchanged, and controlling the first Damping parameters in the next iteration Decrease, adjustment coefficient Reset or remain within the preset range to obtain the first Damping parameters in the next iteration and adjustment coefficient ; S4.9: If Once the maximum number of iterations is reached, stop the calculation and proceed with the current pose. As the optimized button's best pose output, let it be denoted as Otherwise, let Assign to Return to step S4.2 and continue iterative execution.

3. The method for estimating the pose of electrical cabinet opening and closing buttons based on machine vision according to claim 2, characterized in that, S4.3 includes: S4.3.1: Using equation (14) to obtain and Square Euclidean distance between : (14) S4.3.2: Calculate the first... using equations (15) and (16) respectively. The edge point and the first The projection point at the th ... Positive probability association weights in the next iteration Weights associated with inverse probability : (15) (16) In equations (15) and (16), For the first The edge point and the first The projection point at the th ... Squared Euclidean distance in the next iteration For the first The edge point and the first The projection point at the th ... Squared Euclidean distance in the next iteration This is a temperature parameter used to control the smoothness of the probability correlation distribution; S4.3.3: Using equation (17) to obtain : (17) In equation (17), This is the two-way error balance coefficient.

4. The method for estimating the pose of electrical cabinet opening and closing buttons based on machine vision according to claim 3, characterized in that, S4.4 includes: S4.4.1: Using equation (18) to obtain the first... The iteration of the ... scalar residuals corresponding to each edge point Thus, the dimension is reprojection residual column vector : (18) S4.4.2: Calculate the first... In the next iteration, the error function with respect to the Lie algebra perturbation variable Jacobian matrix ; S4.4.3: Calculate the first... The column vector of the objective function gradient in the next iteration ;in, Indicates transpose; S4.4.4: Calculate the first using equation (22) Driving dynamic weights in the next iteration : (22) In equation (22), The constant parameter is set; S4.4.5: Calculate the first using equation (23) Lie algebra increment of pose in the next iteration : (23) In equation (23), Indicates the first Damping parameters in the next iteration This represents the matrix inversion operation. This indicates the norm operation.

5. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor is configured to implement the machine vision-based electrical cabinet switchgear pose estimation method as described in any one of claims 1-4 when executing the computer program.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the machine vision-based electrical cabinet opening and closing button pose estimation method as described in any one of claims 1-4.

Images