Visual guidance robot arm micro-operation control method and system

Abstract

The application provides a visual guidance mechanical arm micro-operation control method and system, which comprises the following steps: analyzing and processing real-time collected power distribution room operation scene images, identifying operation objects and feature points, generating pose parameters and type labels of the operation objects, and simultaneously collecting real-time state parameters of a mechanical arm body and an end clamp; based on the type labels and the pose parameters of the operation objects, corresponding micro-operation modes are matched, and a global motion path of the mechanical arm and an end micro-operation action sequence are generated; based on visual real-time feedback and end force sensing data, force-position hybrid closed-loop control is performed on the motion and the micro-operation action of the mechanical arm, motion deviation and operation force parameters are corrected in real time, and the mechanical arm and the end clamp are driven to perform corresponding operation actions; operation result verification is performed through visual collection of post-operation scene images, and an operation completion report or a supplementary operation instruction is generated. The application can realize high-reliability and high-precision execution of power distribution room micro-operation.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, specifically to a vision-guided robotic arm micro-manipulation control method and system. Background Technology

[0002] High and low voltage substations are the core hubs of power transmission and distribution networks. The operation of drawer-type and frame-type switches, button clicks, knob turns, and handcart insertion and removal within the cabinets are crucial maintenance aspects ensuring the safe and stable operation of the power distribution system. As the power industry transitions towards unmanned and intelligent maintenance, power inspection robots are widely used for environmental monitoring and equipment inspection in substations. However, most are still limited to "patrolling" rather than "operating," failing to meet the high-risk, high-precision operational requirements of the cabinet mechanisms. This has become a core bottleneck for the full-process unmanned maintenance of substations.

[0003] Existing power maintenance robots with operational capabilities are mostly specialized devices developed for single cabinet types and single operational objects. They are generally limited to a single equipment type and fixed operating mode, failing to cover the multi-scenario and multi-type maintenance operation needs of power distribution rooms. They lack a unified micro-operation control strategy for multiple task scenarios such as button pressing, knob turning, clamp holding, and handcart operation. Existing solutions mostly adopt pre-programmed fixed trajectories or open-loop control methods, which lack adaptive compensation capabilities for cabinet installation deviations, visual positioning errors, cumulative errors at the end of the robotic arm, and on-site operational disturbances. This directly leads to insufficient operational accuracy and stability, making them unsuitable for continuous and reliable operation under complex working conditions.

[0004] Existing technologies have significant shortcomings in areas such as multi-grip adaptation, compatibility with multiple manipulated objects, unified control of multiple action modes, and real-time error closed-loop compensation, making it impossible to continuously meet the requirements of high precision and high reliability under complex working conditions. Therefore, developing a vision-guided, high-precision, and universal micro-manipulation control technology for robotic arms has become a key technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] Based on this, and in response to the shortcomings of existing technologies, the present invention provides a vision-guided robotic arm micro-operation control method and system to solve the problem that existing technologies cannot continuously meet the requirements of high precision and high reliability under complex working conditions.

[0006] To solve the above-mentioned technical problems, the first aspect of the present invention proposes: A vision-guided robotic arm micro-manipulation control method, wherein the method includes: The real-time acquired images of the power distribution room operation scene are analyzed and processed to identify the operation object and feature points, generate the pose parameters and type labels of the operation object, and simultaneously acquire the real-time status parameters of the robotic arm body and the end effector. Based on the type label and pose parameters of the operation object, the corresponding micro-operation mode is matched, and the global motion path and end-effector micro-operation action sequence of the robotic arm are generated according to the real-time status parameters of the robotic arm body and the end-effector. Based on real-time visual feedback and end-effector force sensing data, the motion and micro-operation actions of the robotic arm are controlled by a force-position hybrid closed-loop system. The motion deviation and operation force parameters are corrected in real time, and the robotic arm body and the end-effector are driven to perform corresponding operation actions. After visually acquiring scene images of the operation, the operation results are verified, and an operation completion report or supplementary operation instructions are generated based on the verification results.

[0007] The beneficial effects of this invention are as follows: This invention provides precise pose and type benchmarks for micro-operations through image analysis of the substation work scene and accurate identification of the operating object, solving the problem of insufficient positioning accuracy of the operating object under complex working conditions; by matching the corresponding micro-operation mode based on the operation type, it achieves universal adaptation for various types of substation maintenance operations, breaking through the limitations of the single operation mode in existing technologies; through the force-position hybrid closed-loop control that integrates real-time visual feedback and end-point force sensing, it can correct motion deviations and operating force parameters in real time, effectively offsetting visual deviations, installation errors, and operational disturbances, significantly improving operational accuracy and stability; through the operation result verification and supplementary operation closed-loop mechanism, it ensures a high success rate of operations, continuously meeting operational requirements under complex working conditions, and realizing unmanned, highly reliable, and high-precision execution of the entire substation micro-operation process.

[0008] Furthermore, the steps of analyzing and processing the real-time acquired images of the power distribution room operation scene, identifying the operation object and feature points, generating the pose parameters and type labels of the operation object, and simultaneously acquiring the real-time status parameters of the robotic arm body and the end effector include: By deploying a binocular camera and a global camera at the end of the robotic arm, global images and close-up images of the end of the work scene are acquired simultaneously. The global images and close-up images of the end of the work scene are then preprocessed to remove distortion, denoise, and enhance them to generate a standardized set of work images. The standardized operation image set is identified by a pre-trained target detection network, and the operation object types such as buttons, knobs, drawer switches, frame switches and handcart mechanisms are distinguished and corresponding type labels are generated. At the same time, the feature points of each operation object are extracted, and the three-dimensional spatial coordinates and attitude angles of each feature point are calculated by a stereo vision algorithm to generate the pose parameters of the operation object. The robot arm's joint encoder, force sensor, and attitude sensor collect real-time angle and angular velocity parameters of each joint, clamping force and six-dimensional force / torque parameters of the end effector, and real-time pose parameters of the robot arm body, generating real-time status parameters of the robot arm and end effector.

[0009] Furthermore, the step of matching the corresponding micro-operation mode based on the type label and pose parameters of the operation object, and generating the global motion path and end-effector micro-operation sequence of the robotic arm according to the real-time state parameters of the robotic arm body and the end effector includes: Based on the type label of the operation object, the corresponding standard operation mode is matched in the preset micro-operation action library. The standard operation mode includes button pressing mode, knob turning mode, switch opening and closing mode, and handcart cranking in and out mode. The action constraint parameters of the corresponding mode are extracted, including stroke threshold, force threshold, speed threshold, and action rhythm. Combining the pose parameters of the object being operated on with the real-time state parameters of the robotic arm, and taking collision-free operation and shortest path as constraints, a global collision-free motion path for the robotic arm from its current pose to the operation preparation position is planned and generated based on the improved A* algorithm. Based on the action constraint parameters of the matched standard operation mode, and combined with the pose parameters of the operation object, the multi-segment continuous micro-operation action sequence of the end effector is generated, and the position control parameters, force control parameters and timing constraints corresponding to each action segment are marked.

[0010] Furthermore, the step of performing force-position hybrid closed-loop control on the movement and micro-manipulation actions of the robotic arm based on real-time visual feedback and end-effector force sensing data, correcting motion deviations and operating force parameters in real time, and driving the robotic arm body and the end-effector to perform corresponding work actions includes: During the movement of the robotic arm along the global motion path, local images of the end effector are acquired at a preset frequency, and the relative pose deviation between the manipulated object and the end effector is calculated in real time. Based on the relative pose deviation, the motion path of the robotic arm is corrected online in real time, driving the robotic arm to accurately reach the operation preparation position. During the execution of micro-operation actions, the corresponding force-position hybrid control mode is switched according to the timing constraints of the micro-operation action sequence. Position closed-loop control is performed on the translational degrees of freedom, and force closed-loop control is performed on the degrees of freedom in contact with the operated object. The system collects six-dimensional force / torque data and visual pose feedback data at the end effector in real time, compares them with the preset parameters of the micro-operation action sequence, generates position correction and force correction amounts, and superimposes them onto the current control loop to adjust the pose and operating force of the end effector in real time, thereby completing the execution of the corresponding micro-operation action.

[0011] Furthermore, the steps of switching the corresponding force-position hybrid control mode according to the timing constraints of the micro-operation action sequence, performing position closed-loop control on the translational degrees of freedom, and performing force closed-loop control on the degrees of freedom in contact with the manipulated object include: The degrees of freedom of each action in the micro-operation action sequence are decoupled to distinguish between non-contact spatial movement degrees of freedom and contact operation interaction degrees of freedom. For the spatial translation degree of freedom, a position control loop is set up, and a PID control algorithm is adopted to perform high-precision position closed-loop control with preset pose parameters as target values. For the aforementioned operational interaction degrees of freedom, a force control loop is set up, and an impedance control algorithm is adopted. With a preset force threshold as a constraint, adaptive force closed-loop control is executed, and the target value of the position loop is adjusted in real time according to the change of contact force.

[0012] Furthermore, the step of visually acquiring scene images after the operation, verifying the operation results, and generating a job completion report or supplementary operation instructions based on the verification results includes: After the micro-operation action is completed, the binocular camera is used to collect global images and local close-up images of the scene after the operation, and the current state features of the operation object are extracted, including button pressing status, knob turning angle, switch opening and closing position, and handcart cranking stroke. The current state characteristics of the operation object are compared with the preset target operation state, and the state deviation value is calculated. When the state deviation value is within the preset allowable range, the operation is judged to be qualified; when the state deviation value exceeds the preset allowable range, the operation is judged to be unqualified. For tasks that pass the operation, the information of the operation object, operation process parameters, and verification results are integrated to generate a standardized task completion report; for tasks that fail the operation, corresponding supplementary operation instructions are generated based on the state deviation value to update the motion path and micro-operation sequence of the robotic arm.

[0013] Furthermore, the step of generating corresponding supplementary operation instructions based on the state deviation value for unqualified operation tasks and updating the motion path and micro-operation sequence of the robotic arm includes: Analyze the reasons for operational failures, distinguish between abnormal types such as positional deviation, insufficient operating force, and insufficient stroke, and match the corresponding supplementary operation mode. Combining the current pose, state deviation value and supplementary operation mode of the operation object, the supplementary operation preparation position and supplementary operation action parameters of the robotic arm are recalculated to generate a supplementary operation micro-operation action sequence; The supplementary micro-operation sequence is sent to the control unit of the robotic arm, and the vision-guided closed-loop control and operation result verification process is repeated until the operation result is qualified or the preset maximum number of retries is reached.

[0014] The second aspect of the present invention proposes: A vision-guided robotic arm micro-manipulation control system, wherein the system comprises: The acquisition module is used to analyze and process real-time acquired images of the power distribution room operation scene, identify the operation object and feature points, generate the pose parameters and type labels of the operation object, and simultaneously acquire the real-time status parameters of the robotic arm body and the end effector. The planning module is used to match the corresponding micro-operation mode based on the type label and pose parameters of the operation object, and generate the global motion path and end-effector micro-operation sequence of the robotic arm according to the real-time status parameters of the robotic arm body and the end-effector. The control module is used to perform force-position hybrid closed-loop control on the movement and micro-operation of the robotic arm based on real-time visual feedback and end-effector force sensing data, correct motion deviation and operation force parameters in real time, and drive the robotic arm body and the end-effector to perform corresponding operation actions. The verification module is used to verify the operation results by visually acquiring scene images after the operation, and to generate an operation completion report or supplementary operation instructions based on the verification results.

[0015] Furthermore, the acquisition module is specifically used for: By deploying a binocular camera and a global camera at the end of the robotic arm, global images and close-up images of the end of the work scene are acquired simultaneously. The global images and close-up images of the end of the work scene are then preprocessed to remove distortion, denoise, and enhance them to generate a standardized set of work images. The standardized operation image set is identified by a pre-trained target detection network, and the operation object types such as buttons, knobs, drawer switches, frame switches and handcart mechanisms are distinguished and corresponding type labels are generated. At the same time, the feature points of each operation object are extracted, and the three-dimensional spatial coordinates and attitude angles of each feature point are calculated by a stereo vision algorithm to generate the pose parameters of the operation object. The robot arm's joint encoder, force sensor, and attitude sensor collect real-time angle and angular velocity parameters of each joint, clamping force and six-dimensional force / torque parameters of the end effector, and real-time pose parameters of the robot arm body, generating real-time status parameters of the robot arm and end effector.

[0016] Furthermore, the planning module is specifically used for: Based on the type label of the operation object, the corresponding standard operation mode is matched in the preset micro-operation action library. The standard operation mode includes button pressing mode, knob turning mode, switch opening and closing mode, and handcart cranking in and out mode. The action constraint parameters of the corresponding mode are extracted, including stroke threshold, force threshold, speed threshold, and action rhythm. Combining the pose parameters of the object being operated on with the real-time state parameters of the robotic arm, and taking collision-free operation and shortest path as constraints, a global collision-free motion path for the robotic arm from its current pose to the operation preparation position is planned and generated based on the improved A* algorithm. Based on the action constraint parameters of the matched standard operation mode, and combined with the pose parameters of the operation object, the multi-segment continuous micro-operation action sequence of the end effector is generated, and the position control parameters, force control parameters and timing constraints corresponding to each action segment are marked.

[0017] Furthermore, the control module is specifically used for: During the movement of the robotic arm along the global motion path, local images of the end effector are acquired at a preset frequency, and the relative pose deviation between the manipulated object and the end effector is calculated in real time. Based on the relative pose deviation, the motion path of the robotic arm is corrected online in real time, driving the robotic arm to accurately reach the operation preparation position. During the execution of micro-operation actions, the corresponding force-position hybrid control mode is switched according to the timing constraints of the micro-operation action sequence. Position closed-loop control is performed on the translational degrees of freedom, and force closed-loop control is performed on the degrees of freedom in contact with the operated object. The system collects six-dimensional force / torque data and visual pose feedback data at the end effector in real time, compares them with the preset parameters of the micro-operation action sequence, generates position correction and force correction amounts, and superimposes them onto the current control loop to adjust the pose and operating force of the end effector in real time, thereby completing the execution of the corresponding micro-operation action.

[0018] Furthermore, the control module is specifically used for: The degrees of freedom of each action in the micro-operation action sequence are decoupled to distinguish between non-contact spatial movement degrees of freedom and contact operation interaction degrees of freedom. For the spatial translation degree of freedom, a position control loop is set up, and a PID control algorithm is adopted to perform high-precision position closed-loop control with preset pose parameters as target values. For the aforementioned operational interaction degrees of freedom, a force control loop is set up, and an impedance control algorithm is adopted. With a preset force threshold as a constraint, adaptive force closed-loop control is executed, and the target value of the position loop is adjusted in real time according to the change of contact force.

[0019] Furthermore, the verification module is specifically used for: After the micro-operation action is completed, the binocular camera is used to collect global images and local close-up images of the scene after the operation, and the current state features of the operation object are extracted, including button pressing status, knob turning angle, switch opening and closing position, and handcart cranking stroke. The current state characteristics of the operation object are compared with the preset target operation state, and the state deviation value is calculated. When the state deviation value is within the preset allowable range, the operation is judged to be qualified; when the state deviation value exceeds the preset allowable range, the operation is judged to be unqualified. For tasks that pass the operation, the information of the operation object, operation process parameters, and verification results are integrated to generate a standardized task completion report; for tasks that fail the operation, corresponding supplementary operation instructions are generated based on the state deviation value to update the motion path and micro-operation sequence of the robotic arm.

[0020] Furthermore, the verification module is specifically used for: Analyze the reasons for operational failures, distinguish between abnormal types such as positional deviation, insufficient operating force, and insufficient stroke, and match the corresponding supplementary operation mode. Combining the current pose, state deviation value and supplementary operation mode of the operation object, the supplementary operation preparation position and supplementary operation action parameters of the robotic arm are recalculated to generate a supplementary operation micro-operation action sequence; The supplementary micro-operation sequence is sent to the control unit of the robotic arm, and the vision-guided closed-loop control and operation result verification process is repeated until the operation result is qualified or the preset maximum number of retries is reached.

[0021] The third aspect of the present invention proposes: A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the vision-guided robotic arm micro-manipulation control method as described above.

[0022] The fourth aspect of the present invention proposes: A readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the vision-guided robotic arm micro-manipulation control method as described above. Attached Figure Description

[0023] Figure 1 A flowchart of the vision-guided robotic arm micro-manipulation control method provided in the first embodiment of the present invention; Figure 2 This is a structural block diagram of the vision-guided robotic arm micro-operation control system provided in the third embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0025] Please see Figure 1 The first embodiment of this invention provides a vision-guided robotic arm micro-operation control method. This method, through image analysis of the power distribution room work scene and precise identification of the operated object, provides accurate pose and type benchmarks for micro-operation operations, solving the problem of insufficient object identification and positioning accuracy under complex working conditions. By matching corresponding micro-operation modes based on operation type, it achieves universal adaptation for various types of power distribution room maintenance operations, breaking through the limitations of single operation modes in existing technologies. Through force-position hybrid closed-loop control that integrates real-time visual feedback and end-effector force sensing, it can correct motion deviations and operating force parameters in real time, effectively offsetting visual deviations, installation errors, and operational disturbances, significantly improving operational accuracy and stability. Through operation result verification and supplementary operation closed-loop mechanisms, it ensures a high success rate, continuously meeting operational requirements under complex working conditions, and achieving unmanned, highly reliable, and high-precision execution of the entire power distribution room micro-operation process.

[0026] Specifically, the first embodiment of the present invention provides: A vision-guided robotic arm micro-manipulation control method, wherein the method includes: Step S10: Analyze and process the real-time acquired images of the power distribution room operation scene, identify the operation object and feature points, generate the pose parameters and type labels of the operation object, and simultaneously acquire the real-time status parameters of the robotic arm body and the end effector. It's important to note that this step is the sensory input stage of the entire method, forming the foundation for all subsequent planning, control, and verification actions. This step, through the analysis and processing of images of the power distribution room work scene, completes the identification, feature point extraction, and pose calculation of the manipulated object, solving the core fundamental questions of "what the manipulated object is, where it is, and in what posture it exists." This provides a precise target benchmark for subsequent matching of operation modes and planning of work paths. Simultaneously, real-time status parameters of the robotic arm and end effector are collected, comprehensively understanding the current working condition of the actuator. This provides a body state benchmark for subsequent path planning and closed-loop control, avoiding planning failures and control inaccuracies caused by deviations in the body state.

[0027] Step S20: Based on the type label and pose parameters of the operation object, match the corresponding micro-operation mode, and generate the global motion path and end-effector micro-operation sequence of the robotic arm according to the real-time status parameters of the robotic arm body and the end-effector. It's important to note that this step, following the front-end perception results, outputs action commands that the back-end control system can directly execute. This addresses the core questions of "what type of operation to perform, how to move safely, and what specific actions to execute." By matching the type label and pose parameters of the operation object to the corresponding micro-operation mode, it achieves universal adaptation for various operation tasks in the power distribution room, overcoming the limitations of existing single-operation modes. Generating the global motion path and the end-effector micro-operation sequence breaks down the macroscopic task into specific, phased action commands that the robotic arm can execute. This ensures both the safety and efficiency of the robotic arm's movement and clarifies the specific control objectives of the end-effector micro-operations, providing a clear execution basis for subsequent closed-loop control.

[0028] Step S30: Based on real-time visual feedback and end force sensing data, perform force-position hybrid closed-loop control on the movement and micro-operation actions of the robotic arm, correct motion deviation and operation force parameters in real time, and drive the robotic arm body and the end gripper to perform corresponding operation actions. It should be noted that this step addresses the core issues of "how to accurately execute actions, how to offset on-site deviations, and how to ensure safe and stable operation." It constructs a force-position hybrid closed-loop control based on real-time visual feedback and end-effector force sensing data. This integrates the posture deviation correction capability of vision with the contact force compliance control capability of force sensing, ensuring both the positional accuracy of the end-effector operation and preventing damage to equipment from excessive force or operational failure from insufficient force. By real-time correction of motion deviations and operating force parameters, it can dynamically offset various interferences under complex working conditions such as cabinet installation deviations, visual positioning errors, cumulative errors of the robotic arm, and on-site operational disturbances, fundamentally improving operational accuracy and stability.

[0029] Step S40: After visually acquiring the scene image of the operation, verify the operation result, and generate an operation completion report or supplementary operation instructions based on the verification result.

[0030] It is important to note that this step is the closed-loop verification and result output stage of the entire method. It solves the problems of "whether the operation is qualified, how to handle unqualified operations, and how to retain the operation results," forming a complete operation closed loop. This is the key to ensuring a high success rate of the operation. The operation results are verified by visually acquiring scene images after the operation, using objective data to verify the operation effect and avoiding the subjectivity and error of human judgment. Based on the verification results, an operation completion report or supplementary operation instructions are generated. This not only achieves standardized data retention for qualified operations, but also initiates a supplementary operation process for unqualified operations, forming a complete closed loop of "execution-verification-supplementary execution." This process eliminates the problems of missed operations and operation failures, meeting the core requirements of unmanned operation in the power distribution room.

[0031] Second Embodiment Furthermore, the steps of analyzing and processing the real-time acquired images of the power distribution room operation scene, identifying the operation object and feature points, generating the pose parameters and type labels of the operation object, and simultaneously acquiring the real-time status parameters of the robotic arm body and the end effector include: By deploying a binocular camera and a global camera at the end of the robotic arm, global images and close-up images of the end of the work scene are acquired simultaneously. The global images and close-up images of the end of the work scene are then preprocessed to remove distortion, denoise, and enhance them to generate a standardized set of work images. The standardized operation image set is identified by a pre-trained target detection network, and the operation object types such as buttons, knobs, drawer switches, frame switches and handcart mechanisms are distinguished and corresponding type labels are generated. At the same time, the feature points of each operation object are extracted, and the three-dimensional spatial coordinates and attitude angles of each feature point are calculated by a stereo vision algorithm to generate the pose parameters of the operation object. The robot arm's joint encoder, force sensor, and attitude sensor collect real-time angle and angular velocity parameters of each joint, clamping force and six-dimensional force / torque parameters of the end effector, and real-time pose parameters of the robot arm body, generating real-time status parameters of the robot arm and end effector.

[0032] It should be noted that this embodiment clarifies the specific implementation path of high-precision environmental perception and body state acquisition, forming a complete perception logic of "image acquisition and preprocessing - target recognition and pose calculation - body state acquisition", which is a necessary support for achieving 1mm-level positioning accuracy.

[0033] Specifically, by simultaneously acquiring global images and end-effector close-up images using a binocular camera and a global camera deployed at the end of the robotic arm, the system achieves both global coverage of the work scene and detailed capture of the manipulated object. The global camera ensures overall environmental perception of the work scene, while the binocular camera at the end ensures millimeter-level detail capture of the manipulated object. The combination of the two solves the problem of the limited field of view of a single camera and its inability to cover both global and local details. Image preprocessing, including distortion correction, noise reduction, and enhancement, is performed to eliminate environmental interference such as uneven lighting, cabinet reflections, and dust in the power distribution room, correct inherent lens distortion, enhance the feature contrast of the manipulated object, and generate a standardized set of work images. This provides high-quality image input for subsequent target recognition, avoiding recognition errors and pose calculation deviations caused by poor image quality from the source.

[0034] Furthermore, a pre-trained object detection network was used to identify the operation objects in a standardized set of operation images. Utilizing the high-precision feature extraction capabilities of deep learning, five core operation objects—electric control room buttons, knobs, drawer switches, frame switches, and handcart mechanisms—were accurately distinguished, generating corresponding type labels. This provided an accurate type basis for subsequent matching of micro-operation modes. Simultaneously, the operation feature points of each operation object were extracted, and the three-dimensional spatial coordinates and attitude angles of each feature point were calculated using a stereo vision algorithm to generate the pose parameters of the operation object. This solved the problem that two-dimensional images could not provide depth information and could not meet the requirements of high-precision positioning. Accurate three-dimensional pose parameters are the core prerequisite for subsequent robotic arm path planning and precise end-effector operation, directly determining the final operation accuracy.

[0035] By using the joint encoders, force sensors, and attitude sensors of the robotic arm, the angle and angular velocity parameters of each joint of the robotic arm, the clamping force and six-dimensional force / torque parameters of the end effector, and the real-time pose parameters of the robotic arm body are collected in real time. This generates real-time state parameters of the robotic arm and the end effector, comprehensively covering the motion state, force state, and pose state of the actuator. Among them, the joint parameters provide the motion reference of the robotic arm for kinematic calculation and path planning, the force sensor parameters provide real-time feedback input for subsequent force closed-loop control, and the body pose parameters provide the current position reference for global path planning. These real-time state parameters are indispensable inputs for subsequent planning and control. Without accurate acquisition of the body state, accurate path planning and closed-loop control cannot be achieved.

[0036] Furthermore, the step of matching the corresponding micro-operation mode based on the type label and pose parameters of the operation object, and generating the global motion path and end-effector micro-operation sequence of the robotic arm according to the real-time state parameters of the robotic arm body and the end effector includes: Based on the type label of the operation object, the corresponding standard operation mode is matched in the preset micro-operation action library. The standard operation mode includes button pressing mode, knob turning mode, switch opening and closing mode, and handcart cranking in and out mode. The action constraint parameters of the corresponding mode are extracted, including stroke threshold, force threshold, speed threshold, and action rhythm. Combining the pose parameters of the object being operated on with the real-time state parameters of the robotic arm, and taking collision-free operation and shortest path as constraints, a global collision-free motion path for the robotic arm from its current pose to the operation preparation position is planned and generated based on the improved A* algorithm. Based on the action constraint parameters of the matched standard operation mode, and combined with the pose parameters of the operation object, the multi-segment continuous micro-operation action sequence of the end effector is generated, and the position control parameters, force control parameters and timing constraints corresponding to each action segment are marked.

[0037] It should be noted that, based on the type label of the operation object, the corresponding standard operation mode is matched in the preset micro-operation action library. The preset micro-operation action library is customized for the operation characteristics of the five major categories of operation objects in the power distribution room, and has four standard operation modes: button pressing mode, knob turning mode, switch opening and closing mode, and handcart cranking in and out mode. Each mode is adapted to the operation logic and safety requirements of the corresponding operation object. The action constraint parameters of the corresponding mode are extracted, including travel threshold, force threshold, speed threshold, and action cycle. These parameters are the core boundaries to ensure safe and effective operation. For example, the travel threshold of button pressing avoids excessive pressing and damage to the button, and the force threshold of handcart operation avoids mechanism jamming and damage. By matching the standard operation mode and constraint parameters, the universal adaptation of multiple types of power distribution room operation tasks is achieved, breaking through the limitation of existing technologies that can only adapt to a single operation mode.

[0038] By combining the pose parameters of the manipulated object with the real-time status parameters of the robotic arm, and with the constraints of collision-free operation and shortest path, an improved A* algorithm is used to plan and generate a global collision-free motion path for the robotic arm from its current pose to the operation preparation position. The collision-free constraint avoids the safety risk of collisions between the robotic arm and the cabinet or surrounding equipment, while the shortest path constraint ensures the movement efficiency of the robotic arm. The improved A* algorithm takes into account both the global optimality of path planning and the real-time performance on site, solving the safety and efficiency problems of the global movement of the robotic arm in the complex cabinet environment of the power distribution room, and providing a precise and stable starting point for subsequent micro-operations.

[0039] Based on the motion constraint parameters of the matching standard operating mode, combined with the pose parameters of the operated object, a multi-segment continuous micro-operation motion sequence of the end effector is generated. Each motion segment is marked with corresponding position control parameters, force control parameters, and timing constraints. This breaks down the macroscopic work task into staged micro-operation instructions that the end effector can directly execute. It is adapted to the step-by-step operation characteristics of micro-operations in the power distribution room. For example, turning a knob requires the sequential execution of multiple actions such as end-effector engagement, jaw locking, fixed-angle turning, jaw release, and end-effector reset. Marking each motion segment with corresponding control parameters and timing constraints clarifies the control objectives, boundary conditions, and execution sequence of each step. This provides clear and staged control objectives for subsequent closed-loop control links, ensuring the accurate and orderly execution of micro-operation actions.

[0040] Furthermore, the step of performing force-position hybrid closed-loop control on the movement and micro-manipulation actions of the robotic arm based on real-time visual feedback and end-effector force sensing data, correcting motion deviations and operating force parameters in real time, and driving the robotic arm body and the end-effector to perform corresponding work actions includes: During the movement of the robotic arm along the global motion path, local images of the end effector are acquired at a preset frequency, and the relative pose deviation between the manipulated object and the end effector is calculated in real time. Based on the relative pose deviation, the motion path of the robotic arm is corrected online in real time, driving the robotic arm to accurately reach the operation preparation position. During the execution of micro-operation actions, the corresponding force-position hybrid control mode is switched according to the timing constraints of the micro-operation action sequence. Position closed-loop control is performed on the translational degrees of freedom, and force closed-loop control is performed on the degrees of freedom in contact with the operated object. The system collects six-dimensional force / torque data and visual pose feedback data at the end effector in real time, compares them with the preset parameters of the micro-operation action sequence, generates position correction and force correction amounts, and superimposes them onto the current control loop to adjust the pose and operating force of the end effector in real time, thereby completing the execution of the corresponding micro-operation action.

[0041] It should be noted that during the movement of the robotic arm along the global motion path, local images of the end effector are acquired at a preset frequency, and the relative pose deviation between the manipulated object and the end effector is calculated in real time. Based on the deviation value, the movement path of the robotic arm is corrected online in real time, driving the robotic arm to accurately reach the operation preparation position. This introduces visual closed-loop control throughout the entire movement process of the robotic arm, correcting the cumulative error, cabinet installation deviation, and robotic arm movement deviation in real time. This avoids the positioning inaccuracy problem caused by traditional open-loop movement, ensuring that the robotic arm can accurately reach the operation preparation position. This provides a millimeter-level starting point for subsequent micro-operations and is a key link in achieving 1mm-level positioning accuracy.

[0042] During the execution of micro-operation actions, the corresponding force-position hybrid control mode is switched according to the timing constraints of the micro-operation action sequence. Position closed-loop control is performed on the translational degree of freedom, and force closed-loop control is performed on the degree of freedom in contact with the operated object. This is designed for the operational characteristics of micro-operations in the power distribution room, and decouples the multiple degrees of freedom of the robotic arm. The core requirement for the non-contact translational degree of freedom is positional accuracy, so position closed-loop control is adopted. The core requirement for the operational interaction degree of freedom in contact with the operated object is the controllability of force to avoid damage to equipment or operational failure due to hard contact, so force closed-loop control is adopted. By switching the control mode of the degree of freedom, both positional accuracy and safe and controllable operating force are taken into account, solving the core pain point that the traditional single control mode cannot adapt to the contact and non-contact operation stages.

[0043] Real-time acquisition of six-dimensional force / torque data and visual pose feedback data at the end effector, compared with preset parameters of the micro-operation sequence, generates position correction and force correction values, which are superimposed on the current control loop. The pose and operating force of the end effector are adjusted in real time to complete the execution of the corresponding micro-operation. In the entire process of micro-operation execution, visual pose feedback and force sensing feedback are introduced simultaneously to form a dual closed-loop control system. Visual feedback corrects pose deviations in real time, and force sensing feedback adjusts the operating force in real time. The combination of the two can dynamically offset various disturbances such as minor deformation of the operated object, mechanism jamming, and robotic arm vibration during the operation process, and adjust control parameters in real time to ensure the accurate and stable execution of micro-operation actions, fundamentally improving the operation accuracy and reliability under complex working conditions.

[0044] Furthermore, the steps of switching the corresponding force-position hybrid control mode according to the timing constraints of the micro-operation action sequence, performing position closed-loop control on the translational degrees of freedom, and performing force closed-loop control on the degrees of freedom in contact with the manipulated object include: The degrees of freedom of each action in the micro-operation action sequence are decoupled to distinguish between non-contact spatial movement degrees of freedom and contact operation interaction degrees of freedom. For the spatial translation degree of freedom, a position control loop is set up, and a PID control algorithm is adopted to perform high-precision position closed-loop control with preset pose parameters as target values. For the aforementioned operational interaction degrees of freedom, a force control loop is set up, and an impedance control algorithm is adopted. With a preset force threshold as a constraint, adaptive force closed-loop control is executed, and the target value of the position loop is adjusted in real time according to the change of contact force.

[0045] It should be noted that decoupling the degrees of freedom of each segment in the micro-operation sequence, and distinguishing between non-contact spatial movement degrees of freedom and contact operation interaction degrees of freedom, is a prerequisite for achieving precise force-position hybrid control. In the six degrees of freedom motion of the robotic arm, the control objectives of different degrees of freedom are completely different in different operation stages. Only by first completing the precise decoupling of the degrees of freedom and clarifying the control priority and control mode of each degree of freedom can targeted sub-mode control be achieved, avoiding control logic confusion and ensuring the accuracy and stability of control.

[0046] For the spatial movement degree of freedom, a position control loop is set up, and a PID control algorithm is adopted. With the preset pose parameters as the target value, high-precision position closed-loop control is executed. The PID control algorithm has the characteristics of strong robustness, simple parameter tuning, and high steady-state accuracy. It is highly adaptable to the position control requirements of the robotic arm's spatial movement. Through position closed-loop control, pose deviations can be eliminated quickly and accurately, ensuring the positioning accuracy of non-contact spatial movement degree of freedom, meeting the 1mm-level positioning requirements of power distribution room operations, and providing a precise position reference for end-effector micro-operations.

[0047] For the degree of freedom of operation interaction, a force control loop is set up, and an impedance control algorithm is adopted. With a preset force threshold as a constraint, adaptive force closed-loop control is executed. Simultaneously, the target value of the position loop is adjusted in real time according to changes in contact force. The impedance control algorithm can achieve compliant control of force and position, highly adaptable to operation scenarios involving contact with the object. It ensures that the contact force does not exceed the safety threshold to avoid damage to the cabinet equipment, and adaptively adjusts the position according to changes in contact force to compensate for minor deformations and installation deviations of the object, ensuring the fit and effectiveness of the operation. Through the coordinated adjustment of the force closed loop and the position loop, compliant, precise, and safe control of contact operations is achieved, solving the problems of easy equipment damage and operational failure caused by traditional hard contact control. Furthermore, the step of visually acquiring scene images after the operation, verifying the operation results, and generating a job completion report or supplementary operation instructions based on the verification results includes: After the micro-operation action is completed, the binocular camera is used to collect global images and local close-up images of the scene after the operation, and the current state features of the operation object are extracted, including button pressing status, knob turning angle, switch opening and closing position, and handcart cranking stroke. The current state characteristics of the operation object are compared with the preset target operation state, and the state deviation value is calculated. When the state deviation value is within the preset allowable range, the operation is judged to be qualified; when the state deviation value exceeds the preset allowable range, the operation is judged to be unqualified. For tasks that pass the operation, the information of the operation object, operation process parameters, and verification results are integrated to generate a standardized task completion report; for tasks that fail the operation, corresponding supplementary operation instructions are generated based on the state deviation value to update the motion path and micro-operation sequence of the robotic arm.

[0048] It should be noted that after the micro-operation is completed, a binocular camera is used to collect global and close-up images of the post-operation scene to extract the current state features of the operated object, including button press status, knob turning angle, switch open / close position, and handcart cranking stroke. This is a visual acquisition of objective and quantifiable post-operation state data, which accurately extracts core state features that can directly characterize the operation effect. These features directly reflect whether the operation has achieved the preset operation target, providing an objective and accurate basis for subsequent qualification judgment and avoiding the subjectivity and error of manual judgment.

[0049] The current state characteristics of the operation object are compared with the preset target operation state, and the state deviation value is calculated. When the state deviation value is within the preset allowable range, the operation is judged to be qualified. When the state deviation value exceeds the preset allowable range, the operation is judged to be unqualified. Through quantitative deviation calculation, the standardization and objectivity of the operation effect are achieved. The preset allowable range strictly matches the accuracy requirements of power distribution room operation and maintenance. Through quantitative judgment standards, the subjective factors of qualification judgment are eliminated, ensuring the accuracy and consistency of the judgment results.

[0050] For successfully completed tasks, the system integrates information on the target object, operational parameters, and verification results to generate a standardized task completion report. This ensures standardized data retention throughout the entire task process, meeting the requirements for ledger management and full-process traceability in power distribution room maintenance. For unsuccessful tasks, the system generates corresponding supplementary operation instructions based on the status deviation value, updates the robotic arm's motion path and micro-operation sequence, and initiates targeted supplementary operation procedures for unsuccessful task scenarios. This forms a closed-loop management system, preventing task failure due to single-operation failures and ensuring a task success rate of over 95%. This meets the core requirement of fully unmanned operation requirements in power distribution rooms. Furthermore, the step of generating corresponding supplementary operation instructions based on the state deviation value for unqualified operation tasks and updating the motion path and micro-operation sequence of the robotic arm includes: Analyze the reasons for operational failures, distinguish between abnormal types such as positional deviation, insufficient operating force, and insufficient stroke, and match the corresponding supplementary operation mode. Combining the current pose, state deviation value and supplementary operation mode of the operation object, the supplementary operation preparation position and supplementary operation action parameters of the robotic arm are recalculated to generate a supplementary operation micro-operation action sequence; The supplementary micro-operation sequence is sent to the control unit of the robotic arm, and the vision-guided closed-loop control and operation result verification process is repeated until the operation result is qualified or the preset maximum number of retries is reached.

[0051] It should be noted that analyzing the reasons for operational failures and distinguishing between abnormal types such as posture deviation, insufficient operating force, and insufficient stroke, and matching the corresponding supplementary operation mode, is the first step to accurately locate the root cause of the operation failure, and then to match the corresponding supplementary operation strategy. This avoids blindly retrying the action. For example, if the operation failure is caused by posture deviation, the working posture of the robotic arm should be corrected first. If the failure is caused by insufficient operating force, the force control threshold and working stroke should be adjusted. The targeted supplementary operation mode can significantly improve the success rate of supplementary operations, while avoiding equipment damage caused by invalid retry.

[0052] By combining the current pose, state deviation value, and supplementary operation mode of the manipulated object, the supplementary operation preparation position and supplementary operation motion parameters of the robotic arm are recalculated to generate a supplementary operation micro-operation motion sequence. This is to re-plan the specific motion instructions for the supplementary operation based on the root cause of the operation failure and the current actual working state, rather than simply repeating the original motion sequence. For example, for failure scenarios with insufficient travel, the travel threshold and end feed parameters of the supplementary operation are adjusted accordingly. For failure scenarios with pose deviation, the operation preparation position and end pose parameters are corrected. This solves the problem of the previous operation failure from the root, ensuring the relevance and effectiveness of the supplementary operation.

[0053] The supplementary operation micro-manipulation sequence is sent to the control unit of the robotic arm, repeatedly executing the vision-guided closed-loop control and operation result verification process until the operation result is qualified or the preset maximum number of retries is reached. This integrates the supplementary operation process into the original high-precision vision closed-loop control and result verification system, ensuring the execution accuracy and effectiveness of the supplementary operation. At the same time, setting a maximum number of retries avoids equipment damage and safety risks caused by infinite retries. This ensures both a high success rate and safety of the operation, forming a complete supplementary operation closed loop, ultimately achieving unmanned, highly reliable, and high-precision execution of the entire micro-operation process in the power distribution room. Please see Figure 2 The third embodiment of the present invention provides: A vision-guided robotic arm micro-manipulation control system, wherein the system comprises: The acquisition module is used to analyze and process real-time acquired images of the power distribution room operation scene, identify the operation object and feature points, generate the pose parameters and type labels of the operation object, and simultaneously acquire the real-time status parameters of the robotic arm body and the end effector. The planning module is used to match the corresponding micro-operation mode based on the type label and pose parameters of the operation object, and generate the global motion path and end-effector micro-operation sequence of the robotic arm according to the real-time status parameters of the robotic arm body and the end-effector. The control module is used to perform force-position hybrid closed-loop control on the movement and micro-operation of the robotic arm based on real-time visual feedback and end-effector force sensing data, correct motion deviation and operation force parameters in real time, and drive the robotic arm body and the end-effector to perform corresponding operation actions. The verification module is used to verify the operation results by visually acquiring scene images after the operation, and to generate an operation completion report or supplementary operation instructions based on the verification results.

[0054] Furthermore, the acquisition module is specifically used for: By deploying a binocular camera and a global camera at the end of the robotic arm, global images and close-up images of the end of the work scene are acquired simultaneously. The global images and close-up images of the end of the work scene are then preprocessed to remove distortion, denoise, and enhance them to generate a standardized set of work images. The standardized operation image set is identified by a pre-trained target detection network, and the operation object types such as buttons, knobs, drawer switches, frame switches and handcart mechanisms are distinguished and corresponding type labels are generated. At the same time, the feature points of each operation object are extracted, and the three-dimensional spatial coordinates and attitude angles of each feature point are calculated by a stereo vision algorithm to generate the pose parameters of the operation object. The robot arm's joint encoder, force sensor, and attitude sensor collect real-time angle and angular velocity parameters of each joint, clamping force and six-dimensional force / torque parameters of the end effector, and real-time pose parameters of the robot arm body, generating real-time status parameters of the robot arm and end effector.

[0055] Furthermore, the planning module is specifically used for: Based on the type label of the operation object, the corresponding standard operation mode is matched in the preset micro-operation action library. The standard operation mode includes button pressing mode, knob turning mode, switch opening and closing mode, and handcart cranking in and out mode. The action constraint parameters of the corresponding mode are extracted, including stroke threshold, force threshold, speed threshold, and action rhythm. Combining the pose parameters of the object being operated on with the real-time state parameters of the robotic arm, and taking collision-free operation and shortest path as constraints, a global collision-free motion path for the robotic arm from its current pose to the operation preparation position is planned and generated based on the improved A* algorithm. Based on the action constraint parameters of the matched standard operation mode, and combined with the pose parameters of the operation object, the multi-segment continuous micro-operation action sequence of the end effector is generated, and the position control parameters, force control parameters and timing constraints corresponding to each action segment are marked.

[0056] Furthermore, the control module is specifically used for: During the movement of the robotic arm along the global motion path, local images of the end effector are acquired at a preset frequency, and the relative pose deviation between the manipulated object and the end effector is calculated in real time. Based on the relative pose deviation, the motion path of the robotic arm is corrected online in real time, driving the robotic arm to accurately reach the operation preparation position. During the execution of micro-operation actions, the corresponding force-position hybrid control mode is switched according to the timing constraints of the micro-operation action sequence. Position closed-loop control is performed on the translational degrees of freedom, and force closed-loop control is performed on the degrees of freedom in contact with the operated object. The system collects six-dimensional force / torque data and visual pose feedback data at the end effector in real time, compares them with the preset parameters of the micro-operation action sequence, generates position correction and force correction amounts, and superimposes them onto the current control loop to adjust the pose and operating force of the end effector in real time, thereby completing the execution of the corresponding micro-operation action.

[0057] Furthermore, the control module is specifically used for: The degrees of freedom of each action in the micro-operation action sequence are decoupled to distinguish between non-contact spatial movement degrees of freedom and contact operation interaction degrees of freedom. For the spatial translation degree of freedom, a position control loop is set up, and a PID control algorithm is adopted to perform high-precision position closed-loop control with preset pose parameters as target values. For the aforementioned operational interaction degrees of freedom, a force control loop is set up, and an impedance control algorithm is adopted. With a preset force threshold as a constraint, adaptive force closed-loop control is executed, and the target value of the position loop is adjusted in real time according to the change of contact force.

[0058] Furthermore, the verification module is specifically used for: After the micro-operation action is completed, the binocular camera is used to collect global images and local close-up images of the scene after the operation, and the current state features of the operation object are extracted, including button pressing status, knob turning angle, switch opening and closing position, and handcart cranking stroke. The current state characteristics of the operation object are compared with the preset target operation state, and the state deviation value is calculated. When the state deviation value is within the preset allowable range, the operation is judged to be qualified; when the state deviation value exceeds the preset allowable range, the operation is judged to be unqualified. For tasks that pass the operation, the information of the operation object, operation process parameters, and verification results are integrated to generate a standardized task completion report; for tasks that fail the operation, corresponding supplementary operation instructions are generated based on the state deviation value to update the motion path and micro-operation sequence of the robotic arm.

[0059] Furthermore, the verification module is specifically used for: Analyze the reasons for operational failures, distinguish between abnormal types such as positional deviation, insufficient operating force, and insufficient stroke, and match the corresponding supplementary operation mode. Combining the current pose, state deviation value and supplementary operation mode of the operation object, the supplementary operation preparation position and supplementary operation action parameters of the robotic arm are recalculated to generate a supplementary operation micro-operation action sequence; The supplementary micro-operation sequence is sent to the control unit of the robotic arm, and the vision-guided closed-loop control and operation result verification process is repeated until the operation result is qualified or the preset maximum number of retries is reached.

[0060] The fourth embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the vision-guided robotic arm micro-operation control method as described above.

[0061] The fifth embodiment of the present invention provides a readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the vision-guided robotic arm micro-manipulation control method as described above.

[0062] In summary, the vision-guided robotic arm micro-manipulation control method and system provided by this invention can achieve accurate identification of various types of operational objects in the power distribution room and 1mm-level three-dimensional pose calculation through a visual perception system that combines binocular global and local close-up cameras with deep learning target recognition and stereo vision pose calculation, providing a reliable perception benchmark for high-precision operations; and through preset multi-type micro-manipulation mode matching and improvement A The algorithm's global path planning enables universal adaptation to various operation scenarios, including button pressing, knob turning, switch opening and closing, and handcart entry and exit, breaking through the limitations of existing single operation modes. Through a force-position hybrid closed-loop control that integrates real-time visual pose feedback and end-effector force sensing data, it achieves high-precision position control for non-contact degrees of freedom and compliant force control for contact operation degrees of freedom. This can offset interference from complex working conditions such as cabinet installation deviations, cumulative errors of the robotic arm, and on-site operational disturbances in real time, fundamentally improving operational accuracy and stability. Quantitative verification of operation results and a targeted supplementary operation closed-loop mechanism ensure a stable operation success rate of over 95%, ultimately achieving fully unmanned, highly reliable, and high-precision execution of micro-operations in the power distribution room, providing core technical support for the unmanned transformation of intelligent operation and maintenance in power distribution rooms.

[0063] Although the present invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the disclosed embodiments can be combined with each other in any manner. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A micro-manipulation control method for a vision-guided robotic arm, characterized in that, The method includes: The real-time acquired images of the power distribution room operation scene are analyzed and processed to identify the operation object and feature points, generate the pose parameters and type labels of the operation object, and simultaneously acquire the real-time status parameters of the robotic arm body and the end effector. Based on the type label and pose parameters of the operation object, the corresponding micro-operation mode is matched, and the global motion path and end-effector micro-operation action sequence of the robotic arm are generated according to the real-time status parameters of the robotic arm body and the end-effector. Based on real-time visual feedback and end-effector force sensing data, the motion and micro-operation actions of the robotic arm are controlled by a force-position hybrid closed-loop system. The motion deviation and operation force parameters are corrected in real time, and the robotic arm body and the end-effector are driven to perform corresponding operation actions. After visually acquiring scene images of the operation, the operation results are verified, and an operation completion report or supplementary operation instructions are generated based on the verification results.

2. The micro-manipulation control method for a vision-guided robotic arm according to claim 1, characterized in that, The steps of analyzing and processing the real-time acquired images of the power distribution room operation scene, identifying the operation object and feature points, generating the pose parameters and type labels of the operation object, and simultaneously acquiring the real-time status parameters of the robotic arm body and the end effector include: By deploying a binocular camera and a global camera at the end of the robotic arm, global images and close-up images of the end of the work scene are acquired simultaneously. The global images and close-up images of the end of the work scene are then preprocessed to remove distortion, denoise, and enhance them to generate a standardized set of work images. The standardized operation image set is identified by a pre-trained target detection network, and the operation object types such as buttons, knobs, drawer switches, frame switches and handcart mechanisms are distinguished and corresponding type labels are generated. At the same time, the feature points of each operation object are extracted, and the three-dimensional spatial coordinates and attitude angles of each feature point are calculated by a stereo vision algorithm to generate the pose parameters of the operation object. The robot arm's joint encoder, force sensor, and attitude sensor collect real-time angle and angular velocity parameters of each joint, clamping force and six-dimensional force / torque parameters of the end effector, and real-time pose parameters of the robot arm body, generating real-time status parameters of the robot arm and end effector.

3. The micro-manipulation control method for a vision-guided robotic arm according to claim 1, characterized in that, The steps of matching the corresponding micro-operation mode based on the type label and pose parameters of the operation object, and generating the global motion path and end-effector micro-operation sequence of the robotic arm according to the real-time state parameters of the robotic arm body and the end effector include: Based on the type label of the operation object, the corresponding standard operation mode is matched in the preset micro-operation action library. The standard operation mode includes button pressing mode, knob turning mode, switch opening and closing mode, and handcart cranking in and out mode. The action constraint parameters of the corresponding mode are extracted, including stroke threshold, force threshold, speed threshold, and action rhythm. Combining the pose parameters of the object being operated on with the real-time state parameters of the robotic arm, and taking collision-free operation and shortest path as constraints, a global collision-free motion path for the robotic arm from its current pose to the operation preparation position is planned and generated based on the improved A* algorithm. Based on the action constraint parameters of the matched standard operation mode, and combined with the pose parameters of the operation object, the multi-segment continuous micro-operation action sequence of the end effector is generated, and the position control parameters, force control parameters and timing constraints corresponding to each action segment are marked.

4. The micro-manipulation control method for a vision-guided robotic arm according to claim 1, characterized in that, The steps of performing force-position hybrid closed-loop control on the motion and micro-manipulation actions of the robotic arm based on real-time visual feedback and end-effector force sensing data, correcting motion deviations and operating force parameters in real time, and driving the robotic arm body and the end-effector to perform corresponding work actions include: During the movement of the robotic arm along the global motion path, local images of the end effector are acquired at a preset frequency, and the relative pose deviation between the manipulated object and the end effector is calculated in real time. Based on the relative pose deviation, the motion path of the robotic arm is corrected online in real time, driving the robotic arm to accurately reach the operation preparation position. During the execution of micro-operation actions, the corresponding force-position hybrid control mode is switched according to the timing constraints of the micro-operation action sequence. Position closed-loop control is performed on the translational degrees of freedom, and force closed-loop control is performed on the degrees of freedom in contact with the operated object. The system collects six-dimensional force / torque data and visual pose feedback data at the end effector in real time, compares them with the preset parameters of the micro-operation action sequence, generates position correction and force correction amounts, and superimposes them onto the current control loop to adjust the pose and operating force of the end effector in real time, thereby completing the execution of the corresponding micro-operation action.

5. The vision-guided robotic arm micro-manipulation control method according to claim 4, characterized in that, The steps of switching the corresponding force-position hybrid control mode according to the timing constraints of the micro-operation action sequence, performing position closed-loop control on the translational degrees of freedom, and performing force closed-loop control on the degrees of freedom in contact with the manipulated object include: The degrees of freedom of each action in the micro-operation action sequence are decoupled to distinguish between non-contact spatial movement degrees of freedom and contact operation interaction degrees of freedom. For the spatial translation degree of freedom, a position control loop is set up, and a PID control algorithm is adopted to perform high-precision position closed-loop control with preset pose parameters as target values. For the aforementioned operational interaction degrees of freedom, a force control loop is set up, and an impedance control algorithm is adopted. With a preset force threshold as a constraint, adaptive force closed-loop control is executed, and the target value of the position loop is adjusted in real time according to the change of contact force.

6. The micro-manipulation control method for a vision-guided robotic arm according to claim 1, characterized in that, The steps of visually acquiring scene images after the operation, verifying the operation results, and generating an operation completion report or supplementary operation instructions based on the verification results include: After the micro-operation action is completed, the binocular camera is used to collect global images and local close-up images of the scene after the operation, and the current state features of the operation object are extracted, including button pressing status, knob turning angle, switch opening and closing position, and handcart cranking stroke. The current state characteristics of the operation object are compared with the preset target operation state, and the state deviation value is calculated. When the state deviation value is within the preset allowable range, the operation is judged to be qualified; when the state deviation value exceeds the preset allowable range, the operation is judged to be unqualified. For tasks that pass the operation, the information of the operation object, operation process parameters, and verification results are integrated to generate a standardized task completion report; for tasks that fail the operation, corresponding supplementary operation instructions are generated based on the state deviation value to update the motion path and micro-operation sequence of the robotic arm.

7. The micro-manipulation control method for a vision-guided robotic arm according to claim 6, characterized in that, The step of generating corresponding supplementary operation instructions based on the state deviation value for unqualified operation tasks and updating the motion path and micro-operation sequence of the robotic arm includes: Analyze the reasons for operational failures, distinguish between abnormal types such as positional deviation, insufficient operating force, and insufficient stroke, and match the corresponding supplementary operation mode. Combining the current pose, state deviation value and supplementary operation mode of the operation object, the supplementary operation preparation position and supplementary operation action parameters of the robotic arm are recalculated to generate a supplementary operation micro-operation action sequence; The supplementary micro-operation sequence is sent to the control unit of the robotic arm, and the vision-guided closed-loop control and operation result verification process is repeated until the operation result is qualified or the preset maximum number of retries is reached.

8. A vision-guided robotic arm micro-manipulation control system, characterized in that, The system includes: The acquisition module is used to analyze and process real-time acquired images of the power distribution room operation scene, identify the operation object and feature points, generate the pose parameters and type labels of the operation object, and simultaneously acquire the real-time status parameters of the robotic arm body and the end effector. The planning module is used to match the corresponding micro-operation mode based on the type label and pose parameters of the operation object, and generate the global motion path and end-effector micro-operation sequence of the robotic arm according to the real-time status parameters of the robotic arm body and the end-effector. The control module is used to perform force-position hybrid closed-loop control on the movement and micro-operation of the robotic arm based on real-time visual feedback and end-effector force sensing data, correct motion deviation and operation force parameters in real time, and drive the robotic arm body and the end-effector to perform corresponding operation actions. The verification module is used to verify the operation results by visually acquiring scene images after the operation, and to generate an operation completion report or supplementary operation instructions based on the verification results.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the vision-guided robotic arm micro-manipulation control method as described in any one of claims 1 to 7.

10. A readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the vision-guided robotic arm micro-manipulation control method as described in any one of claims 1 to 7.

Images