162 results about "Visual servoing" patented technology

A human-guided vision-force fused impedance iterative learning control method for a robotic arm, comprising: analyzing a robot-environment interaction dynamics equation, solving a visual servo acceleration model, and making use of the equation to establish a human-robotic arm-environment interaction dynamics model in an image feature space; acquiring an image feature position and speed curve of a human-guided robot completing an assembly task, and using dynamic movement primitives for coding and generalization; and designing an impedance iterative learning controller which uses image feature tracking errors as control input, learning impedance characteristics when the human-guided robot performs a contact operation, identifying unknown contact dynamics under the interaction between the robot and the environment, and counteracting identified contact interference in the feature space, so as to implement a flexible assembly operation. The control method solves the problems in existing assembly operations that human-robotic arm-environment coupling nonlinear dynamics, unknown contact dynamics of intensive contact assembly tasks and poor generalization of assembly scenarios require relearning for different scenarios, etc.

The invention discloses a high-precision laser positioning unpacking path planning system, and belongs to the technical field of robot automatic control and industrial automation. The system comprises a data synchronization module used for multi-sensor hardware synchronization and data alignment; the high-precision positioning module is used for outputting a precise pose based on environment skeleton characteristics and sliding window optimization; the semantic map construction module is used for fusing vision and laser data to generate a dynamic semantic grid map; the global path planning module is used for planning a smooth path in the skeleton channel by utilizing a mixed potential field improved A * algorithm; the motion control module is used for realizing closed-loop motion control and safety monitoring through model prediction and tracking; and the operation execution module is used for finishing millimeter-level precise stopping of an operation point by adopting visual servo and triggering unpacking operation. According to the method, the positioning robustness under dynamic shielding is improved through the environmental skeleton features, the safety and the high efficiency of the path are ensured by utilizing semantic understanding and intelligent planning, and the full-process automation from navigation to precise operation is realized.

The invention discloses a robot automatic grabbing path planning method based on visual identification, and relates to the technical field of intelligent grabbing. The method comprises the following steps: acquiring multi-view visual data of a target scene, and generating a scene three-dimensional compact reconstruction model through a multi-modal image fusion algorithm; performing target detection and feature extraction on the model, and screening an optimal capture point in combination with a visual attention mechanism; constructing a dynamic environment obstacle probability map, updating an obstacle state through time sequence visual tracking, and quantifying an interference weight; an initial grabbing path is planned based on an improved fast expansion random tree algorithm, path smoothness constraints and robot joint movement limit parameters are introduced, and path nodes are optimized through a Bezier curve; visual servo feedback and path deviation prediction are fused, path parameters are corrected in real time, and a continuous movement track is generated. The method effectively adapts to the dynamic environment, gives consideration to path safety, smoothness and mechanical arm motion characteristics, and remarkably improves the grabbing success rate and operation reliability.

The invention discloses a mobile robot accurate docking method based on edge calculation, and aims to solve the problems that the dynamic docking precision is reduced and the collision risk is increased due to micro-motion or drifting of a target station. According to the method, unified time reference alignment is carried out on data of a camera, a laser radar, an inertial measurement unit, an ultra-wideband range finder, a station encoder and a programmable logic controller at an edge node, and a three-dimensional special Euclidean group equivariant multi-source fusion network is used for outputting relative pose estimation and covariance; the estimation in the time window is further used as a condition to be input into a conditional diffusion short-time prediction model to obtain a time-varying mean value and a time-varying covariance, an anisotropic probability tube is constructed, and prediction-measurement joint correction is carried out based on a score function; scenarized opportunity constraints are constructed under correction probability tube constraints, a tubular nonlinear model is adopted to predict, control and solve a reference trajectory, an actuator command is generated in combination with depth visual servo and compliance control, and the technical effects of high precision, robustness and safe docking under the station dynamic disturbance condition are achieved.

The invention belongs to the technical field of intelligent and robot operation, and discloses a collaborative robot closed-loop operation method based on scene perception and thinking chain reasoning, which comprises the following steps of: firstly, generating structured semantic representation containing confidence by utilizing a visual language model of LoRA efficient fine tuning and fusing multi-view data, and providing semantic understanding for a robot; secondly, utilizing a three-dimensional diffusion model to generate explicit geometric priori under the condition of no real CAD, and correspondingly completing PnP initial pose estimation in combination with 2D-3D; then, carrying out pose optimization by adopting a tracking-refining strategy, and triggering reinitialization based on geometric prior when a residual error exceeds a threshold value; and finally, decomposing a global task by utilizing a thinking chain reasoning module, and realizing'perception-reasoning-execution 'closed-loop control based on visual servo in cooperation with an execution module. The method realizes effective combination of large model reasoning and physical execution, has strong robustness under dynamic disturbance, and is suitable for intelligent manufacturing and man-machine cooperation scenes.

The invention discloses an unmanned aerial vehicle relay tracking system and method for cloud side-end cooperation in an urban scene, and belongs to the technical field of intelligent traffic and unmanned aerial vehicle cooperative control. In order to solve the problems of insufficient endurance and limited sight distance of existing single-machine tracking in an urban large-range scene, a physically distributed and logically unified cloud-side-end three-level architecture is constructed, wherein a cloud end is responsible for natural language instruction analysis and global road network monitoring; the edge end serves as a regional scheduling hub and is responsible for operating a high-precision detection model, extracting a target Re-ID feature fingerprint and calculating a cross-machine relay time window; and the terminal unmanned aerial vehicle only operates a lightweight tracking model and a visual servo control law. According to the method, the urban area is divided into a plurality of honeycomb grids, seamless locking and relay of a target among unmanned aerial vehicles in different grid responsibility areas are realized by using a cross-view feature relay protocol and a collaborative finite-state machine, and the engineering problem of long-time-sequence and large-range vehicle tracking is effectively solved.

The invention belongs to the technical field of robot control, and discloses a visual servo intelligent robust control method for a mobile mechanical arm for a complex operation task, which comprises the following steps of: 1, acquiring a working scene image, and calculating a space coordinate of a target feature point in real time through a projection transformation model; step 2, dynamically inhibiting visual measurement noise by adopting adaptive Kalman filtering, generating a feedforward compensation signal through an integral sliding mode observer, and decoupling chassis slippage disturbance; step 3, inputting the pose signal into a radial RBF neural network, designing an RBF gain scheduler, and adjusting an output proportion-integral gain in real time to suppress external time-varying disturbance; 4, adopting a hybrid visual servo mode switching mechanism, and generating a mobile platform control instruction through an integral sliding mode surface in a position servo mode; and 5, designing a three-order composite controller to drive the mechanical arm so as to realize accurate grabbing. According to the method, the grabbing deviation caused by kinematics uncertainty of the mobile platform and dynamic environment disturbance is eliminated.

The invention relates to the technical field of intelligent photography, discloses an intelligent composition auxiliary method and system for tourism photography, and aims to solve the problems that scene semantics cannot be deeply understood and dynamic active composition guidance is difficult to provide in the prior art. The method comprises the following steps: acquiring multi-modal data such as images, geographic coordinates, equipment postures and the like in real time; constructing a scene semantic map for describing visual elements and relationships thereof; outputting a current composition score and an optimization gradient through a pre-trained composition aesthetics quantitative evaluation model; and generating a dynamic visual servo guidance instruction, and superposing the instruction on a view-finding interface in an augmented reality mode to guide a user to adjust the pose and focal length of the camera. The system comprises a multi-modal data acquisition module, a map construction module, an aesthetic evaluation module, a guide generation module and an AR rendering module. According to the method and the device, the crossing from a static reference line to semantization and closed-loop intelligent composition assistance is realized, the operation burden of a user is remarkably reduced, and the imaging aesthetic quality is improved.

A humanoid robot system comprises vision sensors for capturing image data, a computing architecture with processing hardware and memory, and a bipedal spatial perception model. The model includes a feature extractor that extracts hierarchical feature maps from input images, a robot data module that detects robot parts, and a robot vector data module that calculates three-dimensional spatial position and orientation data for each detected robot part. The feature extractor uses a feature pyramid network generating multi-scale feature maps through bottom-up and top-down pathways with lateral connections. The robot vector data module predicts 2D-to-3D point correspondences and solves perspective-n-point problems to obtain final position and orientation vectors, enabling real-time robot self-awareness and closed-loop visual servoing for precise object interaction.

An automatic end effector tip position initialization mechanism for a micromanipulation system is provided. The method includes: S1, positioning a macro end effector tip, calculating a camera pose by using ArUco Tag, and realizing a three-dimensional positioning of the end effector tip by using a triangulation method, and moving the end effector to a area near a center of a focal plane based on a visual servo; S2, positioning a micro end effector tip; S3, acquiring calibration data based on a visual servo, calibrating the end effector tip and the petri dish, and obtaining a left calibration matrix and a right calibration matrix reflecting the transformation relationship between left and right micromanipulator coordinates and image coordinates; generating an intuitive human-computer interaction interface based on a mouse and a keyboard according to acquired position information. The method has obvious advantages in terms of operation accuracy, operation efficiency and repeatability.

The invention discloses a flexible material automatic feeding and accurate positioning control system based on machine vision, and belongs to the technical field of machine vision and intelligent decision making. Comprising a sensing and identifying module, a decision-making and planning module and a control and execution module. The perception and recognition module recognizes a material contour and quantifies a deformation parameter of the material by using a deep learning model through an image acquisition module and a flexible material recognition and state analysis module; the decision-making and planning module selects an optimal grabbing strategy from a feeding strategy library according to the deformation state and plans a collision-free path; the control and execution module realizes smooth and stable grabbing through the self-adaptive grabbing control module in combination with a visual feedback and impedance control algorithm, and performs real-time tracking and dynamic pose compensation on materials through the accurate positioning and placing module by utilizing a visual servo technology; the industrial problems of grabbing damage, placement drifting and the like caused by the fact that flexible materials are prone to deformation are solved, and high-precision and high-stability automatic feeding is achieved.

The invention discloses a double-arm cooperative bolt alignment method and system based on hybrid visual servo, and relates to the field of robot visual servo and cooperative control, and the method comprises a visual perception module which is used for obtaining part space pose information based on first image information of a part where a target bolt is located; according to the second image information of the target bolt, obtaining a two-dimensional image coordinate and space depth information of a bolt angular point of the target bolt; the hybrid visual servo control module is used for generating a motion control instruction of the main mechanical arm by utilizing a staged visual alignment control strategy based on the spatial pose information of the part and the two-dimensional image coordinates and spatial depth information of the angular points of the bolts; and the double-arm cooperative control module is used for generating a cooperative motion instruction of the slave mechanical arm based on a cooperative control algorithm in the process that the master mechanical arm carries out visual servo alignment according to the motion control instruction. According to the method, the bolt alignment task under the complex working condition can be completed in a self-adaptive mode, and the intelligent level of overhauling operation is improved.

The invention discloses a bridge intelligent inspection unmanned aerial vehicle integrated measurement and control system, and relates to the technical field of unmanned aerial vehicle measurement and control, and the system comprises a reference construction module, a pose acquisition module, a pose matching module, a coordinate generation module, and a disease locking module. According to the method, the reference three-dimensional model and the space reference control point are constructed, so that accurate registration of the real-time video stream and the historical model is realized; driving the unmanned aerial vehicle to actively adjust the pose and lock the disease area through visual servo control; the absolute three-dimensional space coordinates of the diseases are calculated through reverse ray tracing, integration of disease recognition, active observation and accurate positioning is achieved, the automation degree of bridge inspection is improved, and the problems that in the prior art, bridge disease recognition and space positioning links are disjointed, cross-cycle data cannot be aligned, and an active observation mechanism is lacked are solved.

The invention relates to the technical field of robots and computer vision, in particular to an anti-dynamic-shielding mechanical arm visual servo method and system, and the method comprises the steps: calibrating an illumination response reference model before servo; in the servo process, the environment monitoring system senses the dynamic obstacle and predicts the movement of the dynamic obstacle; according to the predicted illumination disturbance, the active illumination array performs reverse compensation to maintain the image quality; and the system continuously evaluates the comprehensive stability index, and performs hierarchical response according to the index value, including illumination compensation, path maintenance or obstacle avoidance re-planning. According to the mechanical arm visual servo method and system capable of resisting dynamic shielding, the technical problem that in the prior art, due to the fact that illumination sudden change and physical shielding cannot be effectively processed during dynamic shielding, a visual system fails in advance is solved, and the mechanical arm visual servo method and system capable of resisting dynamic shielding can achieve pre-judgment and actively compensate illumination disturbance; servo task continuity is ensured under a multi-level response mechanism, and stability and robustness of a visual system in a complex dynamic environment are guaranteed.

The invention discloses a visual servo mechanical arm system trajectory tracking control method based on an internal model, and belongs to the technical field of mechanical arm control. The method comprises the following steps: firstly, establishing a kinematics model (including a pinhole camera perspective projection relation) and a dynamics model (including external torque interference) of a visual servo mechanical arm system; designing an image space observer aiming at the problem that the image space speed is difficult to measure; designing an internal model to process external interference, and converting a trajectory tracking problem of a visual servo mechanical arm system into a robust stabilization problem of an augmented system; and finally, designing an adaptive controller to solve the robust stabilization problem, and obtaining a final controller. According to the method, the image space speed does not need to be measured, the kinematics / dynamics parameters of the mechanical arm are allowed to be unknown, external interference can be effectively restrained, high-precision trajectory tracking of the visual servo mechanical arm system is achieved, and the method is suitable for scenes with high requirements for the control precision of the mechanical arm in the fields of industry, medical treatment, spaceflight and the like.

The invention relates to the technical field of robot micro-operation, and discloses a micro-operation tail end system integrating multi-dimensional micro-force sensing and visual servo, which comprises a data acquisition module used for synchronously acquiring binocular visual images and six-dimensional force data; the calibration registration module is used for establishing a multi-sensor unified coordinate system; the visual pose estimation module is used for realizing three-dimensional reconstruction and feature point extraction based on a deep neural network and structured light; the state fusion estimation module fuses vision and force information through an extended Kalman filter and judges a contact state; the cooperative control module is used for generating a motion instruction in combination with task priority and impedance control; the parameter self-adaptive module is used for identifying materials on line and self-adaptively adjusting force control parameters; and the motion control module adopts active damping control to suppress vibration. According to the invention, high-precision and multi-degree-of-freedom micro-operation can be realized in a narrow and limited space.

The invention relates to the technical field of port automation, and discloses a port unmanned crane grabbing pose visual positioning method. The invention aims to solve the technical problems of high system cost, poor environmental adaptability and insufficient positioning robustness caused by excessive dependence of a port unmanned crane on a multi-source expensive sensor, complex calibration deployment process and easiness in signal interference by the environment. According to the method, a double-coordinate reference frame in which a site fixed coordinate system and a crane moving space coordinate system are linked is constructed, and full-process autonomous closed-loop control from path planning to accurate grabbing and unloading is realized by deeply fusing multi-dimensional visual image information. The accurate pose of the crane is calculated through a real-time visual self-calibration mechanism; and a multi-stage visual servo strategy is adopted to guide the grabbing unit to complete high-precision operation. By constructing a set of visual positioning and control system with low cost and high robustness, the autonomy level, the environmental adaptability and the economic efficiency of the system are greatly enhanced.

The invention discloses a lunar surface four-footed impact crater detector target following method and system. The system mainly comprises a four-footed mobile platform, a binocular vision module and a data processing and control center. The binocular vision module collects images in real time, and the data processing and control center identifies and positions a front scientific detection platform target by using a deep learning algorithm; resolving a relative distance and an azimuth angle with a target in real time by combining a stereoscopic vision ranging technology; finally, a visual servo control algorithm is adopted, the calculated distance and azimuth error is converted into a control instruction, and the four-footed mobile platform is driven to adjust motion. According to the invention, artifacts and calculation amount of lunar surface reconstruction are effectively reduced, high-precision and high-reliability target perception and distance measurement in a lunar surface extreme environment are realized, the problems that a traditional wheel type platform detector is poor in trafficability in a lunar surface complex terrain environment and a target is easy to lose when a single sensor is used for tracking are solved, and the detection accuracy is improved. And the autonomy and efficiency of the detection task are improved.

This invention discloses an autonomous navigation, positioning, and drilling method for a tracked drilling robot, comprising: a global initialization and registration step, which registers BIM point clouds with real-time laser point clouds to obtain an initial pose; a multi-sensor fusion SLAM step, which fuses data from LiDAR, visual depth cameras, IMUs, and encoders to generate a dense environmental map in real time and output the pose; a BIM-constrained path planning step, which constructs a cost map fused with BIM structural constraints and plans a global path and local trajectory; a parallel mechanism positioning step, which drives the parallel mechanism to the target drilling point through inverse kinematics solving; a vision-force closed-loop drilling step, which integrates visual servoing and force / torque monitoring to achieve safe and accurate drilling; and a global error correction step, which re-registers the point cloud to eliminate accumulated errors. This invention deeply integrates BIM prior information with multi-source perception to achieve full-process automation from navigation to drilling, significantly improving positioning accuracy, safety, and construction efficiency.

This application relates to the field of robotics, and particularly to a visual servo control method and apparatus based on heterogeneous joint weighted motion decomposition. The method includes: weighting the pose deviation of the target pose measurement data based on the target pose measurement data of the agile arm; planning the expected joint angle data for the next cycle of the agile arm, and weighting the expected joint angle data to obtain the joint motion data of the agile arm; converting the joint motion data into motor motion data based on the target kinematic transformation relationship, and weighting the motor motion data according to the motor response characteristics to obtain weighted motor motion data, thereby controlling the agile arm to perform corresponding visual servo control actions. This application can weight the pose information of the target in each direction, the planned joint angle data, and the corresponding motor angle data, ensuring that the response characteristics of each motor are basically consistent, thus enabling the visual servo method to be well adapted to the motion characteristics of the tethered agile arm.

The application discloses an end gripper for adaptive gripping of photovoltaic modules, comprising an end gripper frame, a visual relative pose measurement module, a multi-axis linkage pose compensation mechanism, a vacuum suction execution module and a main controller. The visual relative pose measurement module solves the spatial relative pose parameters of the photovoltaic module relative to the end gripper frame in real time. The main controller extracts the rotational deviation amount in the parameters, generates a servo adjustment instruction to drive the multi-axis linkage pose compensation mechanism to perform iterative approximation type pose correction, forming a visual servo closed-loop control. When the rotational deviation amount converges within the preset tolerance range, it is determined that the pose matching state is reached and the vacuum suction execution module is triggered to work. The application realizes real-time adaptive compensation for dynamic angle deviation of the photovoltaic module during conveying, solves the problems that the existing end gripper lacks integrated multi-dimensional pose adjustment capability and deep collaborative closed-loop control, and significantly improves the gripping precision and automation level.

The present application relates to underwater vehicle control technology field, specifically to a kind of machine manta ray visual servo autonomous precision benthic control method and system for narrow space, comprising: setting the initialization parameter of machine manta ray fixed-point benthic;Obtain the relative pose information between machine manta ray and the mark between fixed-point benthic position;Obtain the real-time phase difference corresponding to each pectoral fin of machine manta ray;Obtain the buoyancy of machine manta ray;Machine manta ray is controlled benthically.The present application can not only realize trajectory convergence and position convergence, but also can guarantee attitude stable convergence, embodies the good convergence and operating stability of the present application in actual operation.

The invention discloses an electron microscope imaging target accurate movement control method based on visual servo, and belongs to the technical field of electron microscopic imaging and movement control. Comprising the following steps: constructing a target data set in a low-magnification scene, training a fine-tuning target tracking model, and outputting a target center point; performing real-time communication with the electron microscope to obtain low-power image frames, identifying each target by using the ideal model, generating an online target tracking task sequence, and continuously outputting a tracking frame of a current target individual in sequence; according to the pixel error, mapping a calibration coefficient related to multiplying power into a displacement reference quantity to execute PID closed loop, and driving the sample table to move until the target individual meets the center; and switching to high magnification to complete observation and returning to low magnification to complete multi-target observation one by one. According to the method, re-search and re-calibration after rate switching are avoided through the process of'retention-rollback recovery-relative displacement inheritance ', the robustness is improved in combination with thread decoupling and adaptive parameters, multi-target rapid, accurate and stable high-power observation can be achieved in a millisecond-level control period, and the method is suitable for large-scale popularization and application. The switching time delay is obviously shortened, the invalid range and the manual burden are reduced, and the experiment verifies that the method has effectiveness and superiority.

This invention discloses a method for autonomous operation of humanoid robots based on ROS. It employs a decision-making and control framework that fuses hierarchical finite state machines with YOLOv8 vision to achieve continuous autonomous execution of multiple tasks in complex unstructured environments. The method includes: system initialization and standardized processing of multimodal perception data; target 3D localization fused with YOLOv8 and coordinate transformation; task scheduling and state transitions using hierarchical finite state machines; segmented navigation and motion planning of joint trajectories; and ROS command issuance and full-process closed-loop control. This invention addresses the rigidity of traditional script-based control by fusing hierarchical state machines with visual servoing. Combined with a full-process anomaly handling mechanism and multi-target scene processing rules, it can dynamically adapt to changes in complex unstructured environments, effectively avoiding problems such as state freezes and erroneous grasping.

The invention discloses a robot visual servo control method, device and equipment and a storage medium. The method comprises the steps that a target global image corresponding to a head camera of a target robot and a target local image corresponding to a wrist camera are synchronously acquired, and the target global image and the target local image both comprise a target task object; based on the target global image and the target local image, performing adaptive pose fusion on the target task object, and determining target pose estimation corresponding to the target task object; and based on the robot motion control model and the target pose estimation, determining a target control instruction corresponding to the target robot, so that the target robot performs motion control based on the target control instruction. According to the method, efficient robot visual servo control can be realized, the adaptability to complex task scenes is improved, the labor cost is reduced, the control precision of robot task operation is improved, and the task requirements of the robot in the complex scenes are met.

The invention discloses a cluster type water surface cleaning system and method based on dynamic role distribution and front line convergence supply, and relates to the technical field of environmental protection and unmanned systems, the cluster type water surface cleaning system comprises a mother ship, a multifunctional sub-ship and an unmanned aerial vehicle cluster; the mother ship serves as a mobile command and supply center and is integrated with a garbage treatment assembly line; the sub-ship adopts a modular design, and the function can be dynamically reconfigured; the unmanned aerial vehicle cluster has garbage drift prediction and visual servo guidance; the mother ship is introduced to serve as a mobile command and supply center, a three-level intelligent framework is constructed, and the problems that in the prior art, the equipment role is fixed, time consumed for going to and fro a base is long, and the overall efficiency is low are fundamentally solved.

This invention discloses a navigation-following unmanned surface vessel (USV) formation control method based on active vision, comprising the following steps: constructing a kinematic model of the USV based on a geodetic coordinate system; establishing a gimbal monocular camera model for the gimbal camera mounted on the USV; defining servo tracking errors for the navigation-following USV formation system by combining the USV kinematic model and the gimbal monocular camera model; defining image coordinate errors based on the viewing angle constraints of the gimbal monocular camera; designing a motion controller and adaptive parameter update law for the USV using the Lyapunov synthesis method; and designing a gimbal camera steering controller and adaptive parameter update law using the Lyapunov synthesis method. This method ensures the smooth implementation of visual servo control of the USV by defining servo tracking errors and image coordinate errors, and then incorporating these two errors into the Lyapunov synthesis method to design the USV motion controller, the gimbal camera steering controller, and the adaptive parameter update law.

The application discloses a humanoid robot pouring method and device based on visual servo, and relates to the technical field of industrial vision intelligence. The method comprises the following steps: acquiring first posture information of a target object in a working environment; determining a pouring target position above a water cup based on a preset height and a water cup position; iteratively adjusting the end posture of the mechanical arm of the humanoid robot to reduce the position deviation between the spout position and the pouring target position by using a first visual servo control algorithm; when the position deviation meets a first preset condition, controlling the mechanical arm to perform a pouring action; during the execution of the pouring action, acquiring a real-time updated spout position, and compensating and adjusting the end posture of the mechanical arm based on the real-time deviation between the real-time updated spout position and the pouring target position by using a second visual servo control algorithm. The application improves the accurate alignment and real-time control of the robot pouring action, and has stronger stability and adaptability.

The application relates to the technical field of robot control, and discloses a terminal position dynamic compensation method based on fusion of visual servoing and force feedback, which comprises the following steps: collecting a top view image, segmenting an irregular residual hole contour, and calculating a dynamic alignment center coordinate; then, a reverse conical spiral descending trajectory is generated in combination with a lateral force vector to guide a fiber tail fiber connector into a guide port; in the insertion stage, the curvature radius characteristic of a force-displacement curve resistance peak point is used to determine the authenticity of bottom touching; for the case that the bottom touching is determined to be false, the side projection chord length is used to deduce an under-insertion depth, and a high-frequency micro-oscillation compensation insertion is driven until a real buckle locking feature is captured; the application realizes accurate alignment and reliable positioning of fiber plug-in and plug-out actions under complex working conditions such as shielding and mechanical wear.

The invention provides an electrical switch cabinet robot switching operation multi-source verification method and device, and the method comprises the steps: determining that a to-be-operated electrical cabinet is consistent with a background instruction through the triple verification of an Internet of Things identity tag, an electrical cabinet position coordinate and an image identifier; and fusing data three-dimensional reconstruction of an RGB camera, a depth camera and an infrared sensor, identifying operation target information by a deep learning algorithm, and comparing the operation target information with a digital monitoring system reading. A visual servo image, a six-dimensional force / torque sensor and other multi-source data are collected, and cross validation is carried out through a consistency judgment algorithm. And a recovery strategy is automatically executed in case of abnormal conditions such as jamming, slipping and tool falling, and a safety alarm is given if two times of recovery fails. Multi-level safety guarantee is constructed through physical sudden stop, software sudden stop and sensor over-limit triggering, and when communication is interrupted or main control fails, the equipment automatically returns to a safety position and keeps braking.