243 results about "Visual servoing" patented technology

A human-guided vision-force fused impedance iterative learning control method for a robotic arm, comprising: analyzing a robot-environment interaction dynamics equation, solving a visual servo acceleration model, and making use of the equation to establish a human-robotic arm-environment interaction dynamics model in an image feature space; acquiring an image feature position and speed curve of a human-guided robot completing an assembly task, and using dynamic movement primitives for coding and generalization; and designing an impedance iterative learning controller which uses image feature tracking errors as control input, learning impedance characteristics when the human-guided robot performs a contact operation, identifying unknown contact dynamics under the interaction between the robot and the environment, and counteracting identified contact interference in the feature space, so as to implement a flexible assembly operation. The control method solves the problems in existing assembly operations that human-robotic arm-environment coupling nonlinear dynamics, unknown contact dynamics of intensive contact assembly tasks and poor generalization of assembly scenarios require relearning for different scenarios, etc.

The invention relates to the technical field of automatic workpiece grabbing and visual servo control, and discloses a workpiece grabbing method and system based on visual feedback, and the method comprises the steps: obtaining a real-time image through a front-view camera, a side-view camera and a top-view camera, and extracting surface features through a feature fusion network; inputting the features into a pose estimation model based on a particle filtering framework, and iteratively updating the pose in combination with a multi-dimensional observation likelihood function to generate prediction parameters; constructing a multi-target path planning model based on the parameters, and optimizing the path by adopting a dynamic planning algorithm with the shortest path and the minimum joint movement as targets; a hierarchical control model is established, a strategy layer performs global planning, an adjustment layer corrects a local path, and an execution layer realizes trajectory tracking through visual servo control and outputs a control instruction to complete grabbing. Through multi-view perception, robust pose estimation, global optimization path planning and hierarchical control, the precision, efficiency and stability of workpiece grabbing in a complex environment are improved.

The invention discloses a high-precision laser positioning unpacking path planning system, and belongs to the technical field of robot automatic control and industrial automation. The system comprises a data synchronization module used for multi-sensor hardware synchronization and data alignment; the high-precision positioning module is used for outputting a precise pose based on environment skeleton characteristics and sliding window optimization; the semantic map construction module is used for fusing vision and laser data to generate a dynamic semantic grid map; the global path planning module is used for planning a smooth path in the skeleton channel by utilizing a mixed potential field improved A * algorithm; the motion control module is used for realizing closed-loop motion control and safety monitoring through model prediction and tracking; and the operation execution module is used for finishing millimeter-level precise stopping of an operation point by adopting visual servo and triggering unpacking operation. According to the method, the positioning robustness under dynamic shielding is improved through the environmental skeleton features, the safety and the high efficiency of the path are ensured by utilizing semantic understanding and intelligent planning, and the full-process automation from navigation to precise operation is realized.

The invention discloses a robot automatic grabbing path planning method based on visual identification, and relates to the technical field of intelligent grabbing. The method comprises the following steps: acquiring multi-view visual data of a target scene, and generating a scene three-dimensional compact reconstruction model through a multi-modal image fusion algorithm; performing target detection and feature extraction on the model, and screening an optimal capture point in combination with a visual attention mechanism; constructing a dynamic environment obstacle probability map, updating an obstacle state through time sequence visual tracking, and quantifying an interference weight; an initial grabbing path is planned based on an improved fast expansion random tree algorithm, path smoothness constraints and robot joint movement limit parameters are introduced, and path nodes are optimized through a Bezier curve; visual servo feedback and path deviation prediction are fused, path parameters are corrected in real time, and a continuous movement track is generated. The method effectively adapts to the dynamic environment, gives consideration to path safety, smoothness and mechanical arm motion characteristics, and remarkably improves the grabbing success rate and operation reliability.

The invention discloses a mobile robot accurate docking method based on edge calculation, and aims to solve the problems that the dynamic docking precision is reduced and the collision risk is increased due to micro-motion or drifting of a target station. According to the method, unified time reference alignment is carried out on data of a camera, a laser radar, an inertial measurement unit, an ultra-wideband range finder, a station encoder and a programmable logic controller at an edge node, and a three-dimensional special Euclidean group equivariant multi-source fusion network is used for outputting relative pose estimation and covariance; the estimation in the time window is further used as a condition to be input into a conditional diffusion short-time prediction model to obtain a time-varying mean value and a time-varying covariance, an anisotropic probability tube is constructed, and prediction-measurement joint correction is carried out based on a score function; scenarized opportunity constraints are constructed under correction probability tube constraints, a tubular nonlinear model is adopted to predict, control and solve a reference trajectory, an actuator command is generated in combination with depth visual servo and compliance control, and the technical effects of high precision, robustness and safe docking under the station dynamic disturbance condition are achieved.

The invention relates to the technical field of intelligent control, and discloses an intelligent control method and system for an electromechanical module intelligent installation robot. The method comprises the following steps: fusing multi-sensor data through a Kalman filtering algorithm to generate a standardized space state vector, carrying out three-dimensional path planning on a BIM model based on the vector to obtain a path instruction set, and carrying out precise positioning correction on the path instruction set by adopting a visual servo algorithm to obtain a corrected motion instruction. A synchronous control instruction is generated through synchronous coordination processing of multiple actuators, and finally safe obstacle avoidance evaluation processing is performed in combination with laser radar scanning data to obtain a safe execution instruction, so that intelligent precision installation of the electromechanical module is realized. The technical problems of inaccurate multi-sensor data fusion, lack of three-dimensional intelligence in path planning, insufficient visual servo precision, difficulty in synchronous coordination of multiple actuators and lack of predictability in safe obstacle avoidance in intelligent installation of the electromechanical module are solved.

The invention belongs to the technical field of intelligent and robot operation, and discloses a collaborative robot closed-loop operation method based on scene perception and thinking chain reasoning, which comprises the following steps of: firstly, generating structured semantic representation containing confidence by utilizing a visual language model of LoRA efficient fine tuning and fusing multi-view data, and providing semantic understanding for a robot; secondly, utilizing a three-dimensional diffusion model to generate explicit geometric priori under the condition of no real CAD, and correspondingly completing PnP initial pose estimation in combination with 2D-3D; then, carrying out pose optimization by adopting a tracking-refining strategy, and triggering reinitialization based on geometric prior when a residual error exceeds a threshold value; and finally, decomposing a global task by utilizing a thinking chain reasoning module, and realizing'perception-reasoning-execution 'closed-loop control based on visual servo in cooperation with an execution module. The method realizes effective combination of large model reasoning and physical execution, has strong robustness under dynamic disturbance, and is suitable for intelligent manufacturing and man-machine cooperation scenes.

The invention relates to the field of industrial robot visual servo control, in particular to a visual servo grabbing dynamic compensation system. The system comprises an image acquisition module, a motion analysis module, a local modeling module, a prediction module and a compensation execution module. A target image sequence is continuously captured, a motion track is extracted, a non-parametric local motion model is established based on recent data, the position of a target at the future grabbing moment is predicted, and a compensation signal is generated according to a prediction result to adjust the grabbing track of a robot. According to the method, the motion model is constructed by adopting a local weighted regression method, accurate description and rapid adaptation of various motion modes are realized through adaptive bandwidth adjustment and an online updating mechanism, and the grabbing success rate of a moving target is effectively improved.

The invention discloses an autonomous sewing system and method based on multi-modal sensing and self-adaptive control. The autonomous sewing system comprises a robot module, a sewing machine module, a visual sensing module and an intelligent control platform. According to the cloth pose registration module, high-precision registration of a flexible target is realized through a two-stage registration process, a local area weighting ICP and a non-rigid cloth surface registration method; the intelligent control module adopts a visual servo dynamic path tracking control method, dynamically corrects the posture of the robot, improves the path tracking precision and the sewing thread consistency, and ensures machine-machine collaborative synchronization and uniform sewing thread uniformity by constructing a speed, track and tension three-factor coupling synchronous control method. By adopting an error adjustment and tension stabilization mechanism based on self-adaptive control, the robustness of the system in a complex sewing state is improved; and through the data analysis module and the exception handling module, full-process monitoring, quality evaluation and exception grading response are realized, and the stability and reliability of the system are improved.

The invention discloses an unmanned aerial vehicle relay tracking system and method for cloud side-end cooperation in an urban scene, and belongs to the technical field of intelligent traffic and unmanned aerial vehicle cooperative control. In order to solve the problems of insufficient endurance and limited sight distance of existing single-machine tracking in an urban large-range scene, a physically distributed and logically unified cloud-side-end three-level architecture is constructed, wherein a cloud end is responsible for natural language instruction analysis and global road network monitoring; the edge end serves as a regional scheduling hub and is responsible for operating a high-precision detection model, extracting a target Re-ID feature fingerprint and calculating a cross-machine relay time window; and the terminal unmanned aerial vehicle only operates a lightweight tracking model and a visual servo control law. According to the method, the urban area is divided into a plurality of honeycomb grids, seamless locking and relay of a target among unmanned aerial vehicles in different grid responsibility areas are realized by using a cross-view feature relay protocol and a collaborative finite-state machine, and the engineering problem of long-time-sequence and large-range vehicle tracking is effectively solved.

The invention belongs to the technical field of robot control, and discloses a visual servo intelligent robust control method for a mobile mechanical arm for a complex operation task, which comprises the following steps of: 1, acquiring a working scene image, and calculating a space coordinate of a target feature point in real time through a projection transformation model; step 2, dynamically inhibiting visual measurement noise by adopting adaptive Kalman filtering, generating a feedforward compensation signal through an integral sliding mode observer, and decoupling chassis slippage disturbance; step 3, inputting the pose signal into a radial RBF neural network, designing an RBF gain scheduler, and adjusting an output proportion-integral gain in real time to suppress external time-varying disturbance; 4, adopting a hybrid visual servo mode switching mechanism, and generating a mobile platform control instruction through an integral sliding mode surface in a position servo mode; and 5, designing a three-order composite controller to drive the mechanical arm so as to realize accurate grabbing. According to the method, the grabbing deviation caused by kinematics uncertainty of the mobile platform and dynamic environment disturbance is eliminated.

The invention discloses a rope-driven flexible mechanical arm target tracking method and system based on visual servo, and the method comprises the steps: carrying out the real-time tracking of a target through a mixed visual module, and continuously outputting the bounding box information and tracking confidence score of the target; according to the method, visual errors are used as guidance, Jacobi is driven by learned data to serve as an accurate conversion model, and a tracking confidence score is introduced to dynamically adjust the response degree of control, so that an accurate and smooth motor driving instruction is solved. The system comprises a mixed target tracking module, a kinematics and jacobian identification module and a self-adaptive control module. By using the method, the tracking precision of the mechanical arm is improved by utilizing a vision and control cooperation mechanism. The method can be widely applied to the field of mechanical control.

The invention relates to the technical field of intelligent photography, discloses an intelligent composition auxiliary method and system for tourism photography, and aims to solve the problems that scene semantics cannot be deeply understood and dynamic active composition guidance is difficult to provide in the prior art. The method comprises the following steps: acquiring multi-modal data such as images, geographic coordinates, equipment postures and the like in real time; constructing a scene semantic map for describing visual elements and relationships thereof; outputting a current composition score and an optimization gradient through a pre-trained composition aesthetics quantitative evaluation model; and generating a dynamic visual servo guidance instruction, and superposing the instruction on a view-finding interface in an augmented reality mode to guide a user to adjust the pose and focal length of the camera. The system comprises a multi-modal data acquisition module, a map construction module, an aesthetic evaluation module, a guide generation module and an AR rendering module. According to the method and the device, the crossing from a static reference line to semantization and closed-loop intelligent composition assistance is realized, the operation burden of a user is remarkably reduced, and the imaging aesthetic quality is improved.

The invention discloses a double-arm robot main and auxiliary moving operation coordination control method based on vision / force sense servo and implementation thereof. Double-arm coordination is supported, multi-sensing information is fused, compliant operation is considered, and the method is adaptive to a moving scene. The mobile main and auxiliary double-arm robot control system comprises a mobile base subsystem, a main and auxiliary double-arm subsystem and a control subsystem. The double-arm main and auxiliary operation coordination control method based on visual / force sense servo comprises visual servo target tracking based on a tail end camera and force sense servo compliant operation control based on a force sensor, operation tasks are achieved in the state that a base moves, and the double-arm main and auxiliary operation coordination control method adapts to the complex working environment; a state machine mechanism is adopted to manage a mechanical arm motion mode, screw assembly task-oriented operation process arrangement is realized, and the efficiency and controllability of the operation process are improved.

Provided herein are methods of tracking the positions of medical devices using photoacoustic visual servoing. Additional methods as well as related systems and computer readable media are also provided.

The invention discloses a transformer oil sample automatic collection method and device based on vision and impedance control, and the method comprises the steps: determining a relative position between an oil taking injector and an oil taking port in a transformer oil sample collection process, and determining an ideal pose of the oil taking injector controlled by a mechanical arm to reach an oil taking port mark based on the relative position; based on image information collected in real time by an image taking device arranged on the mechanical arm, a space coordinate transformation matrix of the ideal pose at the oil taking opening mark and the actual pose of the image taking device is calculated; on the basis of the ideal pose and the space coordinate transformation matrix which are calculated in real time, according to a control law of pose deviation design, a visual servo control rule for the pose of the oil taking injector is determined; based on interaction between the working environment and the tail end of the mechanical arm, a butt joint depth control rule of an oil taking injector and an oil taking opening is determined through an impedance controller; based on a visual servo control rule and a butt joint depth control rule, butt joint of an oil taking injector and an oil taking opening is controlled, and transformer oil sample collection is carried out.

The invention provides a robot mixed visual servo positioning method, device and equipment. The method provided by the invention comprises the steps of collecting a visual image of a curved workpiece under a current pose, and collecting distance data from the tail end of a robot to the curved workpiece; identifying feature points from the visual image, and determining pixel coordinates of the circle center of the pre-assembly hole; determining depth information according to the distance data, constructing a first jacobian matrix of the image features and a second jacobian matrix of the distance features, and determining a joint feature jacobian matrix; determining a feature point error matrix according to the pixel coordinates of the feature points; determining a pose error matrix according to the current pose and the target pose, and determining an overall error matrix; and inputting the overall error matrix and the joint feature Jacobian matrix into a pre-trained decision model to control the robot. The robot hybrid visual servo positioning method, device and equipment provided by the invention are used for remarkably improving the success rate and the operation efficiency of the robot during assembly.

A humanoid robot system comprises vision sensors for capturing image data, a computing architecture with processing hardware and memory, and a bipedal spatial perception model. The model includes a feature extractor that extracts hierarchical feature maps from input images, a robot data module that detects robot parts, and a robot vector data module that calculates three-dimensional spatial position and orientation data for each detected robot part. The feature extractor uses a feature pyramid network generating multi-scale feature maps through bottom-up and top-down pathways with lateral connections. The robot vector data module predicts 2D-to-3D point correspondences and solves perspective-n-point problems to obtain final position and orientation vectors, enabling real-time robot self-awareness and closed-loop visual servoing for precise object interaction.

The invention discloses a robot variable stiffness vision-impedance control method and system based on optimization, and the method comprises the steps: building a coordinate transformation relation between a camera and a six-dimensional force sensor and a coordinate transformation relation between the camera and the tail end of a robot, calibrating the six-dimensional force sensor, and analyzing the interaction dynamics relation between the robot and the environment. A robot-environment interaction mathematical model is constructed, and a feature space human-robot-environment interaction dynamical model is established by solving a visual servo acceleration model and combining a dynamical equation of the robot in the assembly process; establishing a feature space human-guided dynamics model as a planner to obtain a reference feature trajectory implying human recognition contact dynamics; a characteristic space variable stiffness impedance controller based on QP online planning is designed, impedance parameters can be dynamically adjusted in the assembly process, interference force caused by environment uncertainty is compensated in real time, constant force is kept in a rich contact task, and it is ensured that the robot can stably complete the assembly task with high precision.

An automatic end effector tip position initialization mechanism for a micromanipulation system is provided. The method includes: S1, positioning a macro end effector tip, calculating a camera pose by using ArUco Tag, and realizing a three-dimensional positioning of the end effector tip by using a triangulation method, and moving the end effector to a area near a center of a focal plane based on a visual servo; S2, positioning a micro end effector tip; S3, acquiring calibration data based on a visual servo, calibrating the end effector tip and the petri dish, and obtaining a left calibration matrix and a right calibration matrix reflecting the transformation relationship between left and right micromanipulator coordinates and image coordinates; generating an intuitive human-computer interaction interface based on a mouse and a keyboard according to acquired position information. The method has obvious advantages in terms of operation accuracy, operation efficiency and repeatability.

The invention discloses an irregular part uncalibrated grabbing method and system, and belongs to the technical field of mechanical control, and the method comprises the steps: image collection: based on a depth camera installed on an end effector of a robot, capturing a real-time RGB image and a depth image, and outputting image data; rotating target detection and positioning: building a rotating frame grabbing target detection model and training, and executing rotating frame grabbing target detection; servo control of image vision: extracting current image features, defining expected image features, designing an image vision servo control law, and obtaining the joint speed of the robot; grabbing is performed through robot motion execution and iteration. A closed-loop visual servo mode directly calculates errors in an image space and generates a control instruction, so that not only is a tedious hand-eye calibration step in a traditional method avoided, but also an end effector of a robot can be continuously guided to accurately and robustly approach a target grabbing pose, system errors and environmental disturbance are effectively compensated, and grabbing positioning accuracy and success rate are improved.

The invention discloses a flexible material automatic feeding and accurate positioning control system based on machine vision, and belongs to the technical field of machine vision and intelligent decision making. Comprising a sensing and identifying module, a decision-making and planning module and a control and execution module. The perception and recognition module recognizes a material contour and quantifies a deformation parameter of the material by using a deep learning model through an image acquisition module and a flexible material recognition and state analysis module; the decision-making and planning module selects an optimal grabbing strategy from a feeding strategy library according to the deformation state and plans a collision-free path; the control and execution module realizes smooth and stable grabbing through the self-adaptive grabbing control module in combination with a visual feedback and impedance control algorithm, and performs real-time tracking and dynamic pose compensation on materials through the accurate positioning and placing module by utilizing a visual servo technology; the industrial problems of grabbing damage, placement drifting and the like caused by the fact that flexible materials are prone to deformation are solved, and high-precision and high-stability automatic feeding is achieved.

The invention discloses a double-arm cooperative bolt alignment method and system based on hybrid visual servo, and relates to the field of robot visual servo and cooperative control, and the method comprises a visual perception module which is used for obtaining part space pose information based on first image information of a part where a target bolt is located; according to the second image information of the target bolt, obtaining a two-dimensional image coordinate and space depth information of a bolt angular point of the target bolt; the hybrid visual servo control module is used for generating a motion control instruction of the main mechanical arm by utilizing a staged visual alignment control strategy based on the spatial pose information of the part and the two-dimensional image coordinates and spatial depth information of the angular points of the bolts; and the double-arm cooperative control module is used for generating a cooperative motion instruction of the slave mechanical arm based on a cooperative control algorithm in the process that the master mechanical arm carries out visual servo alignment according to the motion control instruction. According to the method, the bolt alignment task under the complex working condition can be completed in a self-adaptive mode, and the intelligent level of overhauling operation is improved.

The present application relates to a robust adaptive variable admittance control method based on different intentions in a feature space. The method includes: using the kinematic and dynamic models of the manipulator to determine the mapping relationship between the actual contact force and the virtual contact force; determining the adaptive stiffness coefficient based on the actual contact force change, and determining the robust adaptive virtual damping coefficient based on the direct intention based on the virtual motion speed of the feature point; adjusting the robust adaptive virtual damping coefficient based on the contact force expected by the robot, and determining the admittance control virtual damping coefficient based on direct and indirect intentions; determining the robust adaptive variable admittance control based on human intention in the feature space based on the mapping relationship, the adaptive stiffness coefficient and the admittance control virtual damping coefficient; determining the robot image visual servoing control law based on the real-time image feature points and the expected feature points, obtaining the real-time joint velocity of the manipulator, and controlling the manipulator movement.

The invention discloses an unmanned aerial vehicle autonomous approaching and netting control system based on visual servo, and belongs to the technical field of unmanned aerial vehicle control. The high-precision two-axis pan-tilt mechanism, the binocular vision module, the netting launching device and the electromechanical system main control unit of a hierarchical control algorithm are integrated, automatic recognition, accurate distance measurement, stable tracking and intelligent striking of a dynamic target are achieved, and the two-axis pan-tilt mechanism enables the netting launching device to have the motion freedom degree independent of an unmanned aerial vehicle platform. The effective striking range of the system is effectively expanded by combining a high-precision target positioning and servo tracking algorithm based on binocular vision, the capturing precision and the adaptive capacity of a dynamic target are remarkably improved, and through fusion of deep learning target detection, binocular vision pose calculation and a hierarchical control strategy, the dynamic target capturing precision and the adaptive capacity of the system are improved. The full-process independence from target identification, tracking approximation to attack decision making is realized, and a multi-subsystem efficient and collaborative working architecture is formed through unified scheduling of the main control unit.

The invention discloses a bridge intelligent inspection unmanned aerial vehicle integrated measurement and control system, and relates to the technical field of unmanned aerial vehicle measurement and control, and the system comprises a reference construction module, a pose acquisition module, a pose matching module, a coordinate generation module, and a disease locking module. According to the method, the reference three-dimensional model and the space reference control point are constructed, so that accurate registration of the real-time video stream and the historical model is realized; driving the unmanned aerial vehicle to actively adjust the pose and lock the disease area through visual servo control; the absolute three-dimensional space coordinates of the diseases are calculated through reverse ray tracing, integration of disease recognition, active observation and accurate positioning is achieved, the automation degree of bridge inspection is improved, and the problems that in the prior art, bridge disease recognition and space positioning links are disjointed, cross-cycle data cannot be aligned, and an active observation mechanism is lacked are solved.

The invention discloses an unmanned aerial vehicle accurate landing method based on multi-mode fusion perception, and relates to the technical field of unmanned aerial vehicle landing control. According to the method, an unmanned aerial vehicle needs to be guided to descend to a preset height through a first positioning module, then the unmanned aerial vehicle is switched to an airborne visual recognition module to recognize a ground visual target, if recognition succeeds, a multi-mode fusion model based on scale and space alignment is adopted to accurately calculate the offset, and the offset is calculated; and finally, a closed-loop control method based on visual servo is adopted, a speed instruction is generated and corrected in real time according to the calculated offset until precise landing at the target position is achieved, and therefore the centimeter-level landing precision, the whole-process autonomy and the high-safety landing process are achieved, and remarkable technical advantages and commercial application value are achieved.

The invention relates to the technical field of robots and computer vision, in particular to an anti-dynamic-shielding mechanical arm visual servo method and system, and the method comprises the steps: calibrating an illumination response reference model before servo; in the servo process, the environment monitoring system senses the dynamic obstacle and predicts the movement of the dynamic obstacle; according to the predicted illumination disturbance, the active illumination array performs reverse compensation to maintain the image quality; and the system continuously evaluates the comprehensive stability index, and performs hierarchical response according to the index value, including illumination compensation, path maintenance or obstacle avoidance re-planning. According to the mechanical arm visual servo method and system capable of resisting dynamic shielding, the technical problem that in the prior art, due to the fact that illumination sudden change and physical shielding cannot be effectively processed during dynamic shielding, a visual system fails in advance is solved, and the mechanical arm visual servo method and system capable of resisting dynamic shielding can achieve pre-judgment and actively compensate illumination disturbance; servo task continuity is ensured under a multi-level response mechanism, and stability and robustness of a visual system in a complex dynamic environment are guaranteed.

The invention discloses a visual servo mechanical arm system trajectory tracking control method based on an internal model, and belongs to the technical field of mechanical arm control. The method comprises the following steps: firstly, establishing a kinematics model (including a pinhole camera perspective projection relation) and a dynamics model (including external torque interference) of a visual servo mechanical arm system; designing an image space observer aiming at the problem that the image space speed is difficult to measure; designing an internal model to process external interference, and converting a trajectory tracking problem of a visual servo mechanical arm system into a robust stabilization problem of an augmented system; and finally, designing an adaptive controller to solve the robust stabilization problem, and obtaining a final controller. According to the method, the image space speed does not need to be measured, the kinematics / dynamics parameters of the mechanical arm are allowed to be unknown, external interference can be effectively restrained, high-precision trajectory tracking of the visual servo mechanical arm system is achieved, and the method is suitable for scenes with high requirements for the control precision of the mechanical arm in the fields of industry, medical treatment, spaceflight and the like.

The invention relates to the technical field of robot micro-operation, and discloses a micro-operation tail end system integrating multi-dimensional micro-force sensing and visual servo, which comprises a data acquisition module used for synchronously acquiring binocular visual images and six-dimensional force data; the calibration registration module is used for establishing a multi-sensor unified coordinate system; the visual pose estimation module is used for realizing three-dimensional reconstruction and feature point extraction based on a deep neural network and structured light; the state fusion estimation module fuses vision and force information through an extended Kalman filter and judges a contact state; the cooperative control module is used for generating a motion instruction in combination with task priority and impedance control; the parameter self-adaptive module is used for identifying materials on line and self-adaptively adjusting force control parameters; and the motion control module adopts active damping control to suppress vibration. According to the invention, high-precision and multi-degree-of-freedom micro-operation can be realized in a narrow and limited space.