321 results about "Visual servoing" patented technology

The invention relates to the technical field of marine monitoring, in particular to a marine target visual servo tracking method, which comprises the steps of multi-modal sensor fusion, a self-adaptive visual tracking algorithm, a servo control and visual collaboration mechanism and a shielding processing and target re-identification strategy. The problem that tracking is unstable under the conditions of illumination change, ship body shaking, target shielding and the like in a traditional method is solved. The IMU, the GNSS and the visual data are fused through extended Kalman filtering, ship body shaking is compensated, and the target state estimation precision is improved; the improved D-Fi ne target detection model is combined with an online feature updating mechanism to dynamically adapt to the appearance change of the target; the prediction and correction control strategy and the double-closed-loop PI D controller cooperate to adjust the camera holder, and the tracking delay is reduced; the multi-clue shielding detection and space-time joint feature matching technology ensures accurate re-identification of the target after shielding is removed. The real-time performance and robustness of the tracking system on an embedded platform are improved, and an efficient and stable target tracking solution is provided.

A human-guided vision-force fused impedance iterative learning control method for a robotic arm, comprising: analyzing a robot-environment interaction dynamics equation, solving a visual servo acceleration model, and making use of the equation to establish a human-robotic arm-environment interaction dynamics model in an image feature space; acquiring an image feature position and speed curve of a human-guided robot completing an assembly task, and using dynamic movement primitives for coding and generalization; and designing an impedance iterative learning controller which uses image feature tracking errors as control input, learning impedance characteristics when the human-guided robot performs a contact operation, identifying unknown contact dynamics under the interaction between the robot and the environment, and counteracting identified contact interference in the feature space, so as to implement a flexible assembly operation. The control method solves the problems in existing assembly operations that human-robotic arm-environment coupling nonlinear dynamics, unknown contact dynamics of intensive contact assembly tasks and poor generalization of assembly scenarios require relearning for different scenarios, etc.

The invention relates to the technical field of automatic workpiece grabbing and visual servo control, and discloses a workpiece grabbing method and system based on visual feedback, and the method comprises the steps: obtaining a real-time image through a front-view camera, a side-view camera and a top-view camera, and extracting surface features through a feature fusion network; inputting the features into a pose estimation model based on a particle filtering framework, and iteratively updating the pose in combination with a multi-dimensional observation likelihood function to generate prediction parameters; constructing a multi-target path planning model based on the parameters, and optimizing the path by adopting a dynamic planning algorithm with the shortest path and the minimum joint movement as targets; a hierarchical control model is established, a strategy layer performs global planning, an adjustment layer corrects a local path, and an execution layer realizes trajectory tracking through visual servo control and outputs a control instruction to complete grabbing. Through multi-view perception, robust pose estimation, global optimization path planning and hierarchical control, the precision, efficiency and stability of workpiece grabbing in a complex environment are improved.

The invention discloses a visual servo double-arm robot migration simulation learning method from simulation to reality, which comprises the following steps: S110, constructing a virtual simulation scene according to a real task scene, and establishing a relation between the virtual simulation scene and the real task scene; s120, the collected teaching mechanical arm and executing mechanical arm operation data are replayed and optimized in the virtual simulation scene, so that a data set used for final training is constructed; s130, a deep imitation learning network is designed and achieved, input of the deep imitation learning network comprises visual data obtained in the real task scene and state data of all joint motors of the teaching mechanical arm, and output of the deep imitation learning network is predicted states of all joint motors of the execution mechanical arm at the next moment; and S140, migrating the fully trained and converged deep imitation learning network from a virtual simulation scene to a double-arm robot in a real task scene. According to the invention, the exploration efficiency and generalization ability of the two-arm robot learning network are improved.

The invention relates to the technical field of welding automation and robot visual perception, and discloses a robot intelligent welding method and system.The method comprises the steps that a three-dimensional point cloud image is constructed through a laser triangulation method; filtering and enhancing the point cloud image, and extracting a weld point cloud set by adopting a U-Net semantic segmentation model; carrying out space path identification and attitude regression based on the set to obtain a track element of a position-attitude pair; fitting a path by adopting a cubic B-spline curve and interpolating to generate a continuous welding track; and in the welding process, visual feedback is combined, and proportional differential control logic is introduced for closed-loop track error compensation. Compared with the technical problem that in the prior art, weld joint path recognition and accurate tracking cannot be stably achieved under complex structures such as curved surfaces and variable cross sections, a perception control closed loop based on visual servo is constructed, high-precision automatic welding of the complex weld joint structures is achieved, and the track fitting precision is improved.

The invention relates to a dynamic cooperative path planning and autonomous landing control method for an unmanned aerial vehicle and an unmanned ship, and the method comprises the following steps: collecting the state data of the unmanned ship and the unmanned aerial vehicle and the high sea condition environment data, and carrying out the data fusion processing; an improved fast extended random tree algorithm is adopted to plan a safe navigation path for the unmanned ship, the generated path of the unmanned ship is optimized based on an unmanned ship motion model considering environmental interference, and a model prediction control algorithm is adopted to plan a path for the unmanned ship to fly to a predicted landing point of the unmanned ship. Dynamic cooperation between the unmanned ship and the unmanned aerial vehicle is realized through information interaction between the unmanned ship and the unmanned aerial vehicle; the unmanned aerial vehicle recognizes an unmanned ship landing identifier through a visual servo guide system, calculates the position and attitude deviation relative to a landing point, adjusts flight control parameters according to interference conditions by using an adaptive control algorithm, and performs trajectory prediction compensation in combination with an unmanned ship motion model to realize autonomous landing. Compared with the prior art, safe landing of the unmanned aerial vehicle on the unmanned ship under the high sea condition is achieved.

The invention discloses a high-precision laser positioning unpacking path planning system, and belongs to the technical field of robot automatic control and industrial automation. The system comprises a data synchronization module used for multi-sensor hardware synchronization and data alignment; the high-precision positioning module is used for outputting a precise pose based on environment skeleton characteristics and sliding window optimization; the semantic map construction module is used for fusing vision and laser data to generate a dynamic semantic grid map; the global path planning module is used for planning a smooth path in the skeleton channel by utilizing a mixed potential field improved A * algorithm; the motion control module is used for realizing closed-loop motion control and safety monitoring through model prediction and tracking; and the operation execution module is used for finishing millimeter-level precise stopping of an operation point by adopting visual servo and triggering unpacking operation. According to the method, the positioning robustness under dynamic shielding is improved through the environmental skeleton features, the safety and the high efficiency of the path are ensured by utilizing semantic understanding and intelligent planning, and the full-process automation from navigation to precise operation is realized.

The invention belongs to the technical field of position tracking control of rigid multi-joint mechanical arms, and discloses a fuzzy self-adaptive visual servo control method and system for a multi-joint mechanical arm system for placing bobbins in a textile workshop, and the method comprises the steps: constructing an image error model according to the positions of feature points on an image plane; the fuzzy logic system is used for processing uncertain items in the system, the instruction filtering and fuzzy self-adaption technology is used for achieving tracking control over the expected trajectory of the system, the problem of calculation complexity is solved, and adverse effects caused by filtering errors are eliminated. In the process of changing the bobbins, the bobbins are placed on a creel through a multi-degree-of-freedom robotic arm, a clamping jaw at the tail end of the robotic arm is required to accurately grab the bobbins for feeding and discharging, and the requirement for accurate and rapid control of the robotic arm is met, so that research on visual servo control of the robotic arm is of great significance. The method has a good control effect, the problem of calculation complexity in the design process of the controller is solved, the trajectory tracking error is smaller, and the system robustness is higher.

The invention discloses a fluid loading and unloading arm dynamic flange automatic docking method based on image visual servo, which comprises the following steps: initializing the tail end position of a loading and unloading arm, and controlling the loading and unloading arm to move until complete image information of a flange to be docked can be acquired; the method comprises the following steps: acquiring an image of a flange to be butted, and extracting initial candidate circles with circle features in the image by adopting a Hough circle detection algorithm; through an RANSAC (Random Sample Consensus) algorithm, performing refined fitting to obtain a refined candidate circle; extracting the center coordinates of the positioning holes from the refined candidate circle; according to the circle center coordinate of the positioning hole and the butt joint state coordinate position, calculating the current time deviation; taking the minimum deviation as a target, solving an angular motion angular velocity instruction of each joint of the loading and unloading arm, driving each joint of the loading and unloading arm to adjust the pose until the deviation converges, and completing the butt joint of the flange at the tail end of the loading and unloading arm and the to-be-butt-jointed flange. The method is suitable for millimeter-level accurate butt joint and control of loading and unloading arm flanges in the dynamic marine environment, and the operation efficiency and safety in the complex environment are remarkably improved.

The invention discloses a visual servo control method and system for assembling multi-shaft-hole parts through multiple mechanical arms in a tight coordination mode, and belongs to the field of multi-robot cooperative control. The invention aims to solve the problems of visual feature extraction and collaborative assembly control in the task of tightly coordinating and assembling large-size multi-shaft-hole parts by multiple mechanical arms. The method is technically characterized in that a shaft hole assembly task image data set containing different hole shapes is established and used for training a positioning neural network and a relative state estimation neural network, and a state classification neural network is used for dividing the states of shafts and holes in images into three types of shielding, separation and overlapping; the positioning neural network is used for determining the positions of the shaft and the hole in the image. A multi-mechanical-arm visual feature fusion method based on the virtual acting force is provided, the virtual acting force acting on an object is calculated according to position information and relative state information of a shaft and a hole in an image, and the virtual acting force is input into a cooperative controller of the multiple mechanical arms to achieve cooperative assembly movement of the multiple mechanical arms. And natural combination of visual features and a multi-mechanical-arm cooperative controller is achieved. Finally, the method provided by the invention is verified in the task of tightly coordinating and assembling the double-shaft-hole parts by the double mechanical arms with the assembling clearance of 0.2 mm, the success rate reaches 85%, and the effectiveness of the method is verified.

The invention discloses a mechanical arm visual servo control method and equipment based on target condition reinforcement learning, and belongs to the technical field of mechanical arm control. A mechanical arm environment is built, a camera is additionally arranged at the tail end of a mechanical arm, a computer is used for designing a safety reinforcement learning model, then online learning is carried out according to a depth deterministic strategy gradient algorithm and a post-experience playback algorithm, and the mechanical arm environment is matched. The position of the feature point in the camera picture serves as input to control the mechanical arm to move and lock the two-dimensional code attached to the surface of the object, and finally the visual servo task is completed. The method is simple in step, low in environment requirement and rapid in response of the mechanical arm, and compared with a traditional control method based on pseudo-inverse, the method has remarkable effect improvement in the aspects of the number of iterations and robustness.

The invention provides an unmanned aerial vehicle visual servo intrusion target tracking and intercepting method based on time delay compensation. The method comprises the following steps: acquiring a latest image of a tracking target through a camera on an unmanned aerial vehicle; based on the latest image, determining the position of the tracking target in an image coordinate system of the latest image as a first position; determining a position error of the first position by adopting a prediction algorithm based on improved Kalman filtering based on the delay time of collecting the latest image; based on the position error, determining a control parameter used for visual servo tracking of the corresponding tracking target; and based on the first position and the control parameter, controlling the unmanned aerial vehicle to fly to track and intercept the corresponding tracking target. Therefore, the position error compensation is performed through the delay time of the latest image, and the unmanned aerial vehicle is controlled to track the corresponding target based on the position error, so that the tracking accuracy is improved, the tracking interception error is reduced, and the interception success rate is improved.

The invention relates to a mechanical arm visual servo control method and system based on deep reinforcement learning. Firstly, real-time state observation and collision detection are conducted on the mechanical arm, and an observation result is obtained; then, a deep reinforcement learning controller is built by means of a maximum entropy deep reinforcement learning SAC algorithm network architecture, and an action space, a state space and a reward function are built according to an observation result and a target position; and then, outputting a first action representation based on the construction content, optimizing a controller according to a calculation result of the first action representation and the reward function, smoothly optimizing the first action representation by using a real-time track interpolation strategy to obtain a second action representation, and controlling the mechanical arm joint according to the second action representation. And the process is continuously repeated until the mechanical arm successfully arrives at the target position. Compared with the prior art, the method has the advantages of high control real-time performance, large application range and the like.

The invention discloses a robot automatic grabbing path planning method based on visual identification, and relates to the technical field of intelligent grabbing. The method comprises the following steps: acquiring multi-view visual data of a target scene, and generating a scene three-dimensional compact reconstruction model through a multi-modal image fusion algorithm; performing target detection and feature extraction on the model, and screening an optimal capture point in combination with a visual attention mechanism; constructing a dynamic environment obstacle probability map, updating an obstacle state through time sequence visual tracking, and quantifying an interference weight; an initial grabbing path is planned based on an improved fast expansion random tree algorithm, path smoothness constraints and robot joint movement limit parameters are introduced, and path nodes are optimized through a Bezier curve; visual servo feedback and path deviation prediction are fused, path parameters are corrected in real time, and a continuous movement track is generated. The method effectively adapts to the dynamic environment, gives consideration to path safety, smoothness and mechanical arm motion characteristics, and remarkably improves the grabbing success rate and operation reliability.

The invention discloses a method and device for controlling grabbing of dynamic strips based on visual servo guidance and a test platform. According to the method for grabbing the dynamic strip-shaped object based on visual servo guidance control, the mechanical arm is guided and controlled to grab the dynamic strip-shaped object by adopting an image-based visual servo method, and real-time tracking of the dynamic pose of the strip-shaped object and immediate response of the action of the mechanical arm are realized through continuous sensing, processing and tight coupling of execution links; therefore, a more accurate, stable and efficient grabbing effect can be achieved in a highly dynamic and uncertain working environment; a test platform capable of simulating dynamic swing of a real strip-shaped object is also built, and a prediction control method and a visual servo control method are combined, so that effective verification and optimization of the high-precision and real-time grabbing process of the dynamic strip-shaped object are realized; the platform can be used for systematically researching the grabbing capability under various disturbance conditions in a laboratory environment, and a solid technical foundation is laid for automatic and intelligent operation in practical application.

The invention discloses a mobile robot accurate docking method based on edge calculation, and aims to solve the problems that the dynamic docking precision is reduced and the collision risk is increased due to micro-motion or drifting of a target station. According to the method, unified time reference alignment is carried out on data of a camera, a laser radar, an inertial measurement unit, an ultra-wideband range finder, a station encoder and a programmable logic controller at an edge node, and a three-dimensional special Euclidean group equivariant multi-source fusion network is used for outputting relative pose estimation and covariance; the estimation in the time window is further used as a condition to be input into a conditional diffusion short-time prediction model to obtain a time-varying mean value and a time-varying covariance, an anisotropic probability tube is constructed, and prediction-measurement joint correction is carried out based on a score function; scenarized opportunity constraints are constructed under correction probability tube constraints, a tubular nonlinear model is adopted to predict, control and solve a reference trajectory, an actuator command is generated in combination with depth visual servo and compliance control, and the technical effects of high precision, robustness and safe docking under the station dynamic disturbance condition are achieved.

The invention discloses a mechanical arm anti-interference control method and system based on visual servo, and relates to the technical field of mechanical arm anti-interference control, and the method comprises the steps: collecting an initial image through an industrial camera during the operation of a mechanical arm, and carrying out the preprocessing; selecting a pixel point from the central area of the initial image as a starting point by using a random walk algorithm, generating a step length in a random direction, calculating the next coordinate of the starting point, cutting, obtaining a walk path set, carrying out pixel value disturbance, generating a pseudo-abnormal image, and carrying out two-dimensional position coding based on the pseudo-abnormal image to obtain an embedded sequence vector; based on the embedded sequence vector, using a repair model to obtain a repair image, and generating a reconstructed image in combination with a deconvolution operation; by introducing the random walk algorithm, the multi-head self-attention mechanism, the Gaussian pyramid and the sparse dictionary, the perceptual robustness, the task execution stability and the generalization ability of the mechanical arm in the disturbed environment are remarkably improved.

The invention provides a lift wing unmanned aerial vehicle high maneuvering target interception method based on visual servo, which is used for using a lift wing unmanned aerial vehicle to cope with an invasion target moving at a high speed and using a lift wing platform. Lift wing modeling based on a rotation matrix and high-precision and high-dynamic autonomous interception control based on visual servo are introduced to serve as key technologies for supporting interception of the lift wing unmanned aerial vehicle. The effectiveness of the algorithm under airborne perception and control is verified, and the feasibility of the algorithm in practical application is consolidated. The contributions jointly promote the development of an unmanned aerial vehicle autonomous interception technology, and an important tool and method are provided for effectively coping with long-distance high-speed target intrusion threats. In addition, an IMU (Inertial Measurement Unit) measurement model and an image delay measurement model continue to be used, and a delay state estimation problem is researched.

The invention relates to the technical field of intelligent control, and discloses an intelligent control method and system for an electromechanical module intelligent installation robot. The method comprises the following steps: fusing multi-sensor data through a Kalman filtering algorithm to generate a standardized space state vector, carrying out three-dimensional path planning on a BIM model based on the vector to obtain a path instruction set, and carrying out precise positioning correction on the path instruction set by adopting a visual servo algorithm to obtain a corrected motion instruction. A synchronous control instruction is generated through synchronous coordination processing of multiple actuators, and finally safe obstacle avoidance evaluation processing is performed in combination with laser radar scanning data to obtain a safe execution instruction, so that intelligent precision installation of the electromechanical module is realized. The technical problems of inaccurate multi-sensor data fusion, lack of three-dimensional intelligence in path planning, insufficient visual servo precision, difficulty in synchronous coordination of multiple actuators and lack of predictability in safe obstacle avoidance in intelligent installation of the electromechanical module are solved.

The invention belongs to the technical field of intelligent and robot operation, and discloses a collaborative robot closed-loop operation method based on scene perception and thinking chain reasoning, which comprises the following steps of: firstly, generating structured semantic representation containing confidence by utilizing a visual language model of LoRA efficient fine tuning and fusing multi-view data, and providing semantic understanding for a robot; secondly, utilizing a three-dimensional diffusion model to generate explicit geometric priori under the condition of no real CAD, and correspondingly completing PnP initial pose estimation in combination with 2D-3D; then, carrying out pose optimization by adopting a tracking-refining strategy, and triggering reinitialization based on geometric prior when a residual error exceeds a threshold value; and finally, decomposing a global task by utilizing a thinking chain reasoning module, and realizing'perception-reasoning-execution 'closed-loop control based on visual servo in cooperation with an execution module. The method realizes effective combination of large model reasoning and physical execution, has strong robustness under dynamic disturbance, and is suitable for intelligent manufacturing and man-machine cooperation scenes.

The invention belongs to the technical field of robot application, and particularly relates to an acupoint recognition and mechanical arm control method for assisting expectoration, which uses a lightweight Inverted Residual convolution module containing depth separable convolution and residual connection to carry out high-precision recognition on 13 acupoints on the back, and on the basis of RGB three-channel image input, the acupoint recognition and mechanical arm control method for assisting expectoration is realized. A spine central axis channel is added, and a user-defined loss function including MSE loss, spine symmetry constraint and acupoint position constraint is designed to optimize the deep learning model, so that the accuracy and symmetry of acupoint recognition are further improved; after identification is completed, the system combines an improved near-end strategy optimization reinforcement learning algorithm and a visual servo technology to design a self-defined reward function which comprehensively considers distance reward, angle reward, cross punishment and target completion reward. And a voice coil motor of an end effector of the mechanical arm is guided to approach the target acupuncture point and align with the normal of the back of the human body, so that accurate control and dynamic motion planning of the voice coil motor are realized, and auxiliary sputum excretion actions are effectively executed.

The invention relates to the field of industrial robot visual servo control, in particular to a visual servo grabbing dynamic compensation system. The system comprises an image acquisition module, a motion analysis module, a local modeling module, a prediction module and a compensation execution module. A target image sequence is continuously captured, a motion track is extracted, a non-parametric local motion model is established based on recent data, the position of a target at the future grabbing moment is predicted, and a compensation signal is generated according to a prediction result to adjust the grabbing track of a robot. According to the method, the motion model is constructed by adopting a local weighted regression method, accurate description and rapid adaptation of various motion modes are realized through adaptive bandwidth adjustment and an online updating mechanism, and the grabbing success rate of a moving target is effectively improved.

The invention discloses an autonomous sewing system and method based on multi-modal sensing and self-adaptive control. The autonomous sewing system comprises a robot module, a sewing machine module, a visual sensing module and an intelligent control platform. According to the cloth pose registration module, high-precision registration of a flexible target is realized through a two-stage registration process, a local area weighting ICP and a non-rigid cloth surface registration method; the intelligent control module adopts a visual servo dynamic path tracking control method, dynamically corrects the posture of the robot, improves the path tracking precision and the sewing thread consistency, and ensures machine-machine collaborative synchronization and uniform sewing thread uniformity by constructing a speed, track and tension three-factor coupling synchronous control method. By adopting an error adjustment and tension stabilization mechanism based on self-adaptive control, the robustness of the system in a complex sewing state is improved; and through the data analysis module and the exception handling module, full-process monitoring, quality evaluation and exception grading response are realized, and the stability and reliability of the system are improved.

The invention relates to the technical field of robots, and provides an image-based simplified robot visual servo control method, in the aspects of image acquisition and illumination compensation, adaptive multi-scale histogram equalization is combined with illumination simulation compensation based on a generative adversarial network (GAN), so that the problems of illumination diversity and complexity are effectively solved, image distortion is avoided, and the visual servo control precision of a robot is improved. The image feature identification degree is greatly improved in a complex illumination environment, a robot can obtain clear visual information, a multi-modal feature fusion optimization part and attention mechanism fusion automatically adjust different modal feature weights according to a scene, a cross-modal feature interaction network breaks the independence of each modal feature, and the multi-modal feature fusion optimization part and the attention mechanism fusion optimization part are integrated. The target recognition accuracy and robustness are enhanced through cooperation of the two methods, the feature fusion effect is improved, and for the shielding problem, the robot can optimize the shielding processing strategy according to the environment state and the decision result through the shielding decision based on reinforcement learning, and the adaptability is enhanced.

The invention relates to a robust adaptive variable admittance control method based on different intentions in a feature space. The method comprises the following steps: determining a mapping relation between an actual contact force and a virtual contact force by adopting a mechanical arm kinematics and dynamics model; determining an adaptive stiffness coefficient based on the actual contact force change, and determining a robust adaptive virtual damping coefficient based on direct intention based on the virtual motion speed of the feature point; adjusting the robust adaptive virtual damping coefficient based on the expected contact force of the adjustment robot, and determining an admittance control virtual damping coefficient based on direct and indirect intentions; based on the mapping relation, the adaptive stiffness coefficient and the admittance control virtual damping coefficient, determining robust adaptive variable admittance control based on human intention in the feature space; and a robot image visual servo control law is determined based on the real-time image feature points and the expected feature points, the real-time joint speed of the mechanical arm is obtained, and the mechanical arm is controlled to move.

The present invention discloses a robot dynamic grasping method based on a sliding window self-attention mechanism. The method first installs a camera and calculates the transformation matrix between the camera and the manipulator coordinate system based on the hand-eye calibration technology; uses affine transformation to process the static data set to generate a grasping dynamic data set with speed information; then constructs a dynamic grasping network model based on the sliding window multi-head self-attention mechanism, and designs a loss function optimization model that simultaneously considers the object's motion speed and grasping posture; finally, deploys the trained model into a robot grasping environment, designs a position-based visual servo grasping control scheme, and realizes the follow-up movement and grasping action of the manipulator end effector to the grasped object. The robot dynamic grasping method of the present invention can quickly and accurately predict the grasping posture and motion speed of an object based on a set of continuous single-channel depth images collected by the camera, solving the grasping problem of general objects in complex dynamic environments.

The invention discloses an unmanned aerial vehicle relay tracking system and method for cloud side-end cooperation in an urban scene, and belongs to the technical field of intelligent traffic and unmanned aerial vehicle cooperative control. In order to solve the problems of insufficient endurance and limited sight distance of existing single-machine tracking in an urban large-range scene, a physically distributed and logically unified cloud-side-end three-level architecture is constructed, wherein a cloud end is responsible for natural language instruction analysis and global road network monitoring; the edge end serves as a regional scheduling hub and is responsible for operating a high-precision detection model, extracting a target Re-ID feature fingerprint and calculating a cross-machine relay time window; and the terminal unmanned aerial vehicle only operates a lightweight tracking model and a visual servo control law. According to the method, the urban area is divided into a plurality of honeycomb grids, seamless locking and relay of a target among unmanned aerial vehicles in different grid responsibility areas are realized by using a cross-view feature relay protocol and a collaborative finite-state machine, and the engineering problem of long-time-sequence and large-range vehicle tracking is effectively solved.

The invention belongs to the technical field of robot control, and discloses a visual servo intelligent robust control method for a mobile mechanical arm for a complex operation task, which comprises the following steps of: 1, acquiring a working scene image, and calculating a space coordinate of a target feature point in real time through a projection transformation model; step 2, dynamically inhibiting visual measurement noise by adopting adaptive Kalman filtering, generating a feedforward compensation signal through an integral sliding mode observer, and decoupling chassis slippage disturbance; step 3, inputting the pose signal into a radial RBF neural network, designing an RBF gain scheduler, and adjusting an output proportion-integral gain in real time to suppress external time-varying disturbance; 4, adopting a hybrid visual servo mode switching mechanism, and generating a mobile platform control instruction through an integral sliding mode surface in a position servo mode; and 5, designing a three-order composite controller to drive the mechanical arm so as to realize accurate grabbing. According to the method, the grabbing deviation caused by kinematics uncertainty of the mobile platform and dynamic environment disturbance is eliminated.

A method and system for robotic skill learning using visual pose estimation. A robot arm performing a task, such as an assembly or workpiece positioning operation, has a camera mounted thereon. The camera provides training images of an operation scene from a variety of positions and under a variety of lighting conditions. For each image, a relative pose of a tool center point with respect to a target pose is recorded. The images are used in a supervised learning process to train a neural network to minimize a difference between an inferred pose and the relative pose. Once trained, the neural network is used to compute a relative target position which is used in visual servoing control of the robot. The robot may also employ a force controller for final positioning or installation of a workpiece once contact is made with a mating piece.

The invention discloses a rope-driven flexible mechanical arm target tracking method and system based on visual servo, and the method comprises the steps: carrying out the real-time tracking of a target through a mixed visual module, and continuously outputting the bounding box information and tracking confidence score of the target; according to the method, visual errors are used as guidance, Jacobi is driven by learned data to serve as an accurate conversion model, and a tracking confidence score is introduced to dynamically adjust the response degree of control, so that an accurate and smooth motor driving instruction is solved. The system comprises a mixed target tracking module, a kinematics and jacobian identification module and a self-adaptive control module. By using the method, the tracking precision of the mechanical arm is improved by utilizing a vision and control cooperation mechanism. The method can be widely applied to the field of mechanical control.