26 results about "Visual servoing system" patented technology

The invention discloses a visual servo method for a transformer substation drainage wire disconnecting and connecting robot, and the method comprises the following steps: building a mapping relation between a mechanical arm tail end camera and a mechanical arm kinematic model, and building an image-based visual servo system IBVS; acquiring a drainage wire clamp image in real time, and extracting image feature points of a drainage wire clamp bolt by using the recognition model; the model prediction controller carries out rolling optimization based on the deviation between the extracted current image features and the expected image features, estimates and compensates external disturbance and unmodeled dynamics borne by the system in real time, and calculates an optimal, smooth and safe mechanical arm motion instruction; and the mechanical arm movement instruction is sent to a mechanical arm movement control unit to drive the mechanical arm to move. According to the method, the pose control of the drainage wire disconnecting robot can be realized, the error convergence in the visual servo process is rapid, the steady-state error is small, the dynamic response is timely, and the requirements of the drainage wire disconnecting operation on the precision, the stability and the real-time performance are fully met.

The invention discloses a six-degree-of-freedom mechanical arm disturbance compensation control method based on an iterative learning observer, and the method comprises the steps: taking a visual servo system as a research object, enabling the system to comprise a mechanical arm, a sensing part and a target object, and building a state space model based on a depth-independent Jacobian matrix model with joint speed disturbance; constructing an auxiliary state system and an error transfer equation based on joint sensor information, and designing an iterative learning observer to estimate joint speed disturbance; the disturbance estimation value is embedded into a visual model prediction controller, an optimal control signal is generated, joint speed input is corrected through a feed-forward compensation mechanism, and active disturbance suppression is achieved. According to the method, the tracking response speed of sudden disturbance can be improved, the system stability is maintained under the condition that interference exists in the joint speed, and therefore the visual servo task of tracking the expected trajectory is achieved.

The invention discloses a cooperative online and offline control method for an unmanned aerial vehicle power transmission line broken strand repairing robot, and aims to realize efficient and safe maintenance of a power transmission line. Based on Newton-Euler and Euler-Lagrange equations, establishing an eight-degree-of-freedom nonlinear dynamic model of the unmanned aerial vehicle-broken strand repair robot; and designing and optimizing an extended state observer and a nonlinear error feedback control law to suppress swinging. An airborne visual servo system is adopted to capture the posture of the robot so as to correct the trajectory of the unmanned aerial vehicle, and a double-closed-loop control strategy based on a spherical swing mechanism is applied to realize stable tracking of an arc trajectory. Nonlinear load trajectory tracking control based on a six-dimensional force sensor is introduced, suspension point tension and the position of the unmanned aerial vehicle are fused to estimate the pose of the robot, self-adaption to different load weights is achieved through a load speed observer, and real-time data exchange between the unmanned aerial vehicle and the robot is supported through a low-delay and high-reliability communication protocol.

The invention discloses an underwater mechanical arm shake compensation control method based on sensor space-time heterogeneous matching. The method comprises the steps that firstly, an eye-in-hand camera at the tail end of a mechanical arm is used for estimating shaking of a mechanical arm base; meanwhile, an inertial measurement unit is used for directly measuring shaking of the mechanical arm base; fusing and optimally estimating two paths of shake measurement data by adopting a multi-sensor space-time heterogeneous matching algorithm based on Kalman filtering and a cross-correlation method; and finally, the joint control quantity of the mechanical arm is obtained through inverse kinematics calculation, so that rotation shaking of a mechanical arm base is effectively compensated, and the stability of the tail end posture of the mechanical arm is kept. According to the method, the problems that the control precision is damaged and the operation reliability is reduced due to shaking of the base in the operation process of a mobile platform mechanical arm, especially an underwater vehicle-mechanical arm system are solved, the method can be combined with a visual servo system or a teleoperation mechanical arm control system, and the operation reliability and practicability are improved.

This invention discloses a robust control method for robot visual servoing based on the theory of all-drive systems, belonging to the field of visual servoing for robotic arms. The steps are as follows: Taking the eye-to-hand robot visual servoing system as the research object, based on the efficient second-order minimization (EMS) method and combined with the virtual visual servoing method, the target depth and attitude parameters are estimated online to construct a high-precision image interaction matrix; utilizing the all-drive characteristics of robot joint dynamics, the image feature error is mapped to joint space error through the interaction matrix, establishing an open-loop error dynamic model of the visual servoing all-drive system; based on the theory of all-drive systems, it is transformed into a linear second-order system through nonlinear state feedback, and the state space equation is constructed; a sliding mode robust model reference tracking controller is designed. This method can maintain system stability under conditions of camera calibration error, dynamic parameter disturbance, and unmodeled dynamics coexisting, achieving high-precision, high-dynamic visual servoing tracking control.

The invention discloses a dynamic trajectory prediction and intelligent visual servo system based on deep learning, and particularly relates to the technical field of intelligent control. The method comprises the following steps: collecting image data of a target motion state in real time, extracting dominant state information and recessive state information of a target, performing dynamic trajectory prediction, and generating time sequence characteristics of a target motion trend and probability information of a potential shielding state; performing prediction stability analysis on the time sequence characteristics of the target motion trend to generate optimized trajectory prediction data; carrying out hidden state effect analysis on the probability information of the potential shielding state, and generating real-time evaluation information of target shielding; fusing the trajectory prediction data and the real-time evaluation information, and outputting a target trajectory prediction result; and the visual servo execution mechanism is controlled based on the target trajectory prediction result, so that stable tracking and prediction control of the incomplete observable target are realized, and the practicability of the visual servo system in a complex dynamic environment is improved.

The application provides a photoacoustic tomography system, method and computer program product. The system comprises a photoacoustic signal acquisition device and a controller; the controller is configured to acquire a structure image by using the photoacoustic signal acquisition device; the structure image is obtained by two-dimensional photoacoustic image reconstruction on photoacoustic information of a target region; image feature extraction and frequency domain processing are performed on the structure image to obtain a frequency domain feature vector; error data between an actual acquisition position and a target acquisition position of a photoacoustic probe of the photoacoustic signal acquisition device is calculated according to the frequency domain feature vector, and the pose of the photoacoustic probe is adjusted according to the error data to lock the target region. Through low-frequency coefficient selection and an adaptive adjustment strategy, the application automatically suppresses high-frequency noise, enhances the anti-interference ability of the image feature extraction process to a complex environment, and thus improves the overall robustness and reliability of the visual servo system.

The present disclosure relates to the technical field of machine vision, and proposes a fast selection and tracking method and system for dynamic visual servo target in complex background, which continuously captures screenshots and allows the operator to control the robot movement during target selection, thereby ensuring that the target is always in the picture and effectively reducing the risk of target loss. At the same time, the proposed automatic selection method of gradually expanding outward simplifies the selection operation process and to some extent avoids human error operation. The target selection tolerance is automatically adjusted according to the moving speed of the interactive tool to adapt to the dynamic changes of the target, further reducing the burden of the operator, so as to realize more accurate and reliable target selection. Through the combination of image texture analysis and deep learning technology, the present disclosure realizes efficient selection and real-time tracking of the target under limited time and computing resources, and provides an innovative solution for the application of dynamic visual servo system.

An unmanned-forklift automatic loading and unloading system for a van-type truck. The system comprises: a positioning map module, which is used for binding with a positioning map platform areas at which a plurality of trucks park; a task-issuing module, which is used by a driver to issue a task to a WMS by means of a pad system; a WMS, which issues, on the basis of platform numbers, goods pick-up and placement tasks corresponding to map points; an AGV, which reaches a designated platform by means of a positioning system based on a reflecting plate, scans the interior of a carriage and sends a scanning result to the WMS; and a storage location planning module, which receives the carriage scanning result, which is sent by the AGV, plans storage locations on the basis of the carriage space, and issues a goods pick-up task to the AGV. The AGV is navigated inside the van-type truck by using a visual servo system based on a solid-state laser radar, such that the AGV can realize precise positioning in a complex carriage environment. The navigation precision and stability of the AGV are improved, and the adaptability of the AGV in different van-type truck environments is also enhanced.

The invention provides a mechanical arm visual servo control method and system based on a self-learning disturbance observer, and relates to the technical field of robot control. Extracting an image feature vector; performing state estimation through extended Kalman filtering to obtain a filtered image feature state; carrying out online learning through a kernel least mean square algorithm to obtain a disturbance estimation value; generating a nominal control sequence and a nominal state trajectory based on the undisturbed ideal model; and generating a final control instruction according to the deviation and the disturbance estimation value and driving the robotic arm to move. According to the method, the extended Kalman filtering and the kernel minimum mean square algorithm are combined to construct the composite disturbance observer, adaptive online learning and high-precision estimation of unknown disturbance are realized, and a double-layer robust control structure is formed in combination with tubular model predictive control. And the control precision and robust stability of the visual servo system of the mechanical arm in a complex disturbance environment are remarkably improved.

This invention relates to the fields of robotics and information technology, and discloses a visual servoing method, apparatus, device, and medium based on a diffusion model. The method includes: forming a training sample by combining a preset tool image, a preset teaching image corresponding to the preset tool image, and a real desired image corresponding to the preset tool image; training a diffusion model based on the training sample; stopping the training of the diffusion model when the total loss value is less than a preset value, thus obtaining a trained diffusion model; fusing the feature vectors of the current tool image and the current teaching image to obtain a second fused vector; processing the second fused vector through a decoder in the trained diffusion model to obtain the current desired image; transmitting the current desired image to the robot's visual servoing system, and controlling the visual servoing system to adjust the position and orientation of the current tool according to the current desired image. This invention can provide the current desired image to the robot's visual servoing system.

The invention discloses a self-adaptive trajectory prediction compound control method and device for a visual servo system, and belongs to the field of target tracking control of a photoelectric tracking system. Comprising the following steps: constructing a transfer function of a photoelectric tracking system based on a kinetic equation of a reflector and a kinetic equation of a voice coil motor in combination with a motion relation of the fast reflector in a deflection process, and obtaining a mathematical model of the photoelectric tracking system through the transfer function; identifying system input delay, designing an optimal position controller based on a photoelectric tracking system mathematical model, and constructing a single feedback controller; designing an adaptive trajectory prediction feedforward controller based on the established single feedback controller and the photoelectric tracking system mathematical model; and constructing adaptive trajectory prediction compound control based on a single feedback controller and an adaptive trajectory prediction feedforward controller. According to the invention, composite control is constructed based on position feedback and feedforward, and the tracking performance of the photoelectric tracking system is effectively improved.

The invention belongs to the technical field of ball picking machines, and discloses an automatic table tennis ball picking machine which comprises a base, guide rails are fixedly mounted at the top of the base, a top tray is fixedly mounted at the tops of the three guide rails, a detachable ball collecting box is fixedly mounted at the top of the top tray, and an elastic shrinkage hole is formed in the center of the ball collecting box. A parallel rod type displacement table structure is adopted, three sets of linear guide rails and a stepping motor cooperatively drive a displacement disc, accurate positioning in the front-back, left-right and up-down three-dimensional space is achieved, and millimeter-level control precision can be achieved in cooperation with a visual servo system; a guide head is arranged at the lower end of the ball picker, so that the target ball can still be efficiently collected when positioning deviation exists, and displacement interference caused by touching an adjacent ball is avoided; a retaining mechanism driven by a torsion spring is arranged inside, the one-way locking function is achieved through elastic overturning of a blocking piece, and reliable picking of the balls can be completed only through vertical lifting action.

The application discloses a kind of underwater man-machine shared teleoperation control methods based on bilateral trust mechanism.Method includes: master end force feedback device teleoperation control underwater mechanical arm moves, so that underwater mechanical arm end underwater hand-eye camera identifies underwater target, first, the master end evaluation model of master end force feedback device is established to obtain master end trust factor;Establish the visual evaluation model of underwater hand-eye camera, and finally obtain visual trust factor by density-based visual measurement outlier rejection method;Master end trust factor, visual trust factor and each control instruction are collectively processed by man-machine shared teleoperation method based on fuzzy logic, and the desired control instruction of underwater mechanical arm is obtained to carry out man-machine shared teleoperation control.The application can effectively eliminate the outlier of visual identification, ensure the reliability of visual servo system, introduce visual auxiliary system, can further reduce the burden of operator, improve the task success rate and operation efficiency of teleoperation system.

The application provides a kind of mechanical arm visual servo control method and system based on self-learning disturbance observer, it is related to robot control technical field, the method comprises: obtaining real-time image data;Extract image feature vector;State estimation is carried out by extended Kalman filtering, and the filtered image feature state is obtained;Disturbance estimation value is obtained by kernel least mean square algorithm online learning;Based on the ideal model without disturbance, nominal control sequence and nominal state trajectory are generated;According to the deviation and disturbance estimation value, the final control command is generated and the robot arm is driven to move.The application combines extended Kalman filtering and kernel least mean square algorithm to construct a composite disturbance observer, realizes adaptive online learning and high-precision estimation of unknown disturbance, and forms a double-layer robust control structure combined with tube model predictive control, which significantly improves the control accuracy and robust stability of the mechanical arm visual servo system in complex disturbance environment.

The invention discloses an underwater man-machine sharing type teleoperation control method based on a bilateral trust mechanism. The method comprises the steps that main end force feedback equipment remotely operates and controls the underwater mechanical arm to move, so that when an underwater hand-eye camera at the tail end of the underwater mechanical arm recognizes an underwater target object, a main end evaluation model of the main end force feedback equipment is firstly established, and a main end trust factor is obtained; establishing a visual evaluation model of the underwater hand-eye camera, and finally obtaining a visual trust factor through a density-based visual measurement abnormal value elimination method; and the main end trust factor, the visual trust factor and all the control instructions are jointly processed through a man-machine sharing type teleoperation method based on fuzzy logic, and then an expected control instruction of the underwater mechanical arm is obtained so that man-machine sharing type teleoperation control can be carried out. Abnormal values of visual identification can be effectively eliminated, the reliability of a visual servo system is ensured, a visual auxiliary system is introduced, the burden of an operator can be further relieved, and the task success rate and the operation efficiency of a teleoperation system are improved.

The invention discloses a method for estimating translation external parameters of a visual servo system of a mobile robot, and relates to the technical field of computers. The method comprises the following steps of: converting translation external parameter estimation of a visual servo system of the mobile robot into a parameter estimation problem of a nonlinear system; representing the nonlinear system as a parameterized form; performing first-order low-pass filtering processing on the first signal, the second signal and the third signal to respectively obtain a first filtering signal, a second filtering signal and a third filtering signal; obtaining an auxiliary variable based on the first filtering signal, the second filtering signal and the third filtering signal; generating an auxiliary signal based on the auxiliary variable; and based on the auxiliary signal, dynamically adjusting the gain matrix through an updating rule, carrying out convergence in a fixed time to obtain an estimated value of the to-be-estimated parameter, and obtaining an estimated value of the translation external parameter according to the estimated value of the to-be-estimated parameter. The method can be used for estimating translation external parameters within fixed time.

The invention belongs to the technical field of textile mechanical arm trajectory tracking control, and discloses a textile mechanical arm visual servo trajectory tracking control method and system based on a fuzzy observer. In order to solve the problems that visual velocity information is difficult to obtain and a system model is uncertain, a fuzzy observer is designed to obtain a visual velocity estimated value, and a fuzzy logic system is utilized to approach unknown nonlinear dynamics. Besides, an instruction filtering technology is applied, the problem of calculation complexity of the visual servo system of the mechanical arm is solved, an error compensation mechanism is introduced to eliminate adverse effects caused by filtering errors, and the control effect of the visual servo system is improved; and meanwhile, the negative influence of the input dead zone on the system performance is also compensated. The method can effectively solve the problems that the visual speed information is difficult to obtain and the calculation is complex when the controller is designed, has a good control effect, and is suitable for a scene where the textile industry production robot has high control precision requirements.

The application discloses a kind of underwater mechanical arm shake compensation control methods based on sensor space-time heterogeneous matching.The method comprises: first, the eye in hand camera of mechanical arm tail end is used to realize the estimation of mechanical arm base shake;While using inertial measurement unit to directly measure the mechanical arm base shake;Then, the multi-sensor space-time heterogeneous matching algorithm based on Kalman filtering and cross-correlation method is used to realize the fusion and optimal estimation of two-way shake measurement data;Finally, the joint control amount of mechanical arm is obtained through inverse kinematics solution, so as to effectively compensate the rotational shake of mechanical arm base and maintain the stability of mechanical arm tail end posture.The method solves the problem that the control accuracy may be damaged and the operation reliability may be reduced due to base shake during the operation of mobile platform mechanical arm, especially underwater vehicle-mechanical arm system, and can be combined with visual servo system or remote operation mechanical arm control system to improve the reliability and practicability of operation.

The application relates to a motion compensation auxiliary optical coherence tomography system, method and working method, which can reduce motion artifacts, improve imaging quality, avoid image blurring caused by physiological motions such as respiration, and obtain a clear image accurately reflecting the real situation of a lesion. An optical distance sensor is used as a visual servo system of a robot, a 7-joint cooperative mechanical arm is fixed at the end of the OCT imaging probe and holds the OCT imaging probe, the optical distance sensor is integrated and installed at the end of the probe, the position of a laser point is recognized based on the principle of triangulation, the accurate distance from the probe to the sample is calculated, the position of the probe is controlled in real time, the axial motion of the sample is compensated, a communication channel based on TCP / IP is established between a host and a robot controller, and a motion compensation control loop is established. The system is simple in structure, convenient to operate, high in stability, and suitable for various clinical and scientific research application fields.

The application discloses a mechanical arm control method based on deep reinforcement learning, which comprises the following steps: a servo controller is designed, a mechanical arm visual servo system is built, an adaptive servo gain DDPG model is trained, a DDPG training environment is built, DDPG training parameters are set and DDPG model training is completed, expected images when the mechanical arm grabs a target object are set and acquired, and expected features are extracted, current target images are acquired, current features are extracted, an extended Kalman filtering method is used to estimate a depth value, the depth value is substituted into calculation of an image Jacobian matrix generalized inverse matrix, meanwhile, expected features and current features are operated to obtain image feature error e, according to a current state of the mechanical arm visual servo system, an adaptive servo gain algorithm based on deep deterministic policy gradient DDPG which is trained is used to determine a servo gain value, and the servo controller is used to perform servo control on the mechanical arm. The application has the characteristics of stronger stability, higher robustness and faster convergence speed.

The invention relates to the technical field of computer vision, in particular to a tank car maintenance equipment mechanical arm movement control method based on computer vision, and the method specifically comprises the steps: constructing a multi-scale visual interference sensitivity three-dimensional distribution model, collecting a real-time image sequence in a tank car, and calculating a visual feature consensus instability threat value; synchronously extracting real-time processing state data of an image processing node of the mechanical arm visual servo system, and uploading and storing the real-time processing state data to a mechanical arm visual supervision data cloud network; and the tracking stability of the mechanical arm visual servo system is analyzed and judged, and mechanical arm dynamic follow-up control strategies are output in a classified mode. The problems that in the prior art, a feature tracker is prone to continuous drifting or even complete failure in a key geometric area, and uncertainty of mechanical arm pose estimation cannot be effectively restrained are solved.

The application discloses a monocular hand-eye robot vision servo method for three-dimensional space dynamic depth estimation, comprising establishing a parameterized vision servo system model, constructing an estimation error driven adaptive law and executing position-based vision servo control. The application establishes a parameterized model of the robot-camera-target unification, estimates the depth changes caused by the target motion and the robot motion in the same framework, which enables the application to effectively deal with the more common and more complex working conditions that both the robot and the target are moving. The application solves the problem that the nonlinear time-varying depth is difficult to measure in the monocular camera vision servo without prior geometric knowledge of the observed object and under the condition that both the target and the robot are moving, and realizes the rapid convergence of the control error and the estimation error in the hand-eye robot system.

The present invention discloses a method for autonomous control of an underground pipeline robot based on visual servoing. First, a binocular camera is used to acquire three-dimensional information of a target object. Hand-eye calibration is then performed to unify the robot's coordinate system with the camera's coordinate system. PBVS is then used to control the robot's movement based on the target's three-dimensional information and the robot's three-dimensional information. Finally, an online supervised visual servoing system is used to correct the input in real time, enabling autonomous control of the robot and ensuring grasping accuracy. The proposed method for autonomous control of an underground pipeline robot based on visual servoing enables the robot to autonomously adjust itself during the grasping process, thereby accurately grasping the target object.

The application discloses a 6-DOF mechanical arm disturbance compensation control method based on an iterative learning observer, and steps are as follows: taking a visual servo system as a research object, the system comprises a mechanical arm, a sensing part and a target object, a state space model is constructed based on a depth-independent Jacobian matrix model with joint speed disturbance; an auxiliary state system and an error transmission equation are constructed based on joint sensor information, an iterative learning observer is designed to estimate joint speed disturbance; a disturbance estimation value is embedded into a visual model predictive controller to generate an optimal control signal, joint speed input is corrected through a feedforward compensation mechanism, and active interference suppression is realized. The application can improve the tracking response speed of sudden disturbance, maintain system stability in the case that joint speed exists interference, and thus realizes a visual servo task of tracking an expected track.

The invention discloses a monocular hand-eye robot visual servo method for three-dimensional space dynamic depth estimation. The monocular hand-eye robot visual servo method comprises the steps of establishing a parameterized visual servo system model, constructing an estimation error driving type self-adaptive law and executing visual servo control based on positions. According to the method, the unified parameterized model of the robot, the camera and the target is established, and the target motion and the depth change caused by the robot motion are estimated under the same frame, so that the method can effectively cope with the more general and more complex working condition that the robot and the target both move. According to the method, the problem that the nonlinear time-varying depth is difficult to measure under the conditions that the prior geometric knowledge of the observed object does not exist and both the target and the robot move in the visual servo of the monocular camera is solved, and the rapid convergence of the control error and the estimation error in the hand-eye robot system is realized.