232 results about "Visual servoing" patented technology

A system and method for providing intuitive, visual based remote control is disclosed. The system can comprise one or more cameras disposed on a remote vehicle. A visual servoing algorithm can be used to interpret the images from the one or more cameras to enable the user to provide visual based inputs. The visual servoing algorithm can then translate that commanded motion into the desired motion at the vehicle level. The system can provide correct output regardless of the relative position between the user and the vehicle and does not require any previous knowledge of the target location or vehicle kinematics.

The present invention provides methods and devices for the knowledge-based discovery and optimization of differences between cell types. In particular, the present invention provides visual servoing optical microscopy, as well as analysis methods. The present invention provides means for the close monitoring of hundreds of individual, living cells over time; quantification of dynamic physiological responses in multiple channels; real-time digital image segmentation and analysis; intelligent, repetitive computer-applied cell stress and cell stimulation; and the ability to return to the same field of cells for long-term studies and observation. The present invention further provides means to optimize culture conditions for specific subpopulations of cells.

The invention discloses a robot vision servo control method based on the image mixing moment. Firstly, construction of mixing moment features in one-to-one correspondence with space attitudes after target object imaging under the robot expected pose is given; and then a target object image is obtained under any attitude, the current mixing moment feature information value is calculated, the deviation of the mixing moment feature value is calculated according to information of an expected image and information of the current image, if the deviation is smaller than the preset threshold value, itis shown that the robot achieves the expected pose, if the deviation is not smaller than the threshold value, an image jacobian matrix relevant to the mixing moment features is deducted, a vision servo robot is used so that the robot can move towards the expected pose till the feather deviation is smaller than the preset threshold value, and the control process is over. By means of the robot vision servo control method, the image field mixing moment feature corresponding to the space movement track of the robot is introduced in to serve as the control input, vision servo control of a eye-in-hand robot system under the working space model unknown circumstance is completed, and the method can be widely applied to robot intelligent control based on machine vision.

The invention relates to an intelligent control method of visual monitoring and visual servo of tumor radiation therapy. The method comprises: (1) according to multi-mode medical images and multi-vision optical images collected by a radiation-oncology-plan-stage simulation and positioning medical imaging system, a radiation-oncology-positioning-stage treatment room internal optical imaging system, and a radiation-oncology-plan-execution-stage onboard medical imaging system and the like, intelligent analysis processing is carried out to monitor, identify and localize a tumor radiotherapy target and an organ at risk, radiotherapy beds with patients, radiotherapy equipment, and the auxiliary treatment staff and objects in a treatment room during the whole radiotherapy process; and (2), on the basis of optical images and/or medical images collected at different stages of the tumor radiotherapy process as well as the quality guaranteeing requirement of the clinical radiotherapy process, a multi-stage multi-vision servo intelligent control system (algorithm) is used for carrying out intelligent, automatic, high-precision, high-robust visual servo control on the radiotherapy beds, radiotherapy equipment, the onboard medical image system, and the optical imaging system in the treatment room during the tumor radiotherapy process.

A robot control system includes a processing unit which performs visual servoing based on a reference image and a picked-up image, a robot control unit which controls a robot based on a control signal, and a storage unit which stores the reference image and a marker. The storage unit stores, as the reference image, a reference image with marker in which the marker is set in an area of a workpiece or a hand of the robot. The processing unit generates, based on the picked-up image, a picked-up image with marker in which the marker is set in an area of the workpiece or the hand of the robot, performs visual servoing based on the reference image with marker and the picked-up image with marker, generates the control signal, and outputs the control signal to the robot control unit.

The invention discloses a container corrugated plate welding robot and a visual servo control system thereof. The visual servo control system comprises an industrial personal computer, a visual sampling mechanism and a motion control mechanism, wherein the visual sampling mechanism comprises a surface laser generator, a network camera, and an optical filter arranged in front of the network camera; the network camera is connected with the industrial personnel computer in a network communication manner; the motion control mechanism comprises a control card connected with the industrial personnel computer, five servo drivers in two-way communication connection with the control card, five servo motors correspondingly driven by the five servo drivers, and limit/zero-point sensors connected with the control card; the five servo motors respectively and correspondingly drive the motions of a welding gun and the visual sampling mechanism; the limit/zero-point sensors are respectively arranged in five motion directions controlled by the five servo motors; and the control card is further connection with the network camera. The visual servo control system controls based on visual servo, can accurately perform the tracking welding of welding lines, and is high in welding efficiency and excellent in welding quality.

The invention discloses a robot dynamic capture method and system. The method comprises the steps that S1, a collected image transmitted from an industrial camera is received in real time, and the obtained image is preprocessed; S2, a template tracking algorithm based on multiple clues is utilized to determine a foreground target region; S3, a template matching algorithm based on an edge gradientis adopted to precisely recognize a foreground target; S4, a random sampling consensus algorithm is adopted to perform modeling on motion information of an object in a view field; and S5, the information is fed back to a robot in real time for dynamic capture in combination with camera calibration and hand-eye calibration parameters. According to the robot dynamic capture method and system, by theadoption of a target tracking algorithm based on multiple clues, the motion target foreground region can be extracted efficiently and accurately; and under the application scene of robot sorting, therunning speed of a visual motion target tracking algorithm can be effectively increased, high-frame-rate real-time recognition is realized, and therefore the precision of visual servo is improved.

The invention discloses a hybrid vision servo system and method combining eye-to-hand and eye-in-hand structures. According to the hybrid vision servo system and method, information of a to-be-identified target is acquired through an eye-to-hand camera and an eye-in-hand camera; position calculation is performed through a computer; and a driver is controlled to drive a mechanical arm to drive the eye-in-hand camera for precisely positioning the to-be-identified target. Compared with traditional vision servo based only on position and vision servo based on images, the hybrid vision servo system and method have higher positioning precision and flexibility; the characteristics that an overall vision field is large in range and a local vision field is high in precision are realized; the cost is low; the applicability is high; the application range is wide; great convenience can be brought to application in scientific research, target searching and positioning and the like; and the hybrid vision servo system and method have very important research value and practical significance.

A method for simultaneous performing visual servo and adaptive depth identification through a robot belongs to the computer vision and mobile robot technology field. The method comprises: obtaining an open loop kinematical equation with stable error according to a robot pose position polar coordinate representation method; and designing an adaptive updating rule capable of identifying depth information according to a concurrence learning strategy, and constructing a vision stabilization control rule of a mobile robot. The parameter adaptive updating rule designed by the invention can perform learning at the initial stage of the robot performing stabilization motion and perform online identification of the depth information in the robot motion process. The method for simultaneous performing visual servo and adaptive depth identification through the robot can prove the simultaneous convergence of the pose posture errors and the depth identification errors according to a Lyapunov method and a LaSalle invariance principle. The method for simultaneous performing visual servo and adaptive depth identification through the robot can accurately and reliability identify the depth information while the mobile robot completes the vision stabilization control so as to prove the simultaneous convergence of the controller and the identification module.

The invention relates to a screw hole positioning and screw locking-and-dismounting device based on visual servo and provides a point-to-face screw or screw hole positioning method. The screw hole positioning and screw locking-and-dismounting device is used for positioning a screw hole and locking and dismounting a screw and comprises a visual measuring module and a control executing module whichare in data transmission and communication with each other through a network. The visual measuring module comprises a binocular camera, a coarse positioning camera and a laser dotting device, and thecontrol executing module comprises a mechanical arm unit and a computer. The mechanical arm unit comprises a large-torque tightening gun, a mechanical arm small arm body, a mechanical arm large arm body, a mechanical arm control box, a base and a screw sleeve. The large-torque tightening gun, the screw sleeve, the binocular camera and the laser dotting device are located at the tail end of the mechanical arm unit. According to the screw hole positioning and screw locking-and-dismounting device based on visual servo, the positioning accuracy of the screw can be improved to a large extent, a mechanical arm is utilized as a moving carrier, thus the work range is increased, and screw locking and dismounting can also be conducted in the environment which workers inconveniently go to.

The invention discloses a visual feedback platform improving the virtual reality immersion degree. The visual feedback platform is characterized by comprising a robot, a visual servo tracking control unit and an immersion type visual feedback unit and forms a closed presence sense system with users; the visual servo tracking control unit is used for tracking movement behaviors of the users and carrying out environment modeling; the immersion type visual feedback unit receives the information of the movement behaviors of the users and environment modeling and then mixes the information with limb models of the robot into a 3D virtual reality interface, and the users wear virtual reality equipment to achieve man-machine interaction. The visual feedback platform has the advantages that the immersion degree of man-machine interaction is high, the hardware investment is small, the system is simple and the integration level is high, thereby being suitable for constructing man-machine interaction systems of robots of different functions.

The invention relates to a mechanical arm dynamic fuzzy approximator based on a visual servo system. The visual servo mechanical arm system is composed of a visual servo controller, a visual module, a motion control module, a driving module, a mechanical arm module, a speed and position collecting module and a detecting module; the mechanical arm dynamic fuzzy approximator in the visual servo controller is used for approximating the unknown mechanical arm dynamic state which is subjected to random disturbance; the approximator comprises a fuzzy approximator body and a self-adaption module, the self-adaption module regulates parameters in the fuzzy approximator body on line to enable the error between the output of the fuzzy approximator body and the mechanical arm system to be approximated to always keep within the bounded errors, and the error tends to zero along with time; on the condition that the mechanical arm dynamic characteristics are unknown and are subjected to random disturbance, the stability of the system is guaranteed, and the trajectory tracking precision on an image plane is improved.

The invention provides a UAV and a visual servo control method and device thereof. The method comprises that a ground target and an expected target position corresponding to the ground target are determined in a flight spatial domain image, a three-dimensional (3D) flight trajectory to reach the expected target position is calculated according to a path planning algorithm and different constrainedconditions, the 3D flight trajectory is projected into an image coordinate system to obtain a two-dimensional (2D) flight trajectory, and navigation and visual servo control care carried out on the UAV based on the 2D flight trajectory. According to the method, device and UAV, the different constraints are integrated in path planning, navigation and visual servo control care carried out on the UAV based on the 2D flight trajectory, an obstacle can be avoided, and image characteristic information collected by a camera is directly used for feedback and servo control, it is not required to estimate the 3D attitude of the UAV, the visual servo control method is less complex, and the flight safety of the UAV is improved.

The invention relates to a robot trajectory tracking control method based on visual guidance. The method comprises the following steps: establishing a robot visual servo control system; establishing abinocular vision unified measurement field; carrying out observing by using a binocular vision device to obtain a pose transformation relationship between an end effector coordinate system and a measurement coordinate system, and converting the pose transformation relationship to a robot base coordinate system through the binocular vision measurement field; carrying out smooth estimation on the observed pose of an end effector by utilizing a Kalman filter; calculating the pose error of the end effector; and designing a visual servo controller based on fuzzy PID, processing the pose error to obtain an expected pose at the next moment, and sending the expected pose to a robot system to control the end effector to move. The technical scheme is oriented to the field of flexible machining of aerospace large parts and the application requirements of robot high-precision machining equipment, the pose of the end effector is sensed in real time through a visual sensor, so that a closed-loop feedback system is formed, and the trajectory motion precision of a six-degree-of-freedom series robot is greatly improved.

The invention discloses a self-adaptive dead zone inverse model generating device of a visual servo mechanical arm system. The visual servo mechanical arm system is composed of a visual servo controller, a visual module, a motion control module, a driving module, a six-degree-of-freedom mechanical arm, a torque feedback module, a speed and position collecting module and a detecting module. The self-adaptive dead zone inverse model generating device of the visual servo controller is used for eliminating dead zone non-linear constraint and comprises a dead zone inverse model module, a self-adapting module and an operating module, wherein the dead zone inverse model module is used for structuring a smooth dead zone nonlinear inverse model; the self-adapting module adjusts estimated dead zone parameters through self-adaptive laws and transmits the parameters into the dead zone inverse model module to change the parameters of the inverse model; by means of the signal communication of the modules, a closed-loop control system can be formed inside the visual servo mechanical arm system. The self-adaptive dead zone inverse model generating device of the visual servo mechanical arm system can effectively eliminate the influence of the dead zone nonlinear restraint and achieve high image tracing precision.

The invention discloses a visual servo switching control method and system. The method comprises the following steps: determining actual coordinates and actual camera poses of image feature points with two-dimensional codes; if a distance between the actual coordinates of the image feature points and an image boundary is greater than a boundary threshold value, adopting the visual servo control method based on image features to calculate first joint angular velocity of a mechanical arm, and controlling a robot according to the first joint angular velocity; judging whether the distance betweenthe actual coordinates of the image feature points and the image boundary is equal to the boundary threshold value or not, distinguishing and calculating the joint angular velocity; and adopting the visual servo control method based on the image positions to calculate second joint angular velocity if a difference value of Lyapunov function values of adjacent twice axle angle errors is smaller thanor equal to 0, and operation time of the visual servo control based on the image features is greater than an operation time threshold value. The method and the system disclosed by the invention havethe advantages of being capable of strengthening stability of a robot control system, and improving the visual servo effect.

The invention relates to a calibration method suitable for a visual servo plug-pull operation, and belongs to the field of image identification. The problems that servo alignment control precision based on the visual servo is low, and the effect of the plug-pull operation controlled by an open loop is poor in a plug-pull stage are solved; the calibration method suitable for the visual servo plug-pull operation is provided, an image containing pins, a socket and a plug clamping device is collected by using a camera; a depth learning algorithm is utilized to obtain a center point of each pin inthe socket and the center point of each calibration board on the plug clamping device; then a socket middle point coordinate, a socket deflection angle, a plug clamping device middle point and a plugclamping device deflection angle are calculated; the point coordinate is transferred into a joint coordinate system at the tail end of a mechanical arm of a robot; the socket middle point coordinatesin a tail end coordinate system of the robot is obtained; a visual servo image feature error is calculated in the tail end coordinate system; and a visual servo feature is sent to a robot visual servoalgorithm, and the robot is controlled to move. According to the method, the operation precision in visual servo plug-pull operation is improved.

The invention discloses a vision-based pose stabilization control method of a moving trolley, which fully considers about a kinematics model and a dynamics model of a trolley and a camera model. The vision-based pose stabilization control method comprises the following steps of: respectively obtaining an initial image and an expected image at a starting pose position and an expected pose position through a camera, and obtaining an existing image in a movement process in real time; by utilizing an antipode geometric relation and a trilinear restrain relation among shot images, designing three independent ordered kinematics controllers based on Epipolar geometry and 1D trifocal tensor by utilizing a three-step conversion control policy; finally designing a dynamic conversion control rule by taking outputs of the kinematics controllers as the inputs of the kinematics controllers by utilizing an retrieval method so that the trolley quickly and stably reaches an expected pose along a shortest path. The invention solves problems in the traditional vision servo method that the dynamics characteristic of the trolley is not considered during pose stability control and slow servo speed is slow, and the vision-based pose stabilization control method is practical and can enable the trolley to quickly and stably reach the expected pose.

A remotely operated orthopedic surgical robot system for performing a fracture reduction operation using a visual-servo control method is provided. The system includes the surgical image acquisition equipment, the fracture reduction robot and the remote operation workstation. The fracture reduction robot has a plurality of types. The remote operation workstation includes a graphical user interface for doctors to examine the fracture reduction path planning result made by an artificial intelligence algorithm and to manually perform the path planning The remote operation workstation calculates the robot control quantity using the visual servo control method according to the path planning result and sends it to the robot.

The invention discloses a mechanical arm visual servo control method based on iterative duration variation. The method includes the steps that 1, a series of images are acquired in a demonstration mode; 2, the relation between the current image and the tracking image is acquired to define image features which express the movement condition of a mechanical arm; 3, a mutual relation model of a visual control system is established based on a mechanical arm kinematics model and a camera model, and an iterative feedforward and feedback control scheme is adopted; 4, if the phenomenon that a target object is beyond a visual range in the operation process of the mechanical arm, current iteration is ended. According to the method, it is not required that the target is continuously visual in the movement process, and meanwhile it is guaranteed that the mechanical arm can track the target image accurately to some degree.

The invention discloses a method for visual servo controlling. The method comprises the steps that a training sample is obtained, a preset fuzzy neural network is trained by using the training sampleto obtain a visual servo controlling model; input expecting image information is received; the expecting image information is input to the visual servo controlling model, and current image informationof a target work piece is collected through an image collecting device; the difference of the current image information and the expecting image information is converted into a controlling signal of amanipulator, and the manipulator is controlled to execute the controlling signal to perform the next movement. According to the method, Jacobian matrix inversion is omitted to greatly improve the speed of visual servo controlling, complicated marking is not needed, the precision of visual servo controlling is effectively improved, and the motion controlling of a robot on automatically grabbing and transporting casting pieces is achieved. The invention further provides a visual servo controlling system, equipment and a computer storage medium which can achieve the above advantages.

The invention provides a redundancy-degree-of-freedom mechanical arm visual servo obstacle avoiding system. The system comprises a visual servo module, an information processing module, a mechanical arm control module, a driving module and a three-dimensional real-time simulating module; the driving module realizes communication between the mechanical arm control module and a redundancy mechanicalarm; the visual servo module acquires environment information of working space, and identifies target body and obstacle information; the information processing module builds three-dimensional solid models of obstacles according to the target body and obstacle information to obtain pose information of target bodies; and the mechanical arm control module converts targets in the working space to control instructions of joint space according to user operation instructions from the three-dimensional real-time simulating module, the three-dimensional solid models of the obstacles and the pose information of the target bodies, and sends the control instructions to the driving module to control mechanical arm terminals to move to the target bodies. The system integrates the advantages of visual servo and the objective function requirements of obstacle avoidance, can be verified offline, and can be connected with solid mechanical arms to achieve broad application prospect.

The invention discloses a human-computer cooperation system based on Kinect skeletal tracking and uncalibrated visual servo and belongs to the technical field of human-computer cooperation. The human-computer cooperation system comprises a robot, a human-computer interaction control module, an information collection module and an intelligent control module. The human-computer interaction module comprises a Kinect and a remote control unit. The Kinect collects the pose information of a human body skeleton through a Kinect camera and transmits the pose information to the remote control unit, and then the remote control unit transmits the pose information to the intelligent control module. After the information collection module collects the information of a robot specific work task scene on a workbench through the camera, the information is sent to the intelligent control module. After receiving and processing the pose information and the image information, the intelligent control module obtains a movement instruction of the robot and sends the movement instruction to the robot. According to the human-computer cooperation system based on Kinect skeletal tracking and uncalibrated visual servo, the untouched control of human body action to the robot is achieved through the Kinect skeletal tracking technology.