436 results about "Teleoperation" patented technology

Systems and methods that allow robots to perform tasks for users are provided. A robot may comprise one or more robotic arms and / or a mobile base. The arms may be controlled by electric actuators and may have six or more degrees of freedom. The robot may have sensors which can be accessed remotely. Robots may have varying levels of autonomy, including, for example, full teleoperation (in which a human can have detailed control over the robot) or full autonomy (in which the robot can complete a task without any human intervention). Various entities can interact with individual robots or groups of robots over a network, locally, directly or in person, or any combination thereof. A management system can allow entities to control and / or monitor the robots.

The invention discloses a dexterous hand teleoperation method based on vision-touch fusion. The method comprises the following steps: 1) vision detection; the method comprises the following steps of (1) carrying out remote control, (2) carrying out redirection, (3) carrying out tactile grabbing, (4) carrying out vision-tactile interaction, (5) carrying out a communication process and (6) carrying out dexterous hand.The method can stably map human hand actions under the complex vision condition, safe and reliable grabbing control is achieved through tactile feedback, meanwhile, multiple dexterous hand configurations and cross-platform deployment are supported, and the precision, robustness and practicability of teleoperation can be improved.

The present invention relates to the field of robotic arm teleoperation technology, and specifically to a construction method and system for real-time teleoperation of a dual arm and hand robot based on vision guidance, which includes setting up a first experimental platform, acquiring pose information of the dual arm and hand robot and a homogeneous transformation matrix of an end joint and transforming a first coordinate system; setting up a second experimental platform; performing calibration by a plurality of cameras and transforming a second coordinate system; obtaining spatial transformation pose information of the end joint of the robotic arm under a tracking state, reading finger pose data, and performing real-time following. The real-time teleoperation system of the present disclosure includes a control system host, an operation glove, an optical locator, a positioning coordinate plate, a marker ball tool, and a dual arm and hand robot. The present disclosure introduces a method of identifying the marker ball tool by the optical locator into the teleoperation control of the dual arm and hand robot, which is more accurate and faster. Meanwhile, the method of remotely controlling the robotic arm by an operator wearing the operation gloves can provide better human-machine interaction function.

The invention discloses a real-time whole-body remote operation control system for a humanoid robot, and the system comprises a motion instruction generation subsystem which is located at an operation end and is used for receiving and analyzing input instructions of various modes, and carrying out the unified conversion and complementation of the input instructions into a whole-body joint motion instruction which is suitable for a skeleton structure of a target humanoid robot; the body motion control subsystem is located at the robot end and used for receiving the whole-body motion instruction, adaptively generating a whole-body motion coordination control signal capable of ensuring the stability of the robot in combination with the state of the robot and environment information, and driving and executing the whole-body motion coordination control signal; through cooperation of the two subsystems, the system provides a dual guarantee path for realizing whole-body teleoperation: a complete whole-body instruction can be generated based on the instruction generation subsystem, and whole-body motion can be adaptively generated by the body motion control system according to a received local instruction; and the robustness, the flexibility and the reliability of the system under different conditions are ensured.

The invention provides a binocular vision teleoperation robot system, a robot and a teleoperation method and device, and relates to the technical field of computers. The method comprises the following steps: receiving a binocular video stream obtained by carrying out image acquisition on an actual scene by a binocular vision robot, and rendering the binocular video stream to obtain a three-dimensional scene; displaying the three-dimensional scene to an operator of the binocular vision robot to capture motion data of the operator observing the three-dimensional scene; determining a target area from the actual scene according to the action data, and obtaining a visual control instruction based on the target area; the vision control instruction is transmitted to the binocular vision robot for teleoperation; wherein the visual control instruction is used for adjusting imaging parameters of the binocular vision robot for image acquisition of an actual scene, and the visual direction of the binocular vision robot can be rapidly and naturally kept consistent with the attention point of the operator.

The invention provides a digital twinning-combined dexterous hand teleoperation hand action migration method. The method comprises the following steps: acquiring an image sequence of a hand through a binocular camera; for each group of images in the image sequence, determining a three-dimensional coordinate sequence of the hand feature points; based on a pre-established digital twin skeleton model of the dexterous hand, establishing a mapping relation between the hand feature points and corresponding points in the digital twin skeleton model; converting the three-dimensional coordinate sequence of the hand feature point into a three-dimensional coordinate sequence of a corresponding mapping point in the digital twin skeleton model based on the mapping relation, taking the three-dimensional coordinate sequence as the position of the mapped hand feature point, and determining an optimal buckling angle and an optimal abduction angle; and based on the optimal buckling angle and the optimal abduction angle, a joint angle sequence is obtained, and corresponding joint actions in the dexterous hand are controlled to enable the dexterous hand to perform action migration according to hand actions in the image sequence, so that the smoothness and precision of the actions of the dexterous hand in the action migration process are improved.

The invention discloses a mechanical arm teleoperation control method and system based on incremental mapping, and the method comprises the steps: carrying out the noise reduction processing of the position and posture of a handle through adaptive low-pass filtering, and achieving the position and posture consistency mapping from a handle space to a mechanical arm working space after coordinate system axis rearrangement and similar transformation mapping; the system provides two kinds of control output of a pose tracking mode and a speed tracking mode, real-time inverse kinematics solution is carried out through a Jacobian matrix and a damping least square method in the speed mode, and resolution scaling and exponential moving average / spherical linear interpolation smooth filtering are carried out on a target pose in the pose mode. And then the handle motion instruction is mapped into a Cartesian instruction or a joint space instruction at the tail end of the mechanical arm, closed-loop execution is conducted through a real-time state feedback link, and the method can be used for scenes such as robot service, teaching, remote operation and data collection.

The invention provides a pipeline welding method and system fusing teleoperation teaching and visual recognition. The method comprises the steps that discrete key point positions in a pipeline welding seam path picked up by teleoperation equipment teaching are obtained; generating an initial welding track by using a track interpolation method based on the key point location; a guide point is generated based on three-dimensional contour data, collected when a line laser contour sensor scans along the initial welding track, of an actual pipeline welding seam; the deviation value between the guide point and a corresponding track point on the initial welding track is obtained, and the initial welding track is compensated and corrected according to the deviation value to generate an actual welding track; and the welding robot is controlled to execute multi-layer welding operation along the actual welding track. According to the method, the experience of people and the precision of laser vision are combined, the planning efficiency and the track precision of the welding path are remarkably improved, and the welding quality and the operation safety of the pipeline under the severe working condition are guaranteed.

The invention provides an acquisition-training-evaluation integrated method and system for a humanoid robot. The method comprises the following steps: acquiring operation data of an operator by using master-slave control teleoperation equipment, and optimizing the operation data; the mechanical arm is controlled to respond, and data collection is conducted in the movement process of the mechanical arm; processing and storing the acquired data; applying the processed required information to a reasoning training process of the robot to form a reasoning model; a reasoning training result is deployed to the robot, so that the robot can start to execute a test task; real data of the robot in the execution process are collected, and the motion effect of robot training is evaluated and judged; a robot training reasoning model which does not conform to an evaluation standard is optimized, a data acquisition process and a training reasoning process are purposefully optimized from an evaluation result, and a'training-adjusting-correcting 'whole-process data active discovery and utilization mechanism is formed, so that the construction of a high-quality and multi-modal universal intelligent data set with a body is accelerated, and the accuracy and the reliability of the system are improved. Dependence on real data acquisition is reduced, and robot perception, understanding, reasoning and general generalization capabilities are enhanced. The problems that in the robot training process, training data and evaluation indexes are separated and separated, so that an integrated closed loop cannot be achieved in the collection-training-evaluation process, the evaluation result cannot act on the robot training improvement process, and the virtuous circle that the evaluation result counteracts on robot training cannot be achieved are solved.

The invention discloses a humanoid robot teleoperation system and a control method thereof. The system comprises the steps that an exoskeleton unit collects joint angle information and arm length information of arms of an operator; the data glove collects finger joint angle information of an operator; the inertial measurement unit collects space attitude information of the head and the chest of an operator; the double-shaft control pedal is used for generating integral moving, lifting and rotating control signals of the humanoid robot on the basis of foot actions of an operator; and the central controller receives the arm joint angle information, the arm length information, the finger joint angle information, the head and chest space attitude information and moving, lifting and rotating control signals of the humanoid robot, obtains a whole-body cooperative motion control instruction of the humanoid robot through kinematics calculation, and sends the whole-body cooperative motion control instruction to a control end of the humanoid robot. The high-fidelity teleoperation of the whole-body motion of the humanoid robot is realized by fusing the collected whole-body motion data of the operator and combining the self-adaptive kinematics of the central controller for resolving.

A vehicle teleoperation system may comprise a single, integrated unit that is installed within a vehicle to enable remote driving operations. The vehicle teleoperation system may include a processing island, a connectivity island, a safety island, and various interfaces by which to connect the vehicle teleoperation system to various systems onboard a vehicle. For example, the vehicle teleoperation system may receive and process various sensor data, transmit the sensor data for presentation to a teleoperator at a teleoperator station, receive and process user commands from the teleoperator at the teleoperator station, and instruct remote execution of the user commands by the vehicle.

The invention discloses a master-slave teleoperation-oriented force sense information master control system and a resolving method thereof, which are applied to a master-slave teleoperation scene, the master-slave teleoperation scene comprises a master-end teleoperation system, a slave-end mechanical arm system and a communication module, and the master-end teleoperation system comprises a master-end hand controller and a master-end upper computer which are directly connected; the master end upper computer is further internally provided with a force sense information master control unit, and the force sense information master control unit obtains interaction force of the slave end mechanical arm and the external environment under the condition that a sensor is arranged at the tail end of the slave end mechanical arm or no sensor exists, and calculates the interaction force into the master end hand controller. A combined filter is provided to cope with force information interference factors, a user interface is further provided to feed back interaction force between a slave end mechanical arm and an external environment to a master end visual interface, master-slave end overall exchange and comprehensive control of force sense information are achieved, a visual reference is provided for a master end operator to make a decision, and a force sense immediacy sense can be established for master end hand controller equipment.

The invention discloses a two-arm smart teleoperation method and system based on motion redirection and self-adaption, and relates to the technical field of robots, and the method comprises the steps: determining the posture of a human hand and the posture of a human arm according to a human body image; the robot is redirected according to the human hand posture and the human arm posture through the space attention gating map convolutional network, and the joint angle of a mechanical hand of the robot and the joint angle of a mechanical arm of the robot are correspondingly obtained; the mechanical arm is driven according to the joint angle of the mechanical arm so that the mechanical arm can grab the target object; and target force needed for grabbing the target object is determined, and the mechanical arm is driven to grab the target object according to the target force and the joint angle of the mechanical arm. According to the method, topology modeling and cross-platform action redirection between the human body and the robot are achieved through the space attention gating map convolutional network, pairing data dependence is eliminated, and therefore the high precision of action reproduction and the safety and flexibility of the grabbing process are both considered in the complex dynamic environment.

The invention provides a mechanical arm teleoperation system adaptive to a quadruped robot and a control method, and relates to the technical field of robot control. According to the method, firstly, a far-end main mechanical arm is introduced on the basis of a mechanical arm of the quadruped robot to form a main-end mechanical arm system and a slave-end mechanical arm system, and a remote operator remotely maps an action instruction to the slave-end mechanical arm system by controlling the main-end mechanical arm; then, a clamping jaw module with an image and tail end contact force sensor is installed at the tail end of the slave end mechanical arm, and image data collection and tail end contact force collection on a working site are achieved; secondly, the image information and the tail end contact force are remotely transmitted to a remote operator in real time through a wireless transmission module and a remote man-machine interaction module; and finally, a remote operator controls the main end mechanical arm in real time according to the real-time image information and the tail end contact force, remote operation of the quadruped robot mounting mechanical arm in a complex scene is achieved, and the operation efficiency and the operation precision are improved.

The embodiment of the invention provides an equipment remote operation method, a remote operation mechanical arm and a robot system, and relates to the field of equipment control. The method comprises the steps that in response to main mode switching operation of a user, the main mode of the teleoperation mechanical arm is switched into a first main mode, and the first main mode is any one of a remote control mode, a standby mode and an automatic mode; under the condition that the master mode of the teleoperation mechanical arm is the remote control mode, in response to sub-mode switching operation of a user, a remote control sub-mode of the teleoperation mechanical arm in the remote control mode is switched into a first sub-mode, and the remote control sub-mode comprises a clutch sub-mode and a master-slave mapping sub-mode; and under the condition that the remote control sub-mode is the master-slave mapping sub-mode, in response to a first moving operation of a user on the teleoperation mechanical arm, a corresponding control instruction is sent to the slave equipment, so that the slave equipment executes a moving operation corresponding to the first moving operation. According to the method provided by the invention, the flexibility of equipment remote operation is improved.

The invention relates to the technical field of robots, in particular to a mobile robot remote operation system and method based on mixed reality, and the method comprises an operator end and a robot end, the operator end completes a complex remote operation task based on visual feedback provided by a man-machine cooperation interaction interface, and the robot end completes a remote operation task based on visual feedback provided by the man-machine cooperation interaction interface. The virtual reality controller integrates a motion state and an operation mode of the mobile robot, and the mixed reality server generates visual feedback according to visual information of a remote operation visual angle of the mobile robot; the robot end comprises a mobile robot, a collection device on the mobile robot and a robot server, and after the mobile robot receives a remote control instruction sent by the virtual reality controller to the robot end, posture and button information of the mobile robot are input into the robot server to drive the mobile robot to move. Therefore, the problems of low real-time performance and accuracy of remote control, poor operation efficiency and flexibility and the like caused by limitation of bandwidth and system time delay in related technologies are solved.

The invention belongs to the technical field of seismic exploration, and particularly relates to a coal mine underground hydraulic hammering type controllable seismic source vehicle device. The device comprises a crawler, a rotating platform and a hydraulic hammering system, the rotating platform, a four-column type lifting hydraulic support installed on the rotating platform and a hydraulic support in the hydraulic hammering system are hydraulically controlled, and the hydraulic hammering system can conduct 360-degree horizontal rotation, height stepless adjustment and horizontal stretching and retracting. And meanwhile, the crawler-type vehicle body and remote control operation are adopted, so that the crawler-type underground coal mine hammering vehicle is suitable for a complex walking environment in an underground coal mine, and the construction efficiency and the safety are improved.

According to an aspect of the disclosure, a method is provided comprising receiving sensor data from a sensor of a vehicle, such as an autonomous vehicle, and determining, based at least in part on the sensor data, a subset of motion options from a predetermined set of motions options associated with controlling the vehicle. A representation of the subset of motion options is transmitted to a remote system. A first motion option of the subset of motion options is received from the remote system, and the vehicle is controlled based at least in part on the first motion option.

The invention discloses a mechanical arm teleoperation training method based on force feedback, and the method comprises the steps: constructing a hot chamber virtual environment containing a mechanical arm model and an operation object through a scene simulation platform Unity, and configuring a joint position according to a real mechanical arm DH parameter; the dynamics simulation platform AppeliaSim builds a corresponding dynamics model, and an inverse kinematics solution and collision detection function is configured; integrating a force feedback device, and constructing a working space mapping model to convert hand motion into an expected pose of the tail end of the mechanical arm; the double platforms synchronize data through a dynamic link library, and the object control and force sense calculation module calculates collision force and bolt assembling and disassembling torque and transmits the collision force and the bolt assembling and disassembling torque to force feedback equipment. The method does not need to depend on a real mechanical arm, safety risks and high cost are avoided, training repeatability and pertinence are improved, immersion and operation precision are enhanced through force sense and visual sense synchronous feedback, the training period is shortened, and the method is suitable for fine operation training of the mechanical arm in a dangerous environment.

A system and method for robotic teleoperation enable a teleoperated robotic element to perform a sequence of actions based on intention estimation, eliminating the need for continuous human control. The system includes a robotic teleoperation input that receives motion inputs and gaze data from a human operator performing a robotic teleoperation task. A robotic teleoperation feature extractor analyzes and processes the motion inputs and the gaze data into sequential input data. A multi-window model assigns hierarchical prediction windows to the input data, generating windowed sequential data. A hierarchical neural network processes the windowed sequential data to determine low-level action intentions and high-level task intentions, and generates an intention estimation based on the low-level action intentions and the high-level task intentions. A hierarchical dependency model incorporates hierarchical dependent loss to refine the intention estimation.

The invention belongs to the technical field of robots, and discloses a teleoperation robot control method and system with self-adaptive admittance and force feedback, and the method comprises the following steps: 1, loading preset parameters, and moving a main arm to an initial calibration pose; 2, reading an original operator acting force vector and an original environment contact force vector to carry out digital filtering; 3, calculating an adaptive parameter matrix in parallel; 4, the expected tail end speed of the main arm in the current period is calculated, safe amplitude limiting processing is conducted on the component, the current actual tail end pose of the main arm is read, the expected speed obtained after amplitude limiting is converted into the target pose of the main arm in the next period, and a target joint angle instruction is obtained through calculation and sent to the main arm; 5, the actual joint angle of the main arm is read in real time and sent to the auxiliary arm. According to the method, the operation intuition, the task adaptability and the man-machine interaction safety of the system are improved.

The invention provides a teleoperation method and a robot. The method comprises the following steps: receiving operator attitude data sent by an inertial measurement unit IMU system; according to the operator attitude data, target attitude data of the humanoid robot are determined, and the target attitude data comprise the tail end target pose and the target arm angle of the seven-degree-of-freedom mechanical arm; the arm angle is a dihedral angle between a plane where the shoulder part, the elbow part and the wrist part of the seven-degree-of-freedom mechanical arm are located and a target pose and a target arm angle surface of a reference flat tail end; according to the target posture data, motion parameters of all joints of the humanoid robot are determined; and generating a control instruction according to the motion parameters. By means of the method, the uncertainty of null-space motion can be effectively eliminated, and the robot motion is more personified.

The invention provides a drilling machine data acquisition method and system based on remote control, and the method comprises the steps: receiving a user instruction through a wireless remote controller, and transmitting the user instruction to a drilling machine controller. The drilling machine controller determines the working states of the drilling machine and dynamically switches the working states of data acquisition based on a received user instruction, the working states of the drilling machine comprise an automatic propelling state, a manual drilling state and a drilling retreating state, and the working states of the data acquisition at least comprise a standby state, an automatic acquisition state and a forced acquisition state. The system has the advantages that the data acquisition working state and the drilling machine working state can be dynamically switched through remote control operation, and the data acquisition working state and the drilling machine working state are coupled, so that invalid data are filtered, the data validity is improved, and the man-machine cooperation efficiency is improved.

The embodiment of the invention belongs to the technical field of artificial intelligence, and relates to a training data acquisition system based on teleoperation, which is characterized in that a master end interaction device, a slave end image sensor, a slave end robot and a force / touch sensor are respectively in communication connection with a controller; the controller is used for sending operation text prompt information to an operator; acquiring action data in the target operation executed by the operator, wherein the action data is acquired through the main end interaction device; generating a motion instruction based on the motion data; after the slave-end robot completes the target following operation, the grabbing posture of the slave-end robot is obtained; and / or acquiring a first target image acquired by a slave end image sensor; and / or acquiring a force parameter acquired by a force / touch sensor; and taking the operation text prompt information and the corresponding grabbing posture, and / or the first target image, and / or the force parameter as operation training data. The invention further provides a related method, device and the like. According to the technical scheme, more comprehensive and accurate training data samples can be acquired.

The invention relates to the technical field of aeronautical manufacturing and robot assembly, and discloses an aeronautical landing gear cable teleoperation collaborative assembly system and method, and the system comprises a master end interaction subsystem, a slave end assembly subsystem and a central control processing subsystem. The method comprises the following steps: acquiring multi-modal data through an environment monitoring camera, a six-dimensional force sensor and a touch sensor, and realizing space-time registration and state estimation by using Kalman filtering; and planning an optimal path by using a genetic algorithm based on the digital twinborn model and providing augmented reality guidance. And the system calculates visual confidence in real time, and adaptively switches a vision dominant control mode according to the visual confidence: when the vision is limited, the rigidity of the mechanical arm is automatically reduced, the force feedback weight is enhanced, and an operator is assisted to complete compliant operation in a blind area. According to the invention, the problems of sensing shielding and precise control of flexible cable assembly in a narrow cabin are solved, and the safety and efficiency of assembly operation are improved.

A teleoperations system may be used to selectively override conditions detected by an autonomous vehicle to enable the autonomous vehicle to effectively ignore detected conditions that are identified as false positives by the teleoperations system. Furthermore, a teleoperations system may be used to generate commands that an autonomous vehicle validates prior to executing to confirm that the commands do not violate any vehicle constraints for the autonomous vehicle. Still further, an autonomous vehicle may be capable of dynamically varying the video quality of one or more camera feeds that are streamed to a teleoperations system over a bandwidth-constrained wireless network based upon a current context of the autonomous vehicle.

The invention discloses a double-arm hot-line work robot system and an autonomous and teleoperation control method thereof, and relates to the technical field of intelligent operation and maintenance of power systems and robot control. When visual detection is normal, the robot can autonomously complete tasks such as alignment, clamping and unscrewing of the lightning arrester by means of target pose recognition and operation planning, and intervention of operators is not needed; when visual recognition is limited, a robot system can be smoothly switched to a man-machine teleoperation mode, an operator takes over real-time control through VR end mapping and low-delay incremental pose control, a real operation scene is synchronously mapped to a virtual environment and presented to the operator in an immersive mode through VR glasses, visual interaction operation and fine control are achieved, and the visual interaction effect is improved. Therefore, the real immediacy and continuity of the operation are ensured. The two modes of autonomous operation and man-machine teleoperation are integrated, the autonomous operation efficiency under the conventional condition is ensured, remote intervention of operators can be achieved under the abnormal condition, and operation interruption is avoided.

The present disclosure belongs to the technical field of robot control, and specifically provides a four-legged robot mechanical arm end force feedback teleoperation control system and method. In view of the key challenges of control errors caused by communication time delay and soft obstacles difficult to identify in a dynamic environment, the system introduces a predictive contact response mechanism, pre-models and judges the end contact state of the mechanical arm, realizes feedforward control and buffer adjustment for communication time delay. At the same time, a dynamic semantic map is constructed combined with multi-source sensing information, and the soft obstacle inference and path optimization are carried out by fusing the end force sense change trend, so as to significantly improve the identification and avoidance ability of the system in a complex and invisible obstacle environment. The present application has good foresight, adaptability and high redundancy safety characteristics, and is suitable for multi-task inspection operation in industrial sites such as thermal power plants, and is particularly suitable for remote human-machine collaborative operation scenes in narrow spaces.

The invention discloses a robot imitation learning method based on multi-modal sensing and stage conditions, relates to the agricultural chemical engineering automation and intelligent robot technology, and can at least partially solve the problems that in the prior art, multi-modal sensing fusion is insufficient, long-time-sequence task planning is complex, and safety requirements are difficult to meet. The method comprises the steps that multi-source sensing data of vision, touch, six-dimensional force and body sensing are collected and preprocessed; defining a task stage; a multi-modal fusion pure decoder Transform architecture is constructed, so that cross-modal feature fusion can be realized; designing a comprehensive loss function including action prediction loss, time sequence characteristic consistency loss and stage consistency loss; collecting demonstration data through teleoperation and preprocessing the demonstration data; carrying out model training by adopting a progressive training strategy; stage detection, multi-modal data preprocessing, model reasoning and time sequence integration are carried out in the real-time reasoning stage. According to the invention, high-precision, high-safety and high-adaptability robot automatic operation is realized.

The invention provides a dexterous hand teleoperation method, device and system based on a data glove, and is applied to the technical field of smart power grids. The method is applied to the electronic equipment, the electronic equipment is in communication connection with a data glove and a dexterous hand which are worn on the hand of an operator, and the method comprises the steps that action data, collected by the data glove, of the hand of the operator are obtained, and the action data comprise first joint freedom degrees corresponding to all joints of each finger; based on a preset human hand prior knowledge base, performing coupling mapping processing on the action data to obtain adjusted action data conforming to a human hand natural movement mode; performing smoothing and state prediction on the adjusted action data by adopting a Kalman filtering algorithm to obtain a target action sequence; and according to the target action sequence, generating a joint space control instruction for the dexterous hand to drive the dexterous hand to reproduce the action of the hand of the operator. The method is used for improving the intelligent level and the natural fluency and collaboration of the actions of the dexterous hand.