56 results about "Robot teleoperation" patented technology

The invention relates to the technical field of robots, and discloses a collision early warning method and device in robot teleoperation, and the method comprises the steps: obtaining the visual information of the working environment of a remote robot and the motion state data of an end effector; environment three-dimensional modeling is carried out based on visual information, a three-dimensional environment model containing obstacles is generated, and a virtual protection area surrounding an end effector is dynamically generated based on end speed parameters in motion state data and system communication processing time delay; wherein the spatial range of the virtual protection area is expanded along with the increase of the terminal speed and the system communication processing time delay; performing collision risk assessment on the virtual protection area and an obstacle in the three-dimensional environment model to obtain a current collision risk level; and generating early warning information based on the collision risk level, and outputting a vision enhancement image through an operator terminal to guide an operator to avoid the collision risk based on the early warning information. According to the invention, the teleoperation safety can be improved.

The invention discloses a low-cost robot imitation learning method and system based on a human video. The method comprises the following steps: S1, data acquisition; s2, data extraction and physical alignment are carried out to eliminate man-machine physical differences; the step is divided into two parallel processing modules of action space alignment and visual space alignment; s3, data set construction: mixing the aligned human data with real robot teleoperation data, carrying out balanced sampling, and constructing a mixed data set Dmix; and S4, cooperative training: constructing a strategy network based on diffusion Transform for training. According to the method, data can be acquired only through the monocular RGB camera, expensive robot teleoperation data are replaced with cheap and easily available human videos, and the data acquisition threshold is greatly reduced. Through a visual alignment strategy of random color grid rendering, a network can learn neglect skin color textures and pay attention to geometric structures without a complex generative model, so that the robot can be seamlessly migrated to robots in different forms.

The invention discloses a man-machine collaborative robot teleoperation method and system based on visual interaction, and belongs to the technical field of robot control and interaction. In order to solve the problems that a traditional teleoperation instruction is not clear, execution is not verified and dependence on a network is high, the method comprises the steps that a robot video stream is displayed on a client; determining a target two-dimensional coordinate in response to a click operation of a user on the video picture; and the coordinates and the three-dimensional environment data space are mapped, and the execution pose of the robot is calculated and verified. The system allows a user to visually specify a target through frame selection / click selection, and a background automatically converts two-dimensional interaction information into an accurate three-dimensional robot instruction subjected to feasibility verification. According to the method, the coordinate precision is guaranteed by introducing the static reference image, batch planning and submission are supported through the task queue to reduce network dependence, and safe, accurate and high-robustness man-machine collaborative operation is realized.

The invention belongs to the technical field of robot teleoperation and haptic interaction, and provides a multi-mode force feedback control method and system for nuclear facility robot operation, and the method comprises the steps: collecting the state information of a master arm and a slave arm in real time, and dynamically scheduling a haptic calculation strategy according to a task state; a fast estimation strategy is enabled in the fast transit mode. Further, real environment interaction force is obtained based on dynamic model compensation and net interaction force extraction; combining the dynamically selected multi-mode force feedback parameters, and synthesizing and outputting a touch instruction to the main arm. Meanwhile, admittance correction and feed-forward compensation are carried out on a slave arm motion instruction through net interaction force and dynamic internal force, synchronous output is carried out after safety optimization, and the slave arm is controlled to move. The problems of poor task adaptability, force sense distortion and insufficient safety in nuclear facility teleoperation are solved, and intelligent, high-fidelity and safe robot teleoperation is achieved.

The invention provides a man-machine interaction robot teleoperation control system and method based on data gloves. The man-machine interaction robot teleoperation control system comprises a data acquisition module, a data processing module and a robot execution module. The data acquisition module acquires and sends finger bending degree and palm posture information; the data processing module pre-processes the collected information, performs gesture recognition and pose mapping based on the pre-processed information, and generates and sends a control instruction; and the robot execution module receives the control instruction and controls the robot to execute corresponding actions. The technical problem that an existing robot control system based on data gloves is insufficient in recognition precision, real-time performance and system compatibility is solved.

The application provides a dexterous arm hand isomeric teleoperation action mapping method and system, and relates to the technical field of robot teleoperation. The method comprises the following steps: collecting three-dimensional space position data of human hand key points; constructing a mapping function from the three-dimensional space position data of the human hand key points to robot joint angles; wherein the robot comprises a mechanical arm and a dexterous hand; constructing a comprehensive weighted optimization objective function, which is a weighted sum of multiple optimization sub-target items; optimizing the mapping function according to the three-dimensional space position data of the human hand key points with the comprehensive weighted optimization objective function as the optimization criterion to obtain an optimized mapping function; in real-time teleoperation, inputting the real-time collected three-dimensional space position data of the human hand key points into the optimized mapping function to obtain target joint angles of the robot, so as to drive the robot to perform corresponding actions.

The invention relates to the technical field of XR operation, and discloses a WebXR-based panoramic immersive robot teleoperation system and method. The method comprises the following steps: carrying out distortion correction and splicing fusion on original images collected by a double-fisheye lens to obtain a panoramic image frame; encoding the panoramic image frame into a video code stream, transmitting the video code stream to a browser end, and decoding the video code stream at the browser end to obtain a panoramic video texture; mapping the panoramic video texture to the inner surface of a sphere, setting a virtual camera at the center of the sphere, and constructing a panoramic sphere rendering model; and converting the attitude quaternion of the VR head-mounted display into a sight decoupling rotation matrix, performing local GPU rendering on the panoramic sphere rendering model to obtain a real-time visual angle picture, and mapping a rocker shaft value of the gamepad into a control instruction to be sent to a robot dog end. In the whole view angle switching process, vertex transformation and texture sampling are completed at the browser end, no pan-tilt rotation instruction is generated or sent to the robot dog end through the network, and delay is reduced.

The invention discloses a wearable teleoperation master hand, and belongs to the technical field of robot teleoperation and wearable equipment. The main hand comprises main glove sets symmetrically arranged on the two sides of a carbon fiber tube, each set comprises a shoulder joint, an elbow joint, a wrist joint and an adjustable large / small arm wearing structure, and each joint is integrated with an angle sensor to collect motion data in real time. The master manipulator adapts to different users through the length adjusting device, and naturalness and precision of teleoperation are improved. The upper limb motion information of an operator is captured through lightweight wearable design, and a more visual and efficient input mode is provided for remote control of the robot.

This invention provides a VR-based dual-arm robot teleoperation method and system, comprising: parallel calculation of left and right arm joint angles based on the VR target pose and the robot's current joint actual position; then checking the remaining steps in the left and right arm execution buffers; if the remaining steps in the left and right arm execution buffers are less than a set value, performing a trajectory mixing and generation step based on the joint angles of the left and right arms to obtain a new mixed trajectory segment for supplementation; calculating the position error of the left and right arms based on the robot's current shutdown position and dynamically adjusting the trajectory generation mixing window; wherein, in the trajectory mixing and generation step, the new trajectory is shifted forward by the sum of inference delay and processing delay to align with the old trajectory time. This invention dynamically adjusts the trajectory cutoff point through an adaptive delay compensation mechanism to avoid using "expired" instructions; and ensures that the execution buffer always has sufficient trajectory through an intelligent buffer management mechanism, completely eliminating trajectory breaks caused by IK delay fluctuations.

The embodiment of the invention provides a robot teleoperation method, device and system, electronic equipment and a computer readable storage medium, and relates to the technical field of robots. The method comprises the steps that based on a handle motion signal of an operation end, a robot instruction clamping jaw pose at the current moment is determined; according to the robot instruction clamping jaw pose and a locally pre-stored first spatial transformation matrix, a virtual clamping jaw predicted pose at the current moment is determined; determining a virtual camera pose according to the pose of the robot head camera in the world coordinate system at the current moment and a locally pre-stored second spatial transformation matrix; and performing clamping jaw rendering locally according to the virtual clamping jaw prediction pose and the virtual camera pose.

The present application relates to the technical field of heterogeneous robot teleoperation, and particularly relates to a flexible motion mapping method and system for heterogeneous robots, wherein the method comprises the following steps: collecting joint motion data of a master end follow-up teach box; extracting core task components corresponding to a task space of a slave end executor, identifying and separating redundant action components; converting at least one motion signal in the redundant action components into a driving instruction of the slave end executor according to a flexible mapping rule; performing safety constraint processing on the driving instruction, receiving the processed instruction by the slave end executor and executing the same; obtaining physical motion parameters of the slave end executor, generating a virtual pose feedback signal conforming to a master end communication protocol by using a reverse reconstruction algorithm, and returning the same to the master end follow-up teach box. The present application can eliminate control stiffness and logic deadlock in heterogeneous teleoperation, avoid mechanical impact caused by instruction mutation, ensure that operation polarity and visual motion are highly consistent, and improve operation efficiency.

The invention discloses a pose mapping method based on a drift function and a synchronous control system, and belongs to the technical field of robot teleoperation and intelligent control. According to the method, a heterogeneous coordinate system mapping relation between a near-end VR head-mounted display and a far-end observation robot is established, a drift function activated based on normalized operability is introduced, and an unreachable area and a kinematics singular configuration are dynamically avoided; the system adopts a double-layer architecture of outer ring mapping and inner ring PID control, and high-continuity and high-precision pose synchronization is realized. According to the invention, accurate pose control of the far-end observation robot by a near-end operator can be realized, and teleoperation safety and telepresence are improved.

This disclosure provides a teleoperation compensation and teleoperation method, apparatus, and electronic device. The teleoperation compensation method includes: at the slave control end of the teleoperation system, modifying the control model of the teleoperation system based on the robot's actual state parameters without network latency and the robot's physical configuration parameters; at the master control end of the teleoperation system, compensating for the robot's network interference torque based on the robot's feedback state parameters including network latency, and further modifying the modified control model; based on the secondary modified control model and the compensated network interference torque, compensating for the robot's feedback state parameters to obtain the robot's target state parameters. This disclosure enables robot teleoperation to possess the advantages of high precision, strong robustness, and low latency.

The invention discloses a robot teleoperation method based on fusion of virtual reality and mixed reality. The robot teleoperation method comprises the following steps: sensing a far-end environment by observing a robot; constructing a near-end interactive virtual environment, and presenting the virtual environment to a near-end operator through a virtual reality device; constructing three-dimensional point cloud data based on a far-end environment through three-dimensional reconstruction; in the virtual environment, pose estimation is carried out on the far-end operation robot and the target object, and three-dimensional registration is carried out in the virtual environment; working space boundary condition constraints and space operation pose change constraints of the working robot are determined, path planning and trajectory planning are conducted on the working robot, and visualization is conducted in the virtual environment; and generating a control program of the operation robot to move along the planned track. By constructing the interactive virtual environment, the working space range and the operation pose of the operation robot are visualized in the interactive virtual environment, and the trajectory is planned and visualized in the interactive virtual environment, so that the safety of teleoperation is greatly improved.

The application discloses a kind of man-machine collaborative multi-arm teleoperation control system, method and robot based on multimodal large language model, it is related to robot teleoperation technical field, the system is equipped with four big modules of visual perception, LLM driven task planning, motion primitive execution, safety supervision of human-in-the-loop, decouples high-dimensional control space with "main pilot-co-pilot" paradigm, human remote control main arm makes fine operation, intelligent agent independently controls auxiliary arm to execute coarse-grained task, voice intervention instruction has absolute priority.This application breaks through the bottleneck of cognition and operation of single-person control multi-arm, realizes single operator to complete four-arm strongly coupled collaborative task, eliminates the communication delay of multi-person cooperation, greatly reduces demonstration trajectory variance, improves the quality of teaching data, optimizes downstream model training performance, while significantly reducing the operation threshold, realizes high-level physical safety through low-delay voice supervision, can be widely applied to industrial assembly, logistics handling and other scenes, and has strong industrial practicability.

The present application specifically relates to the technical field of human-machine collaborative intelligent manufacturing and assembly, and provides a virtual reality and digital twin-based robot teleoperation method and system. The solution comprises: starting up an industrial robot and a wearable device; establishing a bidirectional communication connection between a virtual environment of the wearable device and the industrial robot, and rendering a virtual robot of the wearable device in the virtual environment; and inputting a target pose operation instruction into the virtual robot, and in response to receiving a target motion trajectory planned by the wearable device according to the target pose operation instruction, controlling the industrial robot to perform an operation task according to the motion trajectory. The solution enables information interaction between an operator and a robot, thereby achieving immersive real-time robot teleoperation by means of instant and accurate operation commands and on-site feedback, and significantly improving teleoperation accuracy for complex tasks and flexible manipulation tasks.

The application provides a construction method of a robot teleoperation body model and a robot teleoperation system, and the method comprises the following steps: fine-tuning a pre-training body model by using master-slave robot control data to obtain a fine-tuned body model; deploying the fine-tuned body model to provide robot control services and collecting real machine execution data; obtaining scores of the real machine execution data, and forming reinforcement learning training data pairs of the real machine execution data and corresponding scores, and adding the reinforcement learning training data pairs to a reinforcement training data set; and performing reinforcement learning training on the fine-tuned body model by using the reinforcement training data set. The pre-training model is fine-tuned, the cost of manual intervention is effectively reduced, the reinforcement learning training data is collected by deploying the fine-tuned body model, the reinforcement learning method and the pre-training body model are organically combined, the advantages of the two are fully utilized, and efficient learning and performance improvement of the robot in a complex environment are realized.

A method for probabilistic modeling for intention estimation in an operator-robot teleoperated assembly task is provided. The method may estimate the assembly task to be completed, wherein the assembly task comprises a sequence of actions. The method may predict a next action of the sequence of actions to be performed for the assembly task to be completed.

This application provides a robot remote control operating system, including a master robot, a slave robot, and a control system. The control system is configured to cause the slave robot to move in accordance with the movement of the master robot. The control system is further configured to determine a coefficient K and, based on the displacement difference between a reference point on the master robot and a selected point on the slave robot corresponding to the reference point, and the coefficient K, determine the control force F output by the slave robot at the selected point.

The invention discloses an immersive robot teleoperation master-slave system and method based on bidirectional force sense mapping. The system comprises a master hand controller and a slave end robot, wherein master-slave position-force bidirectional closed-loop control is established through a wireless communication mode; the master hand controller sends a mechanical arm target pose instruction, an execution tail end closing / releasing instruction and a control signal of the mobile platform to the slave end robot to form a forward control channel; the control panel obtains feedback torque based on the current I fed back by the slave end robot and a preset torque coefficient Kt, and the feedback torque serves as force sense feedback of the slave end; the control panel sends a torque resisting instruction to the feedback motor according to the feedback torque, the feedback driving motor rotates according to the torque resisting instruction, the feedback transmission mechanism is driven to act, and a reverse feedback channel is formed. According to the invention, the action of the operator can be accurately mapped to the slave end robot, and the force generated by interaction between the slave end robot and the environment can be truly fed back to the operator in a low-delay manner.

The robot remote control system includes a control device controlling a robot and an HMI used by an operator when remotely controlling the robot. The HMI displays a video of surroundings of the robot on a display, detects a sightline or the like of the operator using an eye tracker, estimates an operation intention of the operator, and transmits sightline data indicating the detected sightline or the like and operation data indicating the estimated operation intention of the operator to the control device. The control device determines a target position of the robot on the basis of the sightline data and the operation data received from the HMI and moves the robot to the target position.

A robot teleoperation control system and method, relating to the technical field of robots. The robot teleoperation control system comprises: a teleoperation device, a control device, and a controlled robot. The teleoperation device comprises an isomorphic body having the same joint configuration as a controlled structure of the controlled robot, the teleoperation device is communicatively connected to the control device, and the control device is communicatively connected to the controlled robot. The control device is configured to receive motion information sent by the teleoperation device or the controlled robot, and generate a motion instruction on the basis of a joint mapping relationship. The motion information of the teleoperation device is generated on the basis of an operation action of the isomorphic body, and the motion information of the controlled robot is generated on the basis of an automated operation action of the controlled structure of the controlled robot. The control device is further configured to send the motion instruction to a device corresponding to the motion information, so that the isomorphic body of the teleoperation device and the controlled structure of the controlled robot perform the same action, thereby realizing bidirectional control of the teleoperation device and the controlled robot.

The invention discloses a hand exoskeleton and a robot teleoperation system, the hand exoskeleton comprises a palm fixing seat and a thumb mechanism, an index finger mechanism, a middle finger mechanism, a ring finger mechanism and a little finger mechanism which are arranged on the palm fixing seat, and the thumb mechanism comprises a first knuckle and a second knuckle; the first knuckles are rotationally connected with the side, close to the index finger mechanism, of the palm fixing base, the second knuckles are rotationally connected with the first knuckles, and the rotating axes of the second knuckles and the rotating axes of the first knuckles intersect on the same vertical projection plane. The first knuckles are rotationally connected with the side, close to the index finger mechanism, of the palm fixing base, the second knuckles are rotationally connected with the first knuckles, and the two rotating axes intersect on the same vertical projection plane, so that the multi-degree-of-freedom linkage motion logic of the thumb of the human body can be accurately matched; complex actions such as bending and stretching, side swaying and rotating of the thumb can be comprehensively covered and accurately followed, and the problem that tiny actions cannot be caught sufficiently due to the fact that an existing exoskeleton is simplified in structure is effectively solved.

1. Name of the product in this design: Remote Control. 2. Purpose of this design: For use as a robot remote control. 3. The key design feature of this product is its shape. 4. The image or photograph that best illustrates the design's key points: 3D view 1.

The invention discloses a robot teleoperation method and system. The method comprises the following steps: in a manual teleoperation mode, acquiring self and / or environment information of the robot and generating autonomous operation options according to the information; providing the autonomous operation option to a human operator; and in response to the acquired selection, causing the robot to enter the autonomous operation mode to perform the selected autonomous operation. Thereby, direct participation of a human operator with persistent or regularized operations is reduced by identifying operation content of a predetermined granularity suitable for autonomous execution by the robot during teleoperation. And by introducing an automatic abnormity monitoring and manual takeover mechanism, the control right can be timely returned to the operator when the automatic execution is abnormal, so that the execution efficiency is improved, and the reliability and the safety of the system are ensured. Based on the above mechanism, the cooperation of the operator and the robot in the same task process is realized, so that the operator and the robot play roles in proper operation links, and the overall man-machine cooperation effect is improved.

The invention discloses a robot-shaped remote controller, which belongs to the technical field of robot remote control, and comprises a head assembly, a body assembly, a leg assembly, a plurality of groups of Hall sensor connecting pieces and a control module, the Hall sensor connecting piece can measure a rotation angle and is electrically connected with the control module; the head assembly can pitch, rotate and laterally swing relative to the body assembly, and the robot is controlled by the control module to complete corresponding actions. The leg assemblies can horizontally rotate and swing forwards, backwards, leftwards and rightwards relative to the body assembly, and the robot is controlled by the control module to complete corresponding actions. The robot-shaped remote controller can effectively solve the problems that in the prior art, a user needs to repeatedly memorize the key positions of a robot remote controller, and the robot can be controlled only after long-time practice and use; for children, old people and people who are slow in response speed or do not use handles frequently, the problems of high operation difficulty, inharmonious action control, non-visual action control and the like exist.

The application discloses a kind of robot teleoperation control methods based on no delay data training under delay environment, comprising the following steps: S1: the off-line data set obtained by robot in no delay environment executes task, off-line data set includes multiple trajectories, each trajectory includes state sequence, action sequence and reward sequence;S2: the state sequence and action sequence in off-line data set are reconstructed, and the information state sequence suitable for delay environment is generated;S3: state estimation is carried out to information state sequence, and the predicted state sequence is obtained;S4: based on information state sequence and predicted state sequence, offline reinforcement learning algorithm is used to train strategy model, and strategy model is used to output the action instruction of control robot based on predicted state sequence in delay environment;S5: the trained strategy model is used to control robot to execute teleoperation task in delay environment.The application realizes the safe, efficient and reliable control of robot teleoperation in delay environment.

The application discloses a kind of robot teleoperation guiding methods based on force feedback, including two parts of buffer-based force feedback obstacle avoidance position guidance and spherical guiding force field force feedback auxiliary grabbing, wherein buffer-based force feedback obstacle avoidance position guidance is based on the establishment of obstacle avoidance buffer based on obstacle bounding box, then whether the collision of mechanical arm and obstacle buffer zone occurs is judged, if collision occurs, force feedback is calculated according to the position of mechanical arm in buffer zone;With better obstacle avoidance effect.Spherical guiding force field force feedback auxiliary grabbing is to construct spherical guiding force field and spring-mass model, then the feedback force of spherical guiding force field is calculated based on spring-mass model, to assist operator to control the position of mechanical arm end within the outer layer between spherical guiding force field, reduce the attention of operator to the position of mechanical arm, so as to adjust the attitude of mechanical arm can efficiently complete clamping operation, realize the effect of reducing burden, reducing misoperation, improving teleoperation grabbing work efficiency.

The invention provides a humanoid robot teleoperation grabbing control method, and belongs to the technical field of robot teleoperation control. The problems that in the prior art, due to the fact that a teleoperation system lacks motion intention prediction, network delay amplification track errors are caused, due to insufficient tactile feedback, force control switching overshoot is caused, and due to unstable joint redundancy solving, attitude jittering is caused can be at least partially solved. The method comprises the following steps: acquiring wrist speed data, joint angular acceleration data and pose trajectory data of an operator, and constructing a multi-modal time sequence coding sequence; outputting a predicted action instruction; collecting pressure distribution data of a tactile array, calculating tactile deviation and generating a torque compensation instruction based on a compliant control law; outputting a whole body joint speed instruction; and according to the tactile data and the force feedback data, physical attribute parameters of a grabbed object are estimated online, and grabbed control parameters are adjusted in a self-adaptive mode. According to the invention, low-delay, high-flexibility and high-stability teleoperation grabbing control is realized, and the method is suitable for fine operation scenes such as glassware grabbing and the like.

The invention discloses a hand exoskeleton and a robot teleoperation system. A finger mechanism of the hand exoskeleton is arranged on a palm fixing seat; the thumb mechanism comprises a knuckle assembly and a connecting rod assembly, the knuckle assembly is rotationally connected with the palm fixing base, the connecting rod assembly comprises a first connecting rod and a second connecting rod, one end of the first connecting rod is rotationally connected with the palm fixing base, the other end of the first connecting rod is connected with one end of the second connecting rod, and the other end of the second connecting rod is rotationally connected with the knuckle assembly. The knuckle assemblies are rotatably connected with the palm fixing base, and are matched with the connecting rod assembly composed of the first connecting rod and the second connecting rod, so that the problem that an existing thumb mechanism is unreasonable in connecting mode is effectively solved, the posture of the thumb can be flexibly adjusted according to the natural physiological structure of the hand in the movement process, and wearing discomfort is avoided; meanwhile, through multi-connecting-rod cooperative guiding, it is ensured that the movement track is highly attached to the physiological track of the hand of the human body, the joint additional burden is reduced, and secondary injury is prevented.