148 results about "Robot teleoperation" patented technology

Systems, methods, and computer program products for automating tasks are described. A multi-step framework enables a gradient towards task automation. An agent performs a task while sensors collect data. The data are used to generate a script that characterizes the discrete actions executed by the agent in the performance of the task. The script is used by a robot teleoperation system to control a robot to perform the task. The robot teleoperation system maps the script into an ordered set of action commands that the robot is operative to auto-complete to enable the robot to semi-autonomously perform the task. The ordered set of action commands is converted into an automation program that may be accessed by an autonomous robot and executed to cause the autonomous robot to autonomously perform the task. In training, simulated instances of the robot may perform simulated instances of the task in simulated environments.

The invention provides a robot teleoperation method and system, a storage medium, equipment and a program product. The method comprises the following steps: receiving skeleton movement data of an operator, selecting pose information of part of skeleton nodes from the skeleton movement data, and taking the pose information as reference movement data; mapping the reference motion data into target motion information of the robot; and solving based on the target motion information to generate a control instruction for driving the robot joint. Therefore, the pose information of part of the skeleton nodes is selected from the skeleton motion data and is taken as the reference motion data, so that core motion characteristics can be reserved, the complexity of mapping from the skeleton motion data to the target motion information of the robot in the related technology is simplified, redundant data interference is reduced, and the reliability of the robot is improved. And therefore, the limited joint freedom degree of the robot can be adapted.

The invention relates to the technical field of robots, and discloses an SLAM navigation-based inspection robot inspection track self-correction method, which comprises the following steps: S1, acquiring environmental data through a 2D laser radar, an attitude sensor and an odometer sensor, and generating an initial environmental map and a navigation path by using an SLAM mapping algorithm; s2, fusing the laser radar pose data, the odometer pose data and the pose sensor pose data to obtain real-time pose information, comparing the current pose with the target path pose, and calculating a path error; s3, a robot motion instruction is adjusted through a compensation control algorithm based on the path error, and the path deviation is corrected; s4, continuously monitoring path errors and correcting the path errors in real time; and S5, storing the corrected track as a navigation path file. According to the invention, by introducing the robot remote controller, complex functions such as mapping, navigation, navigation point marking, path storage, motion control and the like are modularized, and the operation process is greatly simplified.

The invention provides a wearable exoskeleton device, an operation method and a robot teleoperation system. The wearable exoskeleton device comprises a backpack connecting piece and an arm exoskeleton. The bearing connecting piece is used for enabling a user to bear the exoskeleton device, and arm exoskeletons are arranged at the positions, corresponding to the left arm and / or the right arm of the user, of the bearing connecting piece. The information collector is arranged on the arm exoskeleton, so that the motion data of the arm in multiple dimensions when the user operates the arm exoskeleton can be collected. Compared with a traditional mechanical robot teaching system which is complex and high in cost, due to the fact that the structure is relatively simple, the production cost is greatly reduced, and meanwhile the mechanical robot teaching system has the advantages of being light, stable, convenient, wearable and the like. Through the arrangement of the information collector, accurate motion data can be rapidly obtained from all the human body joint components corresponding to the exoskeleton, the precision degree and efficiency of robot teleoperation can be rapidly improved, and the cost of training the robot is effectively reduced.

The invention provides a robot teleoperation method and system based on virtual reality and digital twinning, and particularly relates to the technical field of man-machine collaborative intelligent manufacturing and assemblation.The scheme comprises the steps that an industrial robot and wearable equipment are started; establishing a two-way communication connection between the virtual environment of the wearable device and the industrial robot, and rendering the virtual robot of the wearable device in the virtual environment; and inputting the target pose operation instruction into the virtual robot, responding to a received target motion track planned by the wearable device according to the target pose operation instruction, and controlling the industrial robot to execute an operation task according to the motion track. According to the scheme, information interaction between an operator and the robot can be achieved, so that immersive real-time robot teleoperation is achieved through instant and accurate operation commands and field feedback, and the teleoperation precision of complex tasks and flexible operation tasks can be remarkably improved.

The invention provides a construction method for a robot teleoperation body-sized model and a robot teleoperation system, and the method comprises the steps: carrying out the training and fine adjustment of a pre-trained body-sized model through master-slave robot control data, and obtaining a fine-adjusted body-sized model; deploying the fine-tuning body large model to provide robot control service, and collecting real machine execution data; obtaining scores of the real machine execution data, forming reinforcement learning training data pairs by the real machine execution data and the corresponding scores, and adding the reinforcement learning training data pairs into a reinforcement training data set; and performing reinforcement learning training on the fine-tuning body size model by using the reinforcement training data set. According to the method, the pre-training model is finely adjusted, the cost of manual intervention is effectively reduced, the reinforcement learning training data is collected by deploying the fine adjustment body model, the reinforcement learning method and the pre-training body model are organically combined, the advantages of the reinforcement learning method and the pre-training body model are fully played, and efficient learning and performance improvement of the robot in a complex environment are achieved.

The invention discloses an immersive robot teleoperation method and system, and belongs to the technical field of robot teleoperation. An existing teleoperation scheme lacks an effective and timely feedback mechanism, is poor in operation intuition and low in control precision, and is difficult to compete with complex and fine operation tasks. According to the immersive robot teleoperation method, by constructing the first person mapping model, the interaction data acquisition model, the teleoperation processing model, the execution perception model and the immersive visual angle simulation model, real-time mapping can be performed on posture actions of an operator, and a real-time response three-dimensional image capable of being displayed in VR equipment is generated; an effective and timely immersive feedback mechanism from a first person perspective is formed, an operator can execute a task as like being personally on the scene, and the immersion, intuition and naturalness of interaction are improved, so that the control capability and the task execution effect can be remarkably improved, the operation efficiency and precision are improved, and then the operator can competent for complex and fine operation tasks.

The invention provides a robot teleoperation method and a robot. The robot teleoperation method comprises the steps that an acceleration command, an angular acceleration command and an expected attitude quaternion of an end effector are determined based on the current linear acceleration, the current angular acceleration and the current attitude quaternion of a control end; the current position and the current speed fed back by the end effector are obtained, a torque control signal for controlling the position of the end effector is determined based on the acceleration command, the current position and the current speed, and end acceleration feedforward is obtained; determining a torque control signal for controlling the attitude of the end effector based on the angular acceleration command and the expected attitude quaternion to obtain attitude feedforward; and a joint torque instruction of a mechanical arm joint corresponding to the end effector is determined based on the end acceleration feedforward and the posture feedforward, and the joint torque instruction is sent to a corresponding joint driver so that the end effector can synchronously act along with the control end. According to the invention, the problem of attitude lag of the end effector can be avoided, and the tracking error of the end effector is reduced.

The invention belongs to the technical field of robots, and discloses a man-machine-data three-element integrated robot teleoperation and data acquisition system and method, and the system comprises a man-machine interaction module, a physical robot, a prediction simulation module, a digital robot, a virtual environment and a data management module. Wherein the prediction simulation module is used for aligning information of a natural person to execution action information of the physical robot, predicting an action instruction sequence and correcting the state of the digital robot in real time according to state information fed back by the physical robot; the digital robot is used for executing a corresponding action in the virtual environment according to the combined action instruction sequence, if a safety problem exists in the execution process, execution is stopped, and otherwise, the combined action instruction sequence is sent to the physical robot; the data management module is used for collecting data in the system operation process. The technical problems of operation delay and lack of humanoid robot training data in a traditional teleoperation system are solved.

The invention relates to a robot teleoperation algorithm system based on motion capture equipment. The system comprises a motion capture data receiving module, a pose conversion module, a safety monitoring module and a control instruction issuing module. Teleoperation is carried out by using a mode of converting data of the motion capture equipment into a robot control instruction, the efficiency of completing a specific task by the teleoperation robot can be greatly improved, multiple control modes are realized, and an operator can more intuitively operate the robot to complete the task; meanwhile, the safety monitoring module designed in the system can ensure the safety of operators and the robot when the motion capture data is abnormal or drifts, and has high robustness. Moreover, the system can achieve the adaptation to different robots, and can reduce the adaptation development amount during hardware migration.

A waterproof fire-fighting robot remote controller relates to the technical field of electronic product equipment. The utility model discloses a remote controller, and aims to solve the problems that in the prior art, generally, a waterproof function is not provided, water falls on the remote controller and easily permeates into the remote controller to damage the remote controller, and fire extinguishing operation is affected. A waterproof fire-fighting robot remote controller comprises a remote controller body, the remote controller body is fixedly connected with grips which are symmetrically distributed, the remote controller body is fixedly connected with a cambered surface glass plate, the remote controller body is fixedly connected with a plurality of groups of control buttons which are symmetrically distributed, and the remote controller body is fixedly connected with direction control rockers which are symmetrically distributed. The remote controller body is fixedly connected with symmetrically distributed threaded connection seats, and the threaded connection seats are provided with antennas in a threaded manner. The top surface of the cambered surface glass plate is cambered, so that water falling on the cambered surface glass plate can be guided from the front side and the rear side, the water can be prevented from permeating into the remote controller body, and the waterproof purpose is further achieved.

The invention provides a robot teleoperation control system and method, and relates to the technical field of robots. The robot teleoperation control system comprises teleoperation equipment, control equipment and a controlled robot, the teleoperation equipment comprises an isomorphic body with the same joint configuration as a controlled structure of the controlled robot, the teleoperation equipment is in communication connection with the control equipment, and the control equipment is in communication connection with the controlled robot; the teleoperation equipment is used for generating motion information according to the operation action of a user on the isomorphic body and sending the motion information to the control equipment; the control equipment is used for generating a motion control instruction of the controlled robot according to the motion information and based on a joint mapping relation between an isomorphic body of the teleoperation equipment and a controlled structure of the controlled robot, and sending the motion control instruction to the controlled robot; and the controlled robot is used for executing the same action as the isomorphic body action of the teleoperation equipment according to the motion control instruction. The robot teleoperation control precision can be improved.

The invention relates to a time-delay variable-gain robot virtual-real interaction remote control system and method. The method comprises the following steps: acquiring data of wrist, finger and pedal actions; mapping the collected action data to obtain a motion instruction of the slave end robot, and selecting and recording the motion instruction in a certain time period according to different treading states of the pedal action to obtain a corresponding motion instruction sequence; a motion instruction sequence is received, and according to the distance between a slave-end robot virtual simulation model and a virtual simulation environment space obstacle and the time delay generated when the motion instruction is sent from the master-end controller to the slave-end controller to be processed and fed back, mapping proportion adjustment is conducted on the motion instruction to obtain an operation instruction; and according to the operation instruction, the slave end robot is driven to execute a corresponding action, and action state feedback is carried out. The problems that in the current robot teleoperation technology, operation is unstable and operation is prone to collision due to unstable time delay jitter and sensing limitation are solved.

The invention relates to a teleoperation sharing control method and device for an electric robot and a medium, and the method comprises the steps: selecting a key point on a mechanical arm of the electric robot, and building a kinematics state expression of an end effector of the mechanical arm and the key point in teleoperation; based on the kinematics state expression, considering the influence of obstacles, and combining a control obstacle function to construct safety constraints of teleoperation of the power robot; acquiring a reference trajectory of the mechanical arm of the electric robot set by an operator, introducing the safety constraint into nonlinear model predictive control, and constructing an NMPC-CBF controller to track the reference trajectory; and calculating the resultant force fed back by the repulsive force of the mechanical arm of the electric robot and the obstacle by adopting an artificial potential field method, and correcting the reference track according to the resultant force to complete the sharing control process. Compared with the prior art, the method has the advantages of efficient obstacle avoidance and the like.

The invention discloses double-arm heavy-load robot teleoperation exoskeleton equipment with a limiting protection function. The robot comprises a fixed base and exoskeletons installed on the left side and the right side, the exoskeleton on each side is provided with eight active degrees of freedom, and the eight active degrees of freedom comprise shoulder joint three degrees of freedom, elbow joint one degree of freedom, wrist joint three degrees of freedom and tail end clamping jaw one degree of freedom. According to the human engineering size, four passive adjustment freedom degrees of the shoulder height, the shoulder width, the large arm length and the small arm length are increased so as to adapt to the body types of different operators. The device has the following obvious advantages that the joint limiting device is arranged, so that joints are prevented from being damaged due to over-limit movement, and the operation safety is effectively improved; a passive degree-of-freedom design is introduced, so that the fitting degree of the equipment and a human body joint natural movement track is improved; an operator can conveniently and directly observe the state of the slave end mechanical arm clamping jaw through the tail end clamping jaw, the hand enabling holding rod is provided with a multi-stage button, adjustment in multiple mapping modes is supported, and the precise control requirement of a complex two-arm rescue task is met.

The invention relates to the technical field of robots, and discloses a collision early warning method and device in robot teleoperation, and the method comprises the steps: obtaining the visual information of the working environment of a remote robot and the motion state data of an end effector; environment three-dimensional modeling is carried out based on visual information, a three-dimensional environment model containing obstacles is generated, and a virtual protection area surrounding an end effector is dynamically generated based on end speed parameters in motion state data and system communication processing time delay; wherein the spatial range of the virtual protection area is expanded along with the increase of the terminal speed and the system communication processing time delay; performing collision risk assessment on the virtual protection area and an obstacle in the three-dimensional environment model to obtain a current collision risk level; and generating early warning information based on the collision risk level, and outputting a vision enhancement image through an operator terminal to guide an operator to avoid the collision risk based on the early warning information. According to the invention, the teleoperation safety can be improved.

The invention discloses a position-limited master-slave heterogeneous bilateral teleoperation control method based on an improved wave variable, a storage medium and a robot, and the method comprises the steps: building an improved wave variable algorithm framework, and adding a compensation wave to a slave input wave; a master robot task space reference trajectory and mapping of positions in a master-slave robot task space are constructed, and a master-slave robot joint space reference trajectory is solved based on a closed-loop inverse kinematics algorithm; based on a human operator, a far-end environment and master-slave robot characteristics, a combined robot dynamic model in a joint space is constructed; and aiming at the combined robot dynamic model, designing an adaptive neural network controller under limited joint positions based on a state transfer function. By means of the method, high-precision safety control over teleoperation of the master-slave heterogeneous robot with the limited position is effectively achieved, the stability and transparency of a teleoperation system are improved based on the designed improved wave variable algorithm, and a new safety operation method is provided for teleoperation control of the robot.

The invention provides a robot teleoperation control method and system, a medium, equipment and a program product. The method comprises the following steps: receiving multi-degree-of-freedom control input information from teleoperation equipment; based on a one-to-one correspondence mapping relationship between control input information for each degree of freedom in the multi-degree-of-freedom control input information and each degree of freedom of a plurality of controllable parts in the robot, generating a control instruction for controlling each degree of freedom in the controllable parts; the mapping relation is represented by one or more of a mapping formula and a mapping model; and sending the control instruction to the robot to control the robot to execute an operation corresponding to the control instruction. The control input information aiming at each degree of freedom and the response between each degree of freedom of the controllable component are respectively and independently mapped, so that the defect that refined cooperative control cannot be realized due to a single mapping relation in a related method is reduced, independent and accurate control of the multi-degree-of-freedom component of the robot is realized, and the operation flexibility is improved.

The present invention discloses an immersive robot teleoperation method and system, which belongs to the field of robot teleoperation technology. Existing teleoperation schemes lack an effective and timely feedback mechanism, have poor intuitive operation, low control accuracy, and are difficult to perform complex and delicate operation tasks. An immersive robot teleoperation method of the present invention, by constructing a first-person mapping model, an interactive data acquisition model, a teleoperation processing model, an execution perception model, and an immersive perspective simulation model, can map the operator's posture and movements in real time, and generate a real-time responsive three-dimensional image that can be displayed in a VR device, forming an effective and timely first-person perspective immersive feedback mechanism, which can make the operator perform tasks as if they were in person, improve the immersion, intuitiveness and naturalness of the interaction, thereby significantly improving the control ability and task execution effect, improving the operation efficiency and accuracy, and thus being able to perform complex and delicate operation tasks.

The invention belongs to the related technical field of robot teleoperation, and discloses a method and a system for calculating expected speed by using a hidden Markov model and double-layer Bayesian in teleoperation. The method comprises the following steps: acquiring an off-line motion track in off-line motion of a master-end robot, and performing off-line training on a hidden Markov model by utilizing the off-line motion track until the hidden Markov model is converged, so as to determine model parameters in the hidden Markov model; inputting the movement speed of the master-end robot at the current moment into the trained hidden Markov model to obtain hidden Markov model parameters; calculating the occurrence probability of each hidden state at the current moment and the next moment by using a Bayesian formula; and the expected speed of the slave end robot at the current moment is calculated by using the occurrence probability of each hidden state at the next moment. According to the invention, the problem of hand shake suppression in a complex track teleoperation task is solved.

The invention discloses a low-cost robot imitation learning method and system based on a human video. The method comprises the following steps: S1, data acquisition; s2, data extraction and physical alignment are carried out to eliminate man-machine physical differences; the step is divided into two parallel processing modules of action space alignment and visual space alignment; s3, data set construction: mixing the aligned human data with real robot teleoperation data, carrying out balanced sampling, and constructing a mixed data set Dmix; and S4, cooperative training: constructing a strategy network based on diffusion Transform for training. According to the method, data can be acquired only through the monocular RGB camera, expensive robot teleoperation data are replaced with cheap and easily available human videos, and the data acquisition threshold is greatly reduced. Through a visual alignment strategy of random color grid rendering, a network can learn neglect skin color textures and pay attention to geometric structures without a complex generative model, so that the robot can be seamlessly migrated to robots in different forms.

The invention discloses a man-machine collaborative robot teleoperation method and system based on visual interaction, and belongs to the technical field of robot control and interaction. In order to solve the problems that a traditional teleoperation instruction is not clear, execution is not verified and dependence on a network is high, the method comprises the steps that a robot video stream is displayed on a client; determining a target two-dimensional coordinate in response to a click operation of a user on the video picture; and the coordinates and the three-dimensional environment data space are mapped, and the execution pose of the robot is calculated and verified. The system allows a user to visually specify a target through frame selection / click selection, and a background automatically converts two-dimensional interaction information into an accurate three-dimensional robot instruction subjected to feasibility verification. According to the method, the coordinate precision is guaranteed by introducing the static reference image, batch planning and submission are supported through the task queue to reduce network dependence, and safe, accurate and high-robustness man-machine collaborative operation is realized.

The invention provides a robot learning data set construction method and related equipment, and relates to the technical field of humanoid robots. According to the method, a robot operator remotely operates the humanoid robot to execute various operation tasks based on inertial motion capture equipment, and real-time operation images and real-time joint motion data of the humanoid robot in the process of executing the operation tasks under manual guidance are collected to construct a data set; through organic combination of a human body posture direct extraction characteristic of inertial motion capture equipment and task execution flexibility and task execution reliability of robot teleoperation, construction efficiency, data set validity and data set integrity of a task learning data set are improved; and generalization and accuracy of simulation learning of the humanoid robot can be further improved.

The invention belongs to the technical field of robot teleoperation and haptic interaction, and provides a multi-mode force feedback control method and system for nuclear facility robot operation, and the method comprises the steps: collecting the state information of a master arm and a slave arm in real time, and dynamically scheduling a haptic calculation strategy according to a task state; a fast estimation strategy is enabled in the fast transit mode. Further, real environment interaction force is obtained based on dynamic model compensation and net interaction force extraction; combining the dynamically selected multi-mode force feedback parameters, and synthesizing and outputting a touch instruction to the main arm. Meanwhile, admittance correction and feed-forward compensation are carried out on a slave arm motion instruction through net interaction force and dynamic internal force, synchronous output is carried out after safety optimization, and the slave arm is controlled to move. The problems of poor task adaptability, force sense distortion and insufficient safety in nuclear facility teleoperation are solved, and intelligent, high-fidelity and safe robot teleoperation is achieved.

The invention provides a man-machine interaction robot teleoperation control system and method based on data gloves. The man-machine interaction robot teleoperation control system comprises a data acquisition module, a data processing module and a robot execution module. The data acquisition module acquires and sends finger bending degree and palm posture information; the data processing module pre-processes the collected information, performs gesture recognition and pose mapping based on the pre-processed information, and generates and sends a control instruction; and the robot execution module receives the control instruction and controls the robot to execute corresponding actions. The technical problem that an existing robot control system based on data gloves is insufficient in recognition precision, real-time performance and system compatibility is solved.

The application provides a dexterous arm hand isomeric teleoperation action mapping method and system, and relates to the technical field of robot teleoperation. The method comprises the following steps: collecting three-dimensional space position data of human hand key points; constructing a mapping function from the three-dimensional space position data of the human hand key points to robot joint angles; wherein the robot comprises a mechanical arm and a dexterous hand; constructing a comprehensive weighted optimization objective function, which is a weighted sum of multiple optimization sub-target items; optimizing the mapping function according to the three-dimensional space position data of the human hand key points with the comprehensive weighted optimization objective function as the optimization criterion to obtain an optimized mapping function; in real-time teleoperation, inputting the real-time collected three-dimensional space position data of the human hand key points into the optimized mapping function to obtain target joint angles of the robot, so as to drive the robot to perform corresponding actions.

The invention relates to the technical field of XR operation, and discloses a WebXR-based panoramic immersive robot teleoperation system and method. The method comprises the following steps: carrying out distortion correction and splicing fusion on original images collected by a double-fisheye lens to obtain a panoramic image frame; encoding the panoramic image frame into a video code stream, transmitting the video code stream to a browser end, and decoding the video code stream at the browser end to obtain a panoramic video texture; mapping the panoramic video texture to the inner surface of a sphere, setting a virtual camera at the center of the sphere, and constructing a panoramic sphere rendering model; and converting the attitude quaternion of the VR head-mounted display into a sight decoupling rotation matrix, performing local GPU rendering on the panoramic sphere rendering model to obtain a real-time visual angle picture, and mapping a rocker shaft value of the gamepad into a control instruction to be sent to a robot dog end. In the whole view angle switching process, vertex transformation and texture sampling are completed at the browser end, no pan-tilt rotation instruction is generated or sent to the robot dog end through the network, and delay is reduced.

The invention discloses a wearable teleoperation master hand, and belongs to the technical field of robot teleoperation and wearable equipment. The main hand comprises main glove sets symmetrically arranged on the two sides of a carbon fiber tube, each set comprises a shoulder joint, an elbow joint, a wrist joint and an adjustable large / small arm wearing structure, and each joint is integrated with an angle sensor to collect motion data in real time. The master manipulator adapts to different users through the length adjusting device, and naturalness and precision of teleoperation are improved. The upper limb motion information of an operator is captured through lightweight wearable design, and a more visual and efficient input mode is provided for remote control of the robot.

The present invention belongs to the technical field related to robot teleoperation, and discloses a robot teleoperation safety control method and system that uses operation intention recognition and adaptive control obstacle functions. The method includes: establishing a virtual agent between the teleoperated master robot and slave robot; calculating the nominal control input of the virtual agent, converting the nominal control input into a safety control input using an adaptive control obstacle function as a constraint, and using the safety control input to control the virtual agent to track the master robot according to a preset desired tracking trajectory; setting the slack variables in the adaptive control obstacle function according to the operator's interaction intention with the target; calculating the expected control input of the slave robot, and using this expected control to control the slave robot to track the virtual agent, thereby achieving teleoperation control of the slave robot by the master robot. The present invention solves the problem of being unable to achieve safety control functions for different operator interaction intentions with the target within a unified control framework.

The invention discloses a remote laser repair method based on a virtual fixture and an autonomous agent, and belongs to the technical field of robot teleoperation. According to the method, the virtual clamp is designed to provide force guidance and motion constraint for an operator; secondly, determining a normal vector of a repair point according to the point cloud data of the to-be-repaired workpiece, and predicting an optimal repair angle and height according to a molten pool state and laser power; and finally, automatically determining the joint angle of the mechanical arm by using an autonomous agent so as to maintain the repair angle and height. According to the invention, the operation difficulty of remote repair is reduced, the training time is shortened, and the effectiveness of remote operation is improved.