107 results about "Robot teleoperation" patented technology

The invention provides a robot teleoperation method and system, a storage medium, equipment and a program product. The method comprises the following steps: receiving skeleton movement data of an operator, selecting pose information of part of skeleton nodes from the skeleton movement data, and taking the pose information as reference movement data; mapping the reference motion data into target motion information of the robot; and solving based on the target motion information to generate a control instruction for driving the robot joint. Therefore, the pose information of part of the skeleton nodes is selected from the skeleton motion data and is taken as the reference motion data, so that core motion characteristics can be reserved, the complexity of mapping from the skeleton motion data to the target motion information of the robot in the related technology is simplified, redundant data interference is reduced, and the reliability of the robot is improved. And therefore, the limited joint freedom degree of the robot can be adapted.

The invention provides a construction method for a robot teleoperation body-sized model and a robot teleoperation system, and the method comprises the steps: carrying out the training and fine adjustment of a pre-trained body-sized model through master-slave robot control data, and obtaining a fine-adjusted body-sized model; deploying the fine-tuning body large model to provide robot control service, and collecting real machine execution data; obtaining scores of the real machine execution data, forming reinforcement learning training data pairs by the real machine execution data and the corresponding scores, and adding the reinforcement learning training data pairs into a reinforcement training data set; and performing reinforcement learning training on the fine-tuning body size model by using the reinforcement training data set. According to the method, the pre-training model is finely adjusted, the cost of manual intervention is effectively reduced, the reinforcement learning training data is collected by deploying the fine adjustment body model, the reinforcement learning method and the pre-training body model are organically combined, the advantages of the reinforcement learning method and the pre-training body model are fully played, and efficient learning and performance improvement of the robot in a complex environment are achieved.

The invention discloses an immersive robot teleoperation method and system, and belongs to the technical field of robot teleoperation. An existing teleoperation scheme lacks an effective and timely feedback mechanism, is poor in operation intuition and low in control precision, and is difficult to compete with complex and fine operation tasks. According to the immersive robot teleoperation method, by constructing the first person mapping model, the interaction data acquisition model, the teleoperation processing model, the execution perception model and the immersive visual angle simulation model, real-time mapping can be performed on posture actions of an operator, and a real-time response three-dimensional image capable of being displayed in VR equipment is generated; an effective and timely immersive feedback mechanism from a first person perspective is formed, an operator can execute a task as like being personally on the scene, and the immersion, intuition and naturalness of interaction are improved, so that the control capability and the task execution effect can be remarkably improved, the operation efficiency and precision are improved, and then the operator can competent for complex and fine operation tasks.

The invention provides a robot teleoperation method and a robot. The robot teleoperation method comprises the steps that an acceleration command, an angular acceleration command and an expected attitude quaternion of an end effector are determined based on the current linear acceleration, the current angular acceleration and the current attitude quaternion of a control end; the current position and the current speed fed back by the end effector are obtained, a torque control signal for controlling the position of the end effector is determined based on the acceleration command, the current position and the current speed, and end acceleration feedforward is obtained; determining a torque control signal for controlling the attitude of the end effector based on the angular acceleration command and the expected attitude quaternion to obtain attitude feedforward; and a joint torque instruction of a mechanical arm joint corresponding to the end effector is determined based on the end acceleration feedforward and the posture feedforward, and the joint torque instruction is sent to a corresponding joint driver so that the end effector can synchronously act along with the control end. According to the invention, the problem of attitude lag of the end effector can be avoided, and the tracking error of the end effector is reduced.

The invention relates to a robot teleoperation algorithm system based on motion capture equipment. The system comprises a motion capture data receiving module, a pose conversion module, a safety monitoring module and a control instruction issuing module. Teleoperation is carried out by using a mode of converting data of the motion capture equipment into a robot control instruction, the efficiency of completing a specific task by the teleoperation robot can be greatly improved, multiple control modes are realized, and an operator can more intuitively operate the robot to complete the task; meanwhile, the safety monitoring module designed in the system can ensure the safety of operators and the robot when the motion capture data is abnormal or drifts, and has high robustness. Moreover, the system can achieve the adaptation to different robots, and can reduce the adaptation development amount during hardware migration.

The invention relates to a teleoperation sharing control method and device for an electric robot and a medium, and the method comprises the steps: selecting a key point on a mechanical arm of the electric robot, and building a kinematics state expression of an end effector of the mechanical arm and the key point in teleoperation; based on the kinematics state expression, considering the influence of obstacles, and combining a control obstacle function to construct safety constraints of teleoperation of the power robot; acquiring a reference trajectory of the mechanical arm of the electric robot set by an operator, introducing the safety constraint into nonlinear model predictive control, and constructing an NMPC-CBF controller to track the reference trajectory; and calculating the resultant force fed back by the repulsive force of the mechanical arm of the electric robot and the obstacle by adopting an artificial potential field method, and correcting the reference track according to the resultant force to complete the sharing control process. Compared with the prior art, the method has the advantages of efficient obstacle avoidance and the like.

The invention relates to the technical field of robots, and discloses a collision early warning method and device in robot teleoperation, and the method comprises the steps: obtaining the visual information of the working environment of a remote robot and the motion state data of an end effector; environment three-dimensional modeling is carried out based on visual information, a three-dimensional environment model containing obstacles is generated, and a virtual protection area surrounding an end effector is dynamically generated based on end speed parameters in motion state data and system communication processing time delay; wherein the spatial range of the virtual protection area is expanded along with the increase of the terminal speed and the system communication processing time delay; performing collision risk assessment on the virtual protection area and an obstacle in the three-dimensional environment model to obtain a current collision risk level; and generating early warning information based on the collision risk level, and outputting a vision enhancement image through an operator terminal to guide an operator to avoid the collision risk based on the early warning information. According to the invention, the teleoperation safety can be improved.

The invention discloses a position-limited master-slave heterogeneous bilateral teleoperation control method based on an improved wave variable, a storage medium and a robot, and the method comprises the steps: building an improved wave variable algorithm framework, and adding a compensation wave to a slave input wave; a master robot task space reference trajectory and mapping of positions in a master-slave robot task space are constructed, and a master-slave robot joint space reference trajectory is solved based on a closed-loop inverse kinematics algorithm; based on a human operator, a far-end environment and master-slave robot characteristics, a combined robot dynamic model in a joint space is constructed; and aiming at the combined robot dynamic model, designing an adaptive neural network controller under limited joint positions based on a state transfer function. By means of the method, high-precision safety control over teleoperation of the master-slave heterogeneous robot with the limited position is effectively achieved, the stability and transparency of a teleoperation system are improved based on the designed improved wave variable algorithm, and a new safety operation method is provided for teleoperation control of the robot.

The invention provides a robot teleoperation control method and system, a medium, equipment and a program product. The method comprises the following steps: receiving multi-degree-of-freedom control input information from teleoperation equipment; based on a one-to-one correspondence mapping relationship between control input information for each degree of freedom in the multi-degree-of-freedom control input information and each degree of freedom of a plurality of controllable parts in the robot, generating a control instruction for controlling each degree of freedom in the controllable parts; the mapping relation is represented by one or more of a mapping formula and a mapping model; and sending the control instruction to the robot to control the robot to execute an operation corresponding to the control instruction. The control input information aiming at each degree of freedom and the response between each degree of freedom of the controllable component are respectively and independently mapped, so that the defect that refined cooperative control cannot be realized due to a single mapping relation in a related method is reduced, independent and accurate control of the multi-degree-of-freedom component of the robot is realized, and the operation flexibility is improved.

The present invention discloses an immersive robot teleoperation method and system, which belongs to the field of robot teleoperation technology. Existing teleoperation schemes lack an effective and timely feedback mechanism, have poor intuitive operation, low control accuracy, and are difficult to perform complex and delicate operation tasks. An immersive robot teleoperation method of the present invention, by constructing a first-person mapping model, an interactive data acquisition model, a teleoperation processing model, an execution perception model, and an immersive perspective simulation model, can map the operator's posture and movements in real time, and generate a real-time responsive three-dimensional image that can be displayed in a VR device, forming an effective and timely first-person perspective immersive feedback mechanism, which can make the operator perform tasks as if they were in person, improve the immersion, intuitiveness and naturalness of the interaction, thereby significantly improving the control ability and task execution effect, improving the operation efficiency and accuracy, and thus being able to perform complex and delicate operation tasks.

The invention discloses a low-cost robot imitation learning method and system based on a human video. The method comprises the following steps: S1, data acquisition; s2, data extraction and physical alignment are carried out to eliminate man-machine physical differences; the step is divided into two parallel processing modules of action space alignment and visual space alignment; s3, data set construction: mixing the aligned human data with real robot teleoperation data, carrying out balanced sampling, and constructing a mixed data set Dmix; and S4, cooperative training: constructing a strategy network based on diffusion Transform for training. According to the method, data can be acquired only through the monocular RGB camera, expensive robot teleoperation data are replaced with cheap and easily available human videos, and the data acquisition threshold is greatly reduced. Through a visual alignment strategy of random color grid rendering, a network can learn neglect skin color textures and pay attention to geometric structures without a complex generative model, so that the robot can be seamlessly migrated to robots in different forms.

The invention discloses a man-machine collaborative robot teleoperation method and system based on visual interaction, and belongs to the technical field of robot control and interaction. In order to solve the problems that a traditional teleoperation instruction is not clear, execution is not verified and dependence on a network is high, the method comprises the steps that a robot video stream is displayed on a client; determining a target two-dimensional coordinate in response to a click operation of a user on the video picture; and the coordinates and the three-dimensional environment data space are mapped, and the execution pose of the robot is calculated and verified. The system allows a user to visually specify a target through frame selection / click selection, and a background automatically converts two-dimensional interaction information into an accurate three-dimensional robot instruction subjected to feasibility verification. According to the method, the coordinate precision is guaranteed by introducing the static reference image, batch planning and submission are supported through the task queue to reduce network dependence, and safe, accurate and high-robustness man-machine collaborative operation is realized.

The invention belongs to the technical field of robot teleoperation and haptic interaction, and provides a multi-mode force feedback control method and system for nuclear facility robot operation, and the method comprises the steps: collecting the state information of a master arm and a slave arm in real time, and dynamically scheduling a haptic calculation strategy according to a task state; a fast estimation strategy is enabled in the fast transit mode. Further, real environment interaction force is obtained based on dynamic model compensation and net interaction force extraction; combining the dynamically selected multi-mode force feedback parameters, and synthesizing and outputting a touch instruction to the main arm. Meanwhile, admittance correction and feed-forward compensation are carried out on a slave arm motion instruction through net interaction force and dynamic internal force, synchronous output is carried out after safety optimization, and the slave arm is controlled to move. The problems of poor task adaptability, force sense distortion and insufficient safety in nuclear facility teleoperation are solved, and intelligent, high-fidelity and safe robot teleoperation is achieved.

The invention provides a man-machine interaction robot teleoperation control system and method based on data gloves. The man-machine interaction robot teleoperation control system comprises a data acquisition module, a data processing module and a robot execution module. The data acquisition module acquires and sends finger bending degree and palm posture information; the data processing module pre-processes the collected information, performs gesture recognition and pose mapping based on the pre-processed information, and generates and sends a control instruction; and the robot execution module receives the control instruction and controls the robot to execute corresponding actions. The technical problem that an existing robot control system based on data gloves is insufficient in recognition precision, real-time performance and system compatibility is solved.

The application provides a dexterous arm hand isomeric teleoperation action mapping method and system, and relates to the technical field of robot teleoperation. The method comprises the following steps: collecting three-dimensional space position data of human hand key points; constructing a mapping function from the three-dimensional space position data of the human hand key points to robot joint angles; wherein the robot comprises a mechanical arm and a dexterous hand; constructing a comprehensive weighted optimization objective function, which is a weighted sum of multiple optimization sub-target items; optimizing the mapping function according to the three-dimensional space position data of the human hand key points with the comprehensive weighted optimization objective function as the optimization criterion to obtain an optimized mapping function; in real-time teleoperation, inputting the real-time collected three-dimensional space position data of the human hand key points into the optimized mapping function to obtain target joint angles of the robot, so as to drive the robot to perform corresponding actions.

The invention relates to the technical field of XR operation, and discloses a WebXR-based panoramic immersive robot teleoperation system and method. The method comprises the following steps: carrying out distortion correction and splicing fusion on original images collected by a double-fisheye lens to obtain a panoramic image frame; encoding the panoramic image frame into a video code stream, transmitting the video code stream to a browser end, and decoding the video code stream at the browser end to obtain a panoramic video texture; mapping the panoramic video texture to the inner surface of a sphere, setting a virtual camera at the center of the sphere, and constructing a panoramic sphere rendering model; and converting the attitude quaternion of the VR head-mounted display into a sight decoupling rotation matrix, performing local GPU rendering on the panoramic sphere rendering model to obtain a real-time visual angle picture, and mapping a rocker shaft value of the gamepad into a control instruction to be sent to a robot dog end. In the whole view angle switching process, vertex transformation and texture sampling are completed at the browser end, no pan-tilt rotation instruction is generated or sent to the robot dog end through the network, and delay is reduced.

The invention discloses a wearable teleoperation master hand, and belongs to the technical field of robot teleoperation and wearable equipment. The main hand comprises main glove sets symmetrically arranged on the two sides of a carbon fiber tube, each set comprises a shoulder joint, an elbow joint, a wrist joint and an adjustable large / small arm wearing structure, and each joint is integrated with an angle sensor to collect motion data in real time. The master manipulator adapts to different users through the length adjusting device, and naturalness and precision of teleoperation are improved. The upper limb motion information of an operator is captured through lightweight wearable design, and a more visual and efficient input mode is provided for remote control of the robot.

The present invention belongs to the technical field related to robot teleoperation, and discloses a robot teleoperation safety control method and system that uses operation intention recognition and adaptive control obstacle functions. The method includes: establishing a virtual agent between the teleoperated master robot and slave robot; calculating the nominal control input of the virtual agent, converting the nominal control input into a safety control input using an adaptive control obstacle function as a constraint, and using the safety control input to control the virtual agent to track the master robot according to a preset desired tracking trajectory; setting the slack variables in the adaptive control obstacle function according to the operator's interaction intention with the target; calculating the expected control input of the slave robot, and using this expected control to control the slave robot to track the virtual agent, thereby achieving teleoperation control of the slave robot by the master robot. The present invention solves the problem of being unable to achieve safety control functions for different operator interaction intentions with the target within a unified control framework.

This invention provides a VR-based dual-arm robot teleoperation method and system, comprising: parallel calculation of left and right arm joint angles based on the VR target pose and the robot's current joint actual position; then checking the remaining steps in the left and right arm execution buffers; if the remaining steps in the left and right arm execution buffers are less than a set value, performing a trajectory mixing and generation step based on the joint angles of the left and right arms to obtain a new mixed trajectory segment for supplementation; calculating the position error of the left and right arms based on the robot's current shutdown position and dynamically adjusting the trajectory generation mixing window; wherein, in the trajectory mixing and generation step, the new trajectory is shifted forward by the sum of inference delay and processing delay to align with the old trajectory time. This invention dynamically adjusts the trajectory cutoff point through an adaptive delay compensation mechanism to avoid using "expired" instructions; and ensures that the execution buffer always has sufficient trajectory through an intelligent buffer management mechanism, completely eliminating trajectory breaks caused by IK delay fluctuations.

The invention discloses a robot teleoperation control method based on non-delay data training in a delay environment, and the method comprises the following steps: S1, collecting an offline data set obtained when a robot executes a task in a non-delay environment, the offline data set comprising a plurality of tracks, and each track comprising a state sequence, an action sequence and a reward sequence; s2, reconstructing a state sequence and an action sequence in the offline data set to generate an information state sequence suitable for a delay environment; s3, performing state estimation on the information state sequence to obtain a prediction state sequence; s4, based on the information state sequence and the prediction state sequence, an offline reinforcement learning algorithm is adopted to train a strategy model, and the strategy model is used for outputting an action instruction for controlling the robot based on the prediction state sequence in a delay environment; and S5, controlling the robot to execute a teleoperation task by adopting the trained strategy model in a delayed environment. According to the invention, safe, efficient and reliable control of robot teleoperation in a delayed environment is realized.

The embodiment of the invention provides a robot teleoperation method, device and system, electronic equipment and a computer readable storage medium, and relates to the technical field of robots. The method comprises the steps that based on a handle motion signal of an operation end, a robot instruction clamping jaw pose at the current moment is determined; according to the robot instruction clamping jaw pose and a locally pre-stored first spatial transformation matrix, a virtual clamping jaw predicted pose at the current moment is determined; determining a virtual camera pose according to the pose of the robot head camera in the world coordinate system at the current moment and a locally pre-stored second spatial transformation matrix; and performing clamping jaw rendering locally according to the virtual clamping jaw prediction pose and the virtual camera pose.

1. Name of the product in this design: Robot Remote Control. 2. Purpose of this design: For use as a robot remote control. 3. The key design point of this design product lies in the combination of shape and pattern. 4. The image or photograph that best illustrates the design's key points: a 3D model.

The invention discloses a dynamic compensation multi-modal interaction control device and method based on a gravity center pressure field, and relates to the field of multi-degree-of-freedom robot control and VR interaction equipment. The device comprises a pedal assembly, an embedded control unit, a dynamic compensation mechanism and a communication module, can collect plantar pressure distribution data of an operator in real time, calculate the gravity center track of a human body and generate a corresponding control signal to control the action of external equipment, and has a dynamic compensation function to maintain the balance of the operator. According to the method, through the steps of pressure data collection, data processing and gravity center track calculation, control signal generation, dynamic compensation adjustment, mode switching and the like, natural, visual and efficient control over robot chassis movement and virtual character displacement is achieved; the problems that in the prior art, robot remote control operation is complex, learning cost is high, VR interaction is high in dizziness, and the equipment size is large are solved, and the wide application prospect is achieved.

The present application relates to the technical field of heterogeneous robot teleoperation, and particularly relates to a flexible motion mapping method and system for heterogeneous robots, wherein the method comprises the following steps: collecting joint motion data of a master end follow-up teach box; extracting core task components corresponding to a task space of a slave end executor, identifying and separating redundant action components; converting at least one motion signal in the redundant action components into a driving instruction of the slave end executor according to a flexible mapping rule; performing safety constraint processing on the driving instruction, receiving the processed instruction by the slave end executor and executing the same; obtaining physical motion parameters of the slave end executor, generating a virtual pose feedback signal conforming to a master end communication protocol by using a reverse reconstruction algorithm, and returning the same to the master end follow-up teach box. The present application can eliminate control stiffness and logic deadlock in heterogeneous teleoperation, avoid mechanical impact caused by instruction mutation, ensure that operation polarity and visual motion are highly consistent, and improve operation efficiency.

Surgical robot systems for remote manipulation having robotic telemanipulators are provided. The surgical robot systems are well adapted for use by the surgeon, seamlessly integratable into the operation room, allow for a surgeon to work between the robot and the patient throughout a surgery in a sterile manner, are relatively low cost, and / or permit integrated laparoscopy. The system preferably includes a master console having a plurality of master links interconnected by a plurality of master joints, and a handle coupled to the master console for operating the telemanipulator. The system further includes a slave console operatively coupled to the master console and having a plurality of slave links interconnected by a plurality of slave joints that move responsive to movement at the master console to permit an end-effector to perform surgery.

The invention discloses a pose mapping method based on a drift function and a synchronous control system, and belongs to the technical field of robot teleoperation and intelligent control. According to the method, a heterogeneous coordinate system mapping relation between a near-end VR head-mounted display and a far-end observation robot is established, a drift function activated based on normalized operability is introduced, and an unreachable area and a kinematics singular configuration are dynamically avoided; the system adopts a double-layer architecture of outer ring mapping and inner ring PID control, and high-continuity and high-precision pose synchronization is realized. According to the invention, accurate pose control of the far-end observation robot by a near-end operator can be realized, and teleoperation safety and telepresence are improved.

This disclosure provides a teleoperation compensation and teleoperation method, apparatus, and electronic device. The teleoperation compensation method includes: at the slave control end of the teleoperation system, modifying the control model of the teleoperation system based on the robot's actual state parameters without network latency and the robot's physical configuration parameters; at the master control end of the teleoperation system, compensating for the robot's network interference torque based on the robot's feedback state parameters including network latency, and further modifying the modified control model; based on the secondary modified control model and the compensated network interference torque, compensating for the robot's feedback state parameters to obtain the robot's target state parameters. This disclosure enables robot teleoperation to possess the advantages of high precision, strong robustness, and low latency.

The present invention provides a master-slave isomorphic robot teleoperation safety control method and system. The method collects an environmental image of the slave robot's workspace and determines whether there are obstacles. If there are obstacles, the method calculates the expected acceleration of the slave robot's end caused by virtual gravity and repulsion and maps it to the joint space to obtain the expected angular acceleration of each joint, and then maps the expected angular acceleration to the joint space. Simultaneously, the expected acceleration of the slave robot's mobile chassis caused by virtual gravity and repulsion is calculated and applied to the motion space. If there are no obstacles, the expected angular acceleration of each joint of the slave robot is calculated and applied to the joint space of the slave robot. Simultaneously, the expected acceleration of the slave robot's mobile chassis caused by virtual gravity is calculated and applied to the motion space. By combining robot teleoperation with obstacle avoidance, the method allows the slave robot to safely avoid obstacles while stably following the motion trajectory of the master robot, achieving the goal of synchronized motion between the master and slave robots.

1. The name of the product of this design: coating robot remote control. 2. Purpose of this design product: used for coating the remote control of the robot. 3. The design key point of this appearance design product lies in the shape. 4. The picture or photo that best illustrates the design points: Stereoscopic diagram 1.

The invention discloses a robot teleoperation method based on fusion of virtual reality and mixed reality. The robot teleoperation method comprises the following steps: sensing a far-end environment by observing a robot; constructing a near-end interactive virtual environment, and presenting the virtual environment to a near-end operator through a virtual reality device; constructing three-dimensional point cloud data based on a far-end environment through three-dimensional reconstruction; in the virtual environment, pose estimation is carried out on the far-end operation robot and the target object, and three-dimensional registration is carried out in the virtual environment; working space boundary condition constraints and space operation pose change constraints of the working robot are determined, path planning and trajectory planning are conducted on the working robot, and visualization is conducted in the virtual environment; and generating a control program of the operation robot to move along the planned track. By constructing the interactive virtual environment, the working space range and the operation pose of the operation robot are visualized in the interactive virtual environment, and the trajectory is planned and visualized in the interactive virtual environment, so that the safety of teleoperation is greatly improved.