83 results about "Robot teleoperation" patented technology

The invention provides a construction method for a robot teleoperation body-sized model and a robot teleoperation system, and the method comprises the steps: carrying out the training and fine adjustment of a pre-trained body-sized model through master-slave robot control data, and obtaining a fine-adjusted body-sized model; deploying the fine-tuning body large model to provide robot control service, and collecting real machine execution data; obtaining scores of the real machine execution data, forming reinforcement learning training data pairs by the real machine execution data and the corresponding scores, and adding the reinforcement learning training data pairs into a reinforcement training data set; and performing reinforcement learning training on the fine-tuning body size model by using the reinforcement training data set. According to the method, the pre-training model is finely adjusted, the cost of manual intervention is effectively reduced, the reinforcement learning training data is collected by deploying the fine adjustment body model, the reinforcement learning method and the pre-training body model are organically combined, the advantages of the reinforcement learning method and the pre-training body model are fully played, and efficient learning and performance improvement of the robot in a complex environment are achieved.

The invention relates to a robot teleoperation algorithm system based on motion capture equipment. The system comprises a motion capture data receiving module, a pose conversion module, a safety monitoring module and a control instruction issuing module. Teleoperation is carried out by using a mode of converting data of the motion capture equipment into a robot control instruction, the efficiency of completing a specific task by the teleoperation robot can be greatly improved, multiple control modes are realized, and an operator can more intuitively operate the robot to complete the task; meanwhile, the safety monitoring module designed in the system can ensure the safety of operators and the robot when the motion capture data is abnormal or drifts, and has high robustness. Moreover, the system can achieve the adaptation to different robots, and can reduce the adaptation development amount during hardware migration.

The invention relates to the technical field of robots, and discloses a collision early warning method and device in robot teleoperation, and the method comprises the steps: obtaining the visual information of the working environment of a remote robot and the motion state data of an end effector; environment three-dimensional modeling is carried out based on visual information, a three-dimensional environment model containing obstacles is generated, and a virtual protection area surrounding an end effector is dynamically generated based on end speed parameters in motion state data and system communication processing time delay; wherein the spatial range of the virtual protection area is expanded along with the increase of the terminal speed and the system communication processing time delay; performing collision risk assessment on the virtual protection area and an obstacle in the three-dimensional environment model to obtain a current collision risk level; and generating early warning information based on the collision risk level, and outputting a vision enhancement image through an operator terminal to guide an operator to avoid the collision risk based on the early warning information. According to the invention, the teleoperation safety can be improved.

The present invention discloses an immersive robot teleoperation method and system, which belongs to the field of robot teleoperation technology. Existing teleoperation schemes lack an effective and timely feedback mechanism, have poor intuitive operation, low control accuracy, and are difficult to perform complex and delicate operation tasks. An immersive robot teleoperation method of the present invention, by constructing a first-person mapping model, an interactive data acquisition model, a teleoperation processing model, an execution perception model, and an immersive perspective simulation model, can map the operator's posture and movements in real time, and generate a real-time responsive three-dimensional image that can be displayed in a VR device, forming an effective and timely first-person perspective immersive feedback mechanism, which can make the operator perform tasks as if they were in person, improve the immersion, intuitiveness and naturalness of the interaction, thereby significantly improving the control ability and task execution effect, improving the operation efficiency and accuracy, and thus being able to perform complex and delicate operation tasks.

The invention discloses a low-cost robot imitation learning method and system based on a human video. The method comprises the following steps: S1, data acquisition; s2, data extraction and physical alignment are carried out to eliminate man-machine physical differences; the step is divided into two parallel processing modules of action space alignment and visual space alignment; s3, data set construction: mixing the aligned human data with real robot teleoperation data, carrying out balanced sampling, and constructing a mixed data set Dmix; and S4, cooperative training: constructing a strategy network based on diffusion Transform for training. According to the method, data can be acquired only through the monocular RGB camera, expensive robot teleoperation data are replaced with cheap and easily available human videos, and the data acquisition threshold is greatly reduced. Through a visual alignment strategy of random color grid rendering, a network can learn neglect skin color textures and pay attention to geometric structures without a complex generative model, so that the robot can be seamlessly migrated to robots in different forms.

The invention discloses a man-machine collaborative robot teleoperation method and system based on visual interaction, and belongs to the technical field of robot control and interaction. In order to solve the problems that a traditional teleoperation instruction is not clear, execution is not verified and dependence on a network is high, the method comprises the steps that a robot video stream is displayed on a client; determining a target two-dimensional coordinate in response to a click operation of a user on the video picture; and the coordinates and the three-dimensional environment data space are mapped, and the execution pose of the robot is calculated and verified. The system allows a user to visually specify a target through frame selection / click selection, and a background automatically converts two-dimensional interaction information into an accurate three-dimensional robot instruction subjected to feasibility verification. According to the method, the coordinate precision is guaranteed by introducing the static reference image, batch planning and submission are supported through the task queue to reduce network dependence, and safe, accurate and high-robustness man-machine collaborative operation is realized.

The invention belongs to the technical field of robot teleoperation and haptic interaction, and provides a multi-mode force feedback control method and system for nuclear facility robot operation, and the method comprises the steps: collecting the state information of a master arm and a slave arm in real time, and dynamically scheduling a haptic calculation strategy according to a task state; a fast estimation strategy is enabled in the fast transit mode. Further, real environment interaction force is obtained based on dynamic model compensation and net interaction force extraction; combining the dynamically selected multi-mode force feedback parameters, and synthesizing and outputting a touch instruction to the main arm. Meanwhile, admittance correction and feed-forward compensation are carried out on a slave arm motion instruction through net interaction force and dynamic internal force, synchronous output is carried out after safety optimization, and the slave arm is controlled to move. The problems of poor task adaptability, force sense distortion and insufficient safety in nuclear facility teleoperation are solved, and intelligent, high-fidelity and safe robot teleoperation is achieved.

The invention provides a man-machine interaction robot teleoperation control system and method based on data gloves. The man-machine interaction robot teleoperation control system comprises a data acquisition module, a data processing module and a robot execution module. The data acquisition module acquires and sends finger bending degree and palm posture information; the data processing module pre-processes the collected information, performs gesture recognition and pose mapping based on the pre-processed information, and generates and sends a control instruction; and the robot execution module receives the control instruction and controls the robot to execute corresponding actions. The technical problem that an existing robot control system based on data gloves is insufficient in recognition precision, real-time performance and system compatibility is solved.

The application provides a dexterous arm hand isomeric teleoperation action mapping method and system, and relates to the technical field of robot teleoperation. The method comprises the following steps: collecting three-dimensional space position data of human hand key points; constructing a mapping function from the three-dimensional space position data of the human hand key points to robot joint angles; wherein the robot comprises a mechanical arm and a dexterous hand; constructing a comprehensive weighted optimization objective function, which is a weighted sum of multiple optimization sub-target items; optimizing the mapping function according to the three-dimensional space position data of the human hand key points with the comprehensive weighted optimization objective function as the optimization criterion to obtain an optimized mapping function; in real-time teleoperation, inputting the real-time collected three-dimensional space position data of the human hand key points into the optimized mapping function to obtain target joint angles of the robot, so as to drive the robot to perform corresponding actions.

The invention relates to the technical field of XR operation, and discloses a WebXR-based panoramic immersive robot teleoperation system and method. The method comprises the following steps: carrying out distortion correction and splicing fusion on original images collected by a double-fisheye lens to obtain a panoramic image frame; encoding the panoramic image frame into a video code stream, transmitting the video code stream to a browser end, and decoding the video code stream at the browser end to obtain a panoramic video texture; mapping the panoramic video texture to the inner surface of a sphere, setting a virtual camera at the center of the sphere, and constructing a panoramic sphere rendering model; and converting the attitude quaternion of the VR head-mounted display into a sight decoupling rotation matrix, performing local GPU rendering on the panoramic sphere rendering model to obtain a real-time visual angle picture, and mapping a rocker shaft value of the gamepad into a control instruction to be sent to a robot dog end. In the whole view angle switching process, vertex transformation and texture sampling are completed at the browser end, no pan-tilt rotation instruction is generated or sent to the robot dog end through the network, and delay is reduced.

The invention discloses a wearable teleoperation master hand, and belongs to the technical field of robot teleoperation and wearable equipment. The main hand comprises main glove sets symmetrically arranged on the two sides of a carbon fiber tube, each set comprises a shoulder joint, an elbow joint, a wrist joint and an adjustable large / small arm wearing structure, and each joint is integrated with an angle sensor to collect motion data in real time. The master manipulator adapts to different users through the length adjusting device, and naturalness and precision of teleoperation are improved. The upper limb motion information of an operator is captured through lightweight wearable design, and a more visual and efficient input mode is provided for remote control of the robot.

The present invention belongs to the technical field related to robot teleoperation, and discloses a robot teleoperation safety control method and system that uses operation intention recognition and adaptive control obstacle functions. The method includes: establishing a virtual agent between the teleoperated master robot and slave robot; calculating the nominal control input of the virtual agent, converting the nominal control input into a safety control input using an adaptive control obstacle function as a constraint, and using the safety control input to control the virtual agent to track the master robot according to a preset desired tracking trajectory; setting the slack variables in the adaptive control obstacle function according to the operator's interaction intention with the target; calculating the expected control input of the slave robot, and using this expected control to control the slave robot to track the virtual agent, thereby achieving teleoperation control of the slave robot by the master robot. The present invention solves the problem of being unable to achieve safety control functions for different operator interaction intentions with the target within a unified control framework.

This invention provides a VR-based dual-arm robot teleoperation method and system, comprising: parallel calculation of left and right arm joint angles based on the VR target pose and the robot's current joint actual position; then checking the remaining steps in the left and right arm execution buffers; if the remaining steps in the left and right arm execution buffers are less than a set value, performing a trajectory mixing and generation step based on the joint angles of the left and right arms to obtain a new mixed trajectory segment for supplementation; calculating the position error of the left and right arms based on the robot's current shutdown position and dynamically adjusting the trajectory generation mixing window; wherein, in the trajectory mixing and generation step, the new trajectory is shifted forward by the sum of inference delay and processing delay to align with the old trajectory time. This invention dynamically adjusts the trajectory cutoff point through an adaptive delay compensation mechanism to avoid using "expired" instructions; and ensures that the execution buffer always has sufficient trajectory through an intelligent buffer management mechanism, completely eliminating trajectory breaks caused by IK delay fluctuations.

The invention discloses a robot teleoperation control method based on non-delay data training in a delay environment, and the method comprises the following steps: S1, collecting an offline data set obtained when a robot executes a task in a non-delay environment, the offline data set comprising a plurality of tracks, and each track comprising a state sequence, an action sequence and a reward sequence; s2, reconstructing a state sequence and an action sequence in the offline data set to generate an information state sequence suitable for a delay environment; s3, performing state estimation on the information state sequence to obtain a prediction state sequence; s4, based on the information state sequence and the prediction state sequence, an offline reinforcement learning algorithm is adopted to train a strategy model, and the strategy model is used for outputting an action instruction for controlling the robot based on the prediction state sequence in a delay environment; and S5, controlling the robot to execute a teleoperation task by adopting the trained strategy model in a delayed environment. According to the invention, safe, efficient and reliable control of robot teleoperation in a delayed environment is realized.

The embodiment of the invention provides a robot teleoperation method, device and system, electronic equipment and a computer readable storage medium, and relates to the technical field of robots. The method comprises the steps that based on a handle motion signal of an operation end, a robot instruction clamping jaw pose at the current moment is determined; according to the robot instruction clamping jaw pose and a locally pre-stored first spatial transformation matrix, a virtual clamping jaw predicted pose at the current moment is determined; determining a virtual camera pose according to the pose of the robot head camera in the world coordinate system at the current moment and a locally pre-stored second spatial transformation matrix; and performing clamping jaw rendering locally according to the virtual clamping jaw prediction pose and the virtual camera pose.

1. Name of the product in this design: Robot Remote Control. 2. Purpose of this design: For use as a robot remote control. 3. The key design point of this design product lies in the combination of shape and pattern. 4. The image or photograph that best illustrates the design's key points: a 3D model.

The present application relates to the technical field of heterogeneous robot teleoperation, and particularly relates to a flexible motion mapping method and system for heterogeneous robots, wherein the method comprises the following steps: collecting joint motion data of a master end follow-up teach box; extracting core task components corresponding to a task space of a slave end executor, identifying and separating redundant action components; converting at least one motion signal in the redundant action components into a driving instruction of the slave end executor according to a flexible mapping rule; performing safety constraint processing on the driving instruction, receiving the processed instruction by the slave end executor and executing the same; obtaining physical motion parameters of the slave end executor, generating a virtual pose feedback signal conforming to a master end communication protocol by using a reverse reconstruction algorithm, and returning the same to the master end follow-up teach box. The present application can eliminate control stiffness and logic deadlock in heterogeneous teleoperation, avoid mechanical impact caused by instruction mutation, ensure that operation polarity and visual motion are highly consistent, and improve operation efficiency.

The invention discloses a pose mapping method based on a drift function and a synchronous control system, and belongs to the technical field of robot teleoperation and intelligent control. According to the method, a heterogeneous coordinate system mapping relation between a near-end VR head-mounted display and a far-end observation robot is established, a drift function activated based on normalized operability is introduced, and an unreachable area and a kinematics singular configuration are dynamically avoided; the system adopts a double-layer architecture of outer ring mapping and inner ring PID control, and high-continuity and high-precision pose synchronization is realized. According to the invention, accurate pose control of the far-end observation robot by a near-end operator can be realized, and teleoperation safety and telepresence are improved.

This disclosure provides a teleoperation compensation and teleoperation method, apparatus, and electronic device. The teleoperation compensation method includes: at the slave control end of the teleoperation system, modifying the control model of the teleoperation system based on the robot's actual state parameters without network latency and the robot's physical configuration parameters; at the master control end of the teleoperation system, compensating for the robot's network interference torque based on the robot's feedback state parameters including network latency, and further modifying the modified control model; based on the secondary modified control model and the compensated network interference torque, compensating for the robot's feedback state parameters to obtain the robot's target state parameters. This disclosure enables robot teleoperation to possess the advantages of high precision, strong robustness, and low latency.

The invention discloses a robot teleoperation method based on fusion of virtual reality and mixed reality. The robot teleoperation method comprises the following steps: sensing a far-end environment by observing a robot; constructing a near-end interactive virtual environment, and presenting the virtual environment to a near-end operator through a virtual reality device; constructing three-dimensional point cloud data based on a far-end environment through three-dimensional reconstruction; in the virtual environment, pose estimation is carried out on the far-end operation robot and the target object, and three-dimensional registration is carried out in the virtual environment; working space boundary condition constraints and space operation pose change constraints of the working robot are determined, path planning and trajectory planning are conducted on the working robot, and visualization is conducted in the virtual environment; and generating a control program of the operation robot to move along the planned track. By constructing the interactive virtual environment, the working space range and the operation pose of the operation robot are visualized in the interactive virtual environment, and the trajectory is planned and visualized in the interactive virtual environment, so that the safety of teleoperation is greatly improved.

The application discloses a kind of man-machine collaborative multi-arm teleoperation control system, method and robot based on multimodal large language model, it is related to robot teleoperation technical field, the system is equipped with four big modules of visual perception, LLM driven task planning, motion primitive execution, safety supervision of human-in-the-loop, decouples high-dimensional control space with "main pilot-co-pilot" paradigm, human remote control main arm makes fine operation, intelligent agent independently controls auxiliary arm to execute coarse-grained task, voice intervention instruction has absolute priority.This application breaks through the bottleneck of cognition and operation of single-person control multi-arm, realizes single operator to complete four-arm strongly coupled collaborative task, eliminates the communication delay of multi-person cooperation, greatly reduces demonstration trajectory variance, improves the quality of teaching data, optimizes downstream model training performance, while significantly reducing the operation threshold, realizes high-level physical safety through low-delay voice supervision, can be widely applied to industrial assembly, logistics handling and other scenes, and has strong industrial practicability.

The present application specifically relates to the technical field of human-machine collaborative intelligent manufacturing and assembly, and provides a virtual reality and digital twin-based robot teleoperation method and system. The solution comprises: starting up an industrial robot and a wearable device; establishing a bidirectional communication connection between a virtual environment of the wearable device and the industrial robot, and rendering a virtual robot of the wearable device in the virtual environment; and inputting a target pose operation instruction into the virtual robot, and in response to receiving a target motion trajectory planned by the wearable device according to the target pose operation instruction, controlling the industrial robot to perform an operation task according to the motion trajectory. The solution enables information interaction between an operator and a robot, thereby achieving immersive real-time robot teleoperation by means of instant and accurate operation commands and on-site feedback, and significantly improving teleoperation accuracy for complex tasks and flexible manipulation tasks.

The application provides a construction method of a robot teleoperation body model and a robot teleoperation system, and the method comprises the following steps: fine-tuning a pre-training body model by using master-slave robot control data to obtain a fine-tuned body model; deploying the fine-tuned body model to provide robot control services and collecting real machine execution data; obtaining scores of the real machine execution data, and forming reinforcement learning training data pairs of the real machine execution data and corresponding scores, and adding the reinforcement learning training data pairs to a reinforcement training data set; and performing reinforcement learning training on the fine-tuned body model by using the reinforcement training data set. The pre-training model is fine-tuned, the cost of manual intervention is effectively reduced, the reinforcement learning training data is collected by deploying the fine-tuned body model, the reinforcement learning method and the pre-training body model are organically combined, the advantages of the two are fully utilized, and efficient learning and performance improvement of the robot in a complex environment are realized.

The invention provides a multi-degree-of-freedom robot teleoperation system based on virtual-real closed-loop self-calibration. The multi-degree-of-freedom robot teleoperation system comprises a virtual-real synchronization module, a bidirectional real-time interaction engine, a self-adaptive control module and a collision protection module, the virtual-real synchronization module is used for realizing synchronization of a virtual-real space coordinate system; the bidirectional real-time interaction engine is used for converting a space pose instruction input in the upper computer into virtual joint angle data and outputting the virtual joint angle data; the actual torque transmitted back by the physical joint motor is transmitted back to the virtual engine, and relevant parameters are updated in real time; the adaptive control module is used for calculating a feed-forward torque and a compensation torque according to the output of the bidirectional real-time interaction engine, synthesizing the feed-forward torque and the compensation torque into a final motor driving instruction, and issuing the final motor driving instruction to the physical joint servo driver; and the collision protection module is used for calculating the Euclidean norm of the collision torque in real time and generating a reverse compensation torque according to a final motor driving instruction when the Euclidean norm exceeds a dynamic threshold triggering protection mechanism.

The invention relates to a robot teleoperation control system and method based on force feedback and guiding force constraint, and belongs to the technical field of electric robots, the control system comprises an operation master end, an operation slave end and an upper computer, the operation master end comprises a master end teleoperation handle and a work scene real scene three-dimensional perception and display module; the operation slave end comprises an operation site environment sensing sensor and an operation mechanical arm, and the operation site environment sensing sensor comprises binocular vision and a laser radar and is carried on the operation mechanical arm; the operation master end and the operation slave end are connected with the upper computer through a network and carry out information exchange between the operation master end and the operation slave end, and the information exchange comprises communication, calculation, control and data interaction between the master end teleoperation handle and the upper computer and between the upper computer and the operation mechanical arm. And control information input and operation state information display are realized through the upper computer. According to the invention, the interactivity and the operation precision between an operator and the robot are improved, and the possibility of misoperation is reduced.

A method for probabilistic modeling for intention estimation in an operator-robot teleoperated assembly task is provided. The method may estimate the assembly task to be completed, wherein the assembly task comprises a sequence of actions. The method may predict a next action of the sequence of actions to be performed for the assembly task to be completed.

The present application belongs to the technical field of robot teleoperation and imitation learning, and particularly relates to a humanoid dual-arm teleoperation and data acquisition exoskeleton device, which comprises a trunk wearing mechanism, opposite sides of the trunk wearing mechanism are provided with arm wearing mechanisms, the connection between the arm wearing mechanisms and the trunk wearing mechanism is located at the upper part of the trunk wearing mechanism, and the arm wearing mechanisms are used for collecting motion information at arm joints; in the present application, the arm wearing mechanisms are provided with seven degrees of freedom, have higher arm fitting degrees, can better follow the upper limbs of the human body to realize compliant motion, and reduce the deviation between the motion trajectory of the mechanical arm and human operation; meanwhile, the mechanism directly collects arm motion data and maps the arm motion data to the joints of the humanoid robot, does not need complex calculation, greatly reduces the dependence on complex programming, can accurately and quickly generate control instructions, drives the humanoid robot dual arms to complete complex tasks, and improves the accuracy, fluency and real-time performance of the teleoperation and data acquisition process.

This application provides a robot remote control operating system, including a master robot, a slave robot, and a control system. The control system is configured to cause the slave robot to move in accordance with the movement of the master robot. The control system is further configured to determine a coefficient K and, based on the displacement difference between a reference point on the master robot and a selected point on the slave robot corresponding to the reference point, and the coefficient K, determine the control force F output by the slave robot at the selected point.

The invention discloses an immersive robot teleoperation master-slave system and method based on bidirectional force sense mapping. The system comprises a master hand controller and a slave end robot, wherein master-slave position-force bidirectional closed-loop control is established through a wireless communication mode; the master hand controller sends a mechanical arm target pose instruction, an execution tail end closing / releasing instruction and a control signal of the mobile platform to the slave end robot to form a forward control channel; the control panel obtains feedback torque based on the current I fed back by the slave end robot and a preset torque coefficient Kt, and the feedback torque serves as force sense feedback of the slave end; the control panel sends a torque resisting instruction to the feedback motor according to the feedback torque, the feedback driving motor rotates according to the torque resisting instruction, the feedback transmission mechanism is driven to act, and a reverse feedback channel is formed. According to the invention, the action of the operator can be accurately mapped to the slave end robot, and the force generated by interaction between the slave end robot and the environment can be truly fed back to the operator in a low-delay manner.

The application discloses a semi-human robot remote control method based on a VR device, and relates to the field of VR devices, and comprises the following steps: step one, data flow tracking and connection: a data flow tracking APP is developed and compiled, the APP is installed on the used VR device, and the VR device collects relative pose data of gestures and heads of operators in real time; step two, coordinate conversion of VR data flow: the coordinate systems of wrists and heads in the VR data flow are one-to-one corresponding to VR base coordinates; and step three, an inverse kinematics solving program is written based on C++, and joint angles and gestures are solved and designed.The application breaks through the perspective limitation of the existing system, combines the head control function of the VR device, so that the movement of the robot head can be adjusted in real time along with the rotation of the head of the operator, thereby realizing dynamic first perspective switching.This not only enhances the immersion of the operator, but also improves the operation accuracy, so that the operator can better operate in a complex environment.