826 results about "Teleoperation" patented technology

Disclosed in the present invention are a multi-modal shared teleoperation system and method for a three-arm space robot. The system at least comprises a master-side teleoperation system, a communication module and a slave-side robot system, wherein the master-side teleoperation system at least comprises two force feedback hand controllers, a microphone array and upper computer software, and the slave-side robot system comprises two operating arms each equipped with a gripper at the end, an observation arm having a binocular camera mounted at the end, a vision unit, a force sensor and lower computer software. The method comprises: an operator controlling two operating arms of an extravehicular robot to execute a task, and controlling an observation arm to acquire a better local field of view. In the method, a multi-modal teleoperation method comprising pose control, voice control and force control is fused with autonomous control of a robot by means of a shared control algorithm, and thus, human-robot collaborative control over the position, orientation and contact force of a robotic arm can be realized on the basis of the requirements of the operator, and the robot autonomously executes other relatively simple tasks, thereby reducing the operation burden of operators, and improving the control efficiency.

The invention belongs to the technical field of robot control, particularly provides a force feedback teleoperation control system and method for the tail end of a mechanical arm of a quadruped robot, and aims at the key challenges that control errors are caused by communication time delay and soft obstacles are difficult to recognize in a dynamic environment. Modeling and judgment are conducted on the contact state of the tail end of the mechanical arm in advance, and feedforward control and buffer adjustment oriented to communication time delay are achieved. And meanwhile, a dynamic semantic map is constructed in combination with multi-source sensing information, and soft obstacle reasoning and path optimization are performed by fusing a tail end force sense change trend, so that the recognition and avoidance capabilities of the system in a complex and invisible obstacle environment are remarkably improved. The system has good perspectiveness, self-adaptability and high redundancy safety characteristics, is suitable for multi-task inspection operation of industrial sites such as a thermal power plant, and is especially suitable for a remote man-machine cooperative operation scene in a narrow space.

In various examples, teleoperation architectures for safe control of machines are described. Systems and methods are disclosed that use an end-to-end safety architecture that covers both a vehicle or machine and a remote system providing a control center, where the vehicle or machine is at least partly or temporarily configured for control by the remote system. In some examples, the end-to-end architecture uses a layered safety policy monitoring system, where the remote system uses first policies to ensure that operator commands are viable and the vehicle uses second policies to ensure that the operator commands are safe to perform (e.g., will not cause collisions with other objects). Additionally, in some examples, the end-to-end architecture allows for the vehicle to perform minimum risk maneuvers, also referred to as “control fallbacks,” if problems were to occur.

A dual-arm cooperative potential field-based force-guided teleoperation system and control method. By means of real-time poses of tool center points of two robotic arms, whether the shortest distances dp-s,i between a target object and workspace boundaries of the two arms are lower than a threshold Ds is iteratively determined; if the shortest distances are both greater than the threshold, a dual-arm symmetric cooperation strategy is adopted, and if the shortest distances are lower than the threshold, a single-arm prioritized cooperation strategy is adopted, and the arm is used as a priority arm; cooperation factors δi of the two arms are respectively determined on the basis of the cooperation strategies; a dual-arm cooperative potential field is constructed on the basis of the cooperation factors, the distances between the target object and the tool center points of the two arms, and the position of an obstacle; and corresponding dynamic virtual constraint forces are generated on the basis of the dual-arm cooperative potential field, and two force feedback hand controllers are used to guide and assist an operator in controlling the two robotic arms to complete a dual-arm cooperative task.

Systems and methods that allow robots to perform tasks for users are provided. A robot may comprise one or more robotic arms and / or a mobile base. The arms may be controlled by electric actuators and may have six or more degrees of freedom. The robot may have sensors which can be accessed remotely. Robots may have varying levels of autonomy, including, for example, full teleoperation (in which a human can have detailed control over the robot) or full autonomy (in which the robot can complete a task without any human intervention). Various entities can interact with individual robots or groups of robots over a network, locally, directly or in person, or any combination thereof. A management system can allow entities to control and / or monitor the robots.

According to the double-arm mechanical arm teleoperation system based on the VR head-mounted display and the control method, high-precision and flexible teleoperation is achieved by accurately capturing movement tracks of the two hands and mapping the movement tracks to the double-arm mechanical arm, meanwhile, the content shot by the scene camera is transmitted to the VR head-mounted display in a streaming mode, and the real-time performance and immersion of operation are enhanced. In addition, the intelligent obstacle avoidance strategy designed by the invention effectively prevents the movement of the mechanical arm from being out of control in the operation process, and ensures the safety and reliability of the operation. The system has the remarkable advantages of high-precision operation, real-time performance, immersion, multi-scene adaptability, safety, reliability and the like, is suitable for interaction and task execution of a universal robot in industrial, household and other scenes, replaces human beings to execute precise operation in a high-risk environment, and has wide application prospects and important practical significance.

A teleoperations system may be used to generate virtual paths of travel for an autonomous vehicle based upon teleoperations system-provided suggestions, and virtual path suggestion constraints, e.g., based upon physical constraints associated with an autonomous vehicle such as minimum turning radius, trailer sweep, and the like, may be used assist in guiding a teleoperations system operator during generation of virtual path suggestions.

The invention provides a robot system based on self-adaptive cruise and remote operation and a control method thereof, and belongs to the field of robot inspection, the system comprises a six-axis mechanical arm, an intelligent inspection robot, an automatic inspection task module and a remote teleoperation control unit; the intelligent inspection robot carries a six-axis mechanical arm and an automatic inspection task module, the automatic inspection task module comprises a preset inspection task, the remote teleoperation control unit sends a control instruction of an operator to the intelligent inspection robot through wireless communication, and the control instruction comprises inspection path planning and operation tasks. The intelligent inspection robot performs an inspection task according to a preset inspection task of the automatic inspection task module or a control instruction sent by the remote operation control unit, and the six-axis mechanical arm is used for executing a target operation task. Through integration of advanced sensor technology, image processing technology, artificial intelligence algorithm and wireless communication technology, efficient, accurate and safe inspection of a complex environment is realized.

The invention provides a five-finger mechanical dexterous hand functional grabbing method based on progressive skill course learning, which comprises the following steps of: controlling the actions of a dexterous hand in a simulation system by a human hand through a human hand posture detection and joint angle mapping module by utilizing a teleoperation teaching system, and grabbing functional parts of an object in the simulation system to obtain corresponding expert demonstration; defining different interaction tasks of the dexterous hand and the object according to the hand shape and the object category, and designing courses in different stages; starting progressive skill course learning, dividing a learning framework into primary, intermediate and advanced courses, and respectively obtaining a primary strategy set corresponding to a low-difficulty task, an intermediate strategy set corresponding to a medium-difficulty task and an advanced strategy corresponding to a high-difficulty task at the end of each stage; after progressive skill course learning, an advanced strategy grabs corresponding parts of different types of objects in correct hand shapes according to grabbed hand shape classification information and object point clouds in a direct execution or small-amount fine adjustment mode.

The invention discloses an immersive master-slave teleoperation anthropomorphic robot system with force feedback, a master end assembly comprises a seat, a master end control arm, a force feedback manipulator, a VR visual device, a data sensing glove and a master end controller, and the master end assembly is a master control end; the slave end assembly comprises a movable chassis, a movable trunk, a slave end mechanical arm, a slave end manipulator, a binocular camera and a slave end controller, and the slave end controller is in electric signal connection with the movable chassis, the movable trunk, the slave end mechanical arm, the slave end manipulator, the binocular camera, the six-dimensional torque sensor and the master end controller; the wireless controller is in electric signal connection with the master end controller and the slave end controller, the wireless controller has a slave end control mode and a force feedback teleoperation mode, and in the slave end control mode, the wireless controller controls the movable chassis to move; in the force feedback teleoperation mode, the master end control arm and the force feedback mechanical arm synchronously move with the slave end mechanical arm and the slave end mechanical arm correspondingly.

The invention discloses a dexterous hand teleoperation method based on vision-touch fusion. The method comprises the following steps: 1) vision detection; the method comprises the following steps of (1) carrying out remote control, (2) carrying out redirection, (3) carrying out tactile grabbing, (4) carrying out vision-tactile interaction, (5) carrying out a communication process and (6) carrying out dexterous hand.The method can stably map human hand actions under the complex vision condition, safe and reliable grabbing control is achieved through tactile feedback, meanwhile, multiple dexterous hand configurations and cross-platform deployment are supported, and the precision, robustness and practicability of teleoperation can be improved.

The invention relates to the technical field of robots, in particular to a dexterous hand control method, a dexterous hand control device, a dexterous hand and a robot, and solves the problem of poor universality of the dexterous hand. The dexterous hand control method comprises the steps of firstly receiving teleoperation data, then determining a target angle of a driving motor according to first mapping data and the teleoperation data, and then sending the target angle of the driving motor to the driving motor to enable the driving motor to rotate to the target angle so as to enable the driving motor to drive a joint to move. Therefore, the fingers are made to move to the target position or the joints are made to rotate to the target angle, control over the joints is achieved, the dexterous hand is further controlled to achieve the teleoperation action, the purpose that the dexterous hand is controlled to achieve various actions through teleoperation equipment is achieved, and the universality of the dexterous hand is improved.

The present invention relates to the field of robotic arm teleoperation technology, and specifically to a construction method and system for real-time teleoperation of a dual arm and hand robot based on vision guidance, which includes setting up a first experimental platform, acquiring pose information of the dual arm and hand robot and a homogeneous transformation matrix of an end joint and transforming a first coordinate system; setting up a second experimental platform; performing calibration by a plurality of cameras and transforming a second coordinate system; obtaining spatial transformation pose information of the end joint of the robotic arm under a tracking state, reading finger pose data, and performing real-time following. The real-time teleoperation system of the present disclosure includes a control system host, an operation glove, an optical locator, a positioning coordinate plate, a marker ball tool, and a dual arm and hand robot. The present disclosure introduces a method of identifying the marker ball tool by the optical locator into the teleoperation control of the dual arm and hand robot, which is more accurate and faster. Meanwhile, the method of remotely controlling the robotic arm by an operator wearing the operation gloves can provide better human-machine interaction function.

The invention discloses an intelligent graph ticketing system with topology verification in a power distribution program. Comprising a plurality of modules such as a data access and preprocessing module, a single line diagram topology coloring module and an equipment state intelligent identification module, and the data access module can access and preprocess data such as a distribution network GIS (Geographic Information System) graph model; the topology coloring module marks an area during ticket simulation; the intelligent identification module can identify the equipment state and the wiring mode; the operation ticket intelligent generation module generates operation steps through a graphic platform and can perform integrated verification; the drawing ticket anti-error checking module carries out ticket face and topology anti-error checking; the operation sequence generation and security check module generates a reasonable operation sequence and checks the reasonable operation sequence; the remote control operation pre-checking module checks the remote control operation working environment state and the equipment condition; the automatic remote control execution module initiates an execution request; the programmed operation monitoring and log recording module provides a monitoring interface and records ticket formation days. The system improves the efficiency and accuracy of operation ticket generation, and guarantees the operation safety.

The invention discloses a real-time whole-body remote operation control system for a humanoid robot, and the system comprises a motion instruction generation subsystem which is located at an operation end and is used for receiving and analyzing input instructions of various modes, and carrying out the unified conversion and complementation of the input instructions into a whole-body joint motion instruction which is suitable for a skeleton structure of a target humanoid robot; the body motion control subsystem is located at the robot end and used for receiving the whole-body motion instruction, adaptively generating a whole-body motion coordination control signal capable of ensuring the stability of the robot in combination with the state of the robot and environment information, and driving and executing the whole-body motion coordination control signal; through cooperation of the two subsystems, the system provides a dual guarantee path for realizing whole-body teleoperation: a complete whole-body instruction can be generated based on the instruction generation subsystem, and whole-body motion can be adaptively generated by the body motion control system according to a received local instruction; and the robustness, the flexibility and the reliability of the system under different conditions are ensured.

The invention provides a robot master-slave arm teleoperation software and hardware system. The system comprises a host, a master arm, a slave arm and a scene acquisition module, the master arm is used for collecting task action data and transmitting the task action data to the host; the host processes the task action data to generate an action instruction conforming to the driven arm, and the driven arm executes a corresponding action according to the action instruction; the scene acquisition module is used for acquiring operation scene data of the driven arm and transmitting the operation scene data to the host; and the host is also used for associatively storing the task action data, the execution action data of the driven arm and the operation scene data as training data of the intelligent model with the body. The host converts the task action data collected by the driving arm into the control instruction suitable for the driven arm, teleoperation of different types of driven arms can be supported, a large amount of training data of the body-equipped intelligent model suitable for different types of driven arms with different arm lengths is conveniently obtained by collecting and associating related data, and the training efficiency is improved. The device has the function of integrating intelligent data acquisition and model verification.

The invention provides a binocular vision teleoperation robot system, a robot and a teleoperation method and device, and relates to the technical field of computers. The method comprises the following steps: receiving a binocular video stream obtained by carrying out image acquisition on an actual scene by a binocular vision robot, and rendering the binocular video stream to obtain a three-dimensional scene; displaying the three-dimensional scene to an operator of the binocular vision robot to capture motion data of the operator observing the three-dimensional scene; determining a target area from the actual scene according to the action data, and obtaining a visual control instruction based on the target area; the vision control instruction is transmitted to the binocular vision robot for teleoperation; wherein the visual control instruction is used for adjusting imaging parameters of the binocular vision robot for image acquisition of an actual scene, and the visual direction of the binocular vision robot can be rapidly and naturally kept consistent with the attention point of the operator.

The invention relates to the technical field of humanoid robot control. The invention provides a humanoid robot bilateral force feedback teleoperation method of a force compensation mechanism. According to the method, a 5G private network is used for deployment, the characteristics of low delay, high reliability and large bandwidth of the 5G private network are utilized, real-time interaction of high-dimensional haptic data in a remote and cross-domain environment is effectively supported, cross-domain teleoperation can be achieved, high-fidelity feedback transmission of environment contact force is effectively achieved, and the real-time interaction of the high-dimensional haptic data in the remote and cross-domain environment is achieved. The precision, the operation efficiency and the overall safety of remote operation of the humanoid robot in complex industrial scenes such as nuclear power maintenance, precise assembly and high-risk environment operation are remarkably improved, the gap between traditional automation and low-efficiency remote operation is filled up, and reliable technical support is provided for advanced industrial application needing man-machine cooperation fine force control.

The invention discloses a master-slave robot force feedback teleoperation control system and method for cleaning the surface of an object, force feedback teleoperation control of a master-slave robot is realized through a double admittance control loop, stable contact with objects with different rigidities is realized by using a variable stiffness algorithm, and the engineering task requirement of cleaning attachments on the surface of the object is realized. Admittance algorithm parameters are modified in real time through a variable stiffness algorithm, uncertain motion such as buffeting caused by contact between a robot and a high-stiffness object in a conventional admittance algorithm is avoided, the robot can be in contact with objects with different rigidities and move stably, meanwhile, the force feedback effect of the admittance algorithm is achieved, and teleoperation control has a sense of immediacy.

The invention relates to the technical field of teleoperation, in particular to a teleoperation robot auxiliary control method based on a safety margin theory. The method comprises the following steps: establishing a three-dimensional digital twinborn model according to an image acquisition device, and obtaining a target object distance, a communication time delay size and a motion tracking error; based on the three factors, a fuzzy control theory is adopted to calculate a teleoperation safety margin value, and the safety of the current teleoperation is quantitatively evaluated. And calculating a master-slave motion mapping ratio and an end view display ratio according to the safety margin value. And constructing virtual force guidance according to the safety margin value, feeding back the virtual force guidance to an operator through force feedback equipment, and guiding the operator to complete a teleoperation task in combination with visual interaction equipment. Compared with a traditional teleoperation method, the method can effectively improve teleoperation efficiency and safety, and can better adapt to a complex long-distance working environment.

The invention relates to the technical field of medical robot control, in particular to a teleoperation control method and system for an alimentary canal catheterization robot. The control method comprises the following steps: acquiring digestive tract three-dimensional structure data of a patient by utilizing imaging equipment; based on the three-dimensional structure data, establishing a patient individualized three-dimensional operation model through a teleoperation system; planning a catheterization path and an operation strategy according to the three-dimensional operation model; collecting motion state data of the alimentary canal catheterization robot in real time through a sensor of the teleoperation system; dynamically matching the motion state data acquired in real time with the three-dimensional operation model of the patient; deviation information fed back by a sensor in the operation process is obtained, an angle error signal is generated, and the catheter arranging robot is corrected in real time by adjusting a control strategy; the control system comprises a target sensor, a teleoperation part, a control module and an upper computer. By means of the mode, delay is reduced, high-precision control is provided, the force sensing technology is introduced, the feedback experience of an operator is enhanced, and therefore the precision and safety of surgical operation are improved.

The invention discloses a tea garden ditching and fertilizing all-in-one machine which adopts an electric crawler chassis which is independently driven on two sides, a fertilizing mechanism is located at the top of a whole machine frame, a ditching and earthing mechanism is located at the rear end of the whole machine frame, and the ditching depth can be adjusted by adjusting the length of a hydraulic telescopic rod; the control console is located at the front end of the whole machine frame and used for a driver to control the movement of the whole machine and the movement of all the mechanisms during driving, and meanwhile the ditching and fertilizing all-in-one machine for the tea garden can be controlled by selecting a wireless control mode under the condition that the road condition is simple. The ditching and fertilizing operation of the tea garden is more flexible; a wired and wireless combined remote control operation mode of ditching, fertilizing and earthing in the tea garden is realized; organic combination of the adjustable soil covering plate and the soil-splitting type fertilizing straight pipe is realized, and the soil crushing and covering link is more efficient; the ditching and fertilizing operation of the tea garden can be carried by a driver, and the complicated tea garden environment can be easily and timely adjusted in the operation process.

Provided herein is a control system and methods thereof for controlling operation of a robot. The control system comprises: a robot, including a plurality of sensors wherein each generating a stream of raw sensor data having a data type, size, and frequency, and a plurality of actuators cause movement of the robot; an autonomous control subsystem configured to receive the sensor data and output autonomous actuator data; and a teleoperation control subsystem configured to receive the sensor data, transmit the sensor data to a human operator, and output teleoperation actuator control signals, wherein the autonomous control subsystem and the teleoperation control subsystem receive, from the robot, sensor data having the same data type, size, and frequency, and wherein the autonomous actuator data and the teleoperation actuator data have the same data type, size, and frequency; and a control-determining subsystem configured to switch between autonomous and teleoperation control.

The invention discloses a robot dexterous arm teleoperation sharing control method and a related device, and relates to the technical field of robotics.The method comprises the steps that smooth expected pose information and expected angle state information are obtained, and pose information of a mechanical arm of a slave end at the current stage and angle state information and contact force information of a dexterous hand are obtained; based on the expected pose information and the joint angle information of the mechanical arm, the inverse kinematics of the mechanical arm is solved through a nonlinear optimization method, and a joint angle instruction of the mechanical arm is obtained; based on the expected angle state information, the angle state information of the dexterous hand and the contact force information, utilizing a fuzzy logic theory and a dynamic compliance primitive model to carry out adaptive variable impedance control on the dexterous hand to obtain a joint torque instruction of the dexterous hand; and feeding back the contact force information of the dexterous hand in the next stage to a main end operator. The success rate and the working efficiency of the teleoperation robot can be improved.

The invention discloses a dexterous hand general grabbing method and device based on single-visual-angle vision and demonstration optimization, and relates to the field of dexterous hand grabbing and artificial intelligence. The method comprises the following steps: firstly, building a real field scene and teleoperation equipment, and collecting expert data of objects with multiple sizes and shapes; and in combination with the collected expert data, on the basis of an algorithm obtained by improving a DAPG algorithm, a general grabbing strategy of cross-object shape primitives is learned in combination with domain randomization operation in a simulation environment. Geometric features of an unknown object are analyzed in real time through the first visual angle visual information, an initial grabbing track is generated, the grabbing force and posture are adjusted by fusing dynamic feedback of a force touch sensor, and high-generalization and high-reliability grabbing is carried out. The problems that a traditional method depends on a large amount of real expert data, a large amount of object model training is needed, strategy generalization is poor, and simulation migration stability is insufficient are solved.

The invention relates to the field of substations, in particular to an intelligent remote control device and method for substation secondary equipment. Comprising a control box matched with the existing switch equipment, and the control box is installed between the switch equipment and a screen panel; the micro motor is arranged in the control box and is used for providing operation kinetic energy and driving a rotating shaft of the switch equipment to rotate to a preset position; the intelligent sensing module is arranged in the control box and is used for detecting the gear state of the switch equipment to obtain a state feedback signal; the control unit is arranged in the control box, the control unit is connected with the micro motor and the intelligent sensing module, and the control unit is used for receiving a remote control command, controlling the micro motor to operate the switch equipment according to the remote control command and receiving a state feedback signal of the intelligent sensing module at the same time; the communication module is connected with the control unit and used for enabling the control unit to communicate with a remote control system. According to the invention, the remote control operation is efficient, the operation risk is low, the state monitoring is accurate, and the adaptability with existing equipment is high.

The invention provides a digital twinning-combined dexterous hand teleoperation hand action migration method. The method comprises the following steps: acquiring an image sequence of a hand through a binocular camera; for each group of images in the image sequence, determining a three-dimensional coordinate sequence of the hand feature points; based on a pre-established digital twin skeleton model of the dexterous hand, establishing a mapping relation between the hand feature points and corresponding points in the digital twin skeleton model; converting the three-dimensional coordinate sequence of the hand feature point into a three-dimensional coordinate sequence of a corresponding mapping point in the digital twin skeleton model based on the mapping relation, taking the three-dimensional coordinate sequence as the position of the mapped hand feature point, and determining an optimal buckling angle and an optimal abduction angle; and based on the optimal buckling angle and the optimal abduction angle, a joint angle sequence is obtained, and corresponding joint actions in the dexterous hand are controlled to enable the dexterous hand to perform action migration according to hand actions in the image sequence, so that the smoothness and precision of the actions of the dexterous hand in the action migration process are improved.

A computer-implemented method executed by data processing hardware of a robot causes the data processing hardware to perform operations. The operations include receiving a sensor pointing command that commands the robot to use a sensor to capture sensor data of a location in an environment of the robot. The sensor is disposed on the robot. The operations include determining, based on an orientation of the sensor relative to the location, a direction for pointing the sensor toward the location, and an alignment pose of the robot to cause the sensor to point in the direction toward the location. The operations include commanding the robot to move from a current pose to the alignment pose. After the robot moves to the alignment pose and the sensor is pointing in the direction toward the location, the operations include commanding the sensor to capture the sensor data of the location in the environment.

The invention discloses a smart type library-based teleoperation method, system and device, and a storage medium. The method comprises the following steps of: constructing a smart operation type library; analyzing the task instruction through a preset multi-modal large language model MLLM to obtain a corresponding operation step, and matching a dexterous operation type corresponding to the current step from a dexterous operation type library; intuitive accurate control between the hand and the manipulator is realized through an interpolation mapping strategy. The interpolation mapping strategy specifically comprises the following steps: capturing a fingertip position of a human hand in real time; calculating the normalized projection proportion of the current posture of the human hand relative to the stretching state and the contraction state; and according to the projection proportion, linear interpolation is carried out on the extension and contraction joint angles of the manipulator, and a manipulator joint angle instruction is generated. According to the method, the adaptability and stability of teleoperation are provided by establishing a systematized smart type library and combining a type automatic retrieval module assisted by a multi-modal large language model and an interpolation mapping control strategy.

The invention discloses a two-arm robot turnover skill learning method based on force control and a neural dynamic model, and the method comprises the following steps: carrying out the master-slave teleoperation through a teleoperation module, and constructing a space mapping model to achieve the master-slave motion matching of heterogeneous joints; a variable stiffness admittance control model is adopted for radial force control, and flexible contact force control is achieved by dynamically adjusting stiffness parameters; and constructing a neural dynamic model to perform coding modeling on the teleoperation data, constructing an energy function and a dynamic equation containing a learnable neural network, and realizing skill generalization and autonomous execution. According to the method, the contact stability is guaranteed through a force-position hybrid control strategy, the flexibility, the anti-interference capability and the skill generalization level of box turnover operation are remarkably improved in combination with the nonlinear modeling capability of the neural dynamic model, and a solution is provided for two-arm cooperative operation in a complex contact scene.