61 results about "Virtual robot" patented technology

A system and method are disclosed for training a machine learning model configured to determine a traversability of a real-world environment by a mobile robot based on an image of the real-world environment. The method advantageously generates high quality synthetic training data for training a traversability detection model to discriminate between traversable and untraversable regions in images captured of a real-world environment. The synthetic training data is generated through simulation of a virtual robot in a virtual environment. Once the traversability detection model is trained, it can be deployed to the mobile robot for the purpose of predicting traversability of a real-world environment.

Provided in the present invention are a virtual-robot and physical-robot collaborative safety control method and apparatus based on mixed reality and a digital twin, which method and apparatus specifically relate to the technical field of smart manufacturing and assembly involving virtual-robot and physical-robot collaboration. In the solution, the method comprises: acquiring first posture information of a physical robot, and acquiring second posture information of a virtual robot; on the basis of the first posture information and the second posture information, performing pose registration on the physical robot and the virtual robot, so as to obtain a registration result; on the basis of the registration result and a preset virtual-robot and physical-robot workspace in combination with deep reinforcement learning, performing path planning to determine a virtual-robot and physical-robot collaborative motion path; and on the basis of the virtual-robot and physical-robot collaborative motion path, executing a virtual-robot and physical-robot collaborative safety control action. The solution uses a mixed-reality device to extract virtual-robot and physical-robot posture information, implements virtual-robot and physical-robot pose registration by means of a virtual-real space mapping method, and designs a bidirectional virtual-robot and physical-robot collaborative safety interaction policy based on deep reinforcement learning, thereby effectively improving the safety and effectiveness of a virtual-robot and physical-robot interaction operation.

The embodiment of the invention provides a virtual robot generation method and device, equipment and a storage medium. Displaying a content input entry; wherein the content input entry comprises a character input entry and / or a content uploading entry; receiving demand information input through the character input entry, and / or receiving training data input through the content uploading entry; generating a virtual robot based on the demand information and / or the training data; wherein the virtual robot is used for performing question and answer interaction with a user. According to the generation method of the virtual robot provided by the embodiment of the invention, the virtual robot is automatically generated based on the input demand information and / or training data, manual configuration of a user is not needed, and not only can the generation efficiency of the virtual robot be improved, but also the interaction capability of the virtual robot can be improved.

For a (stationary or mobile) robot within a monitored space, a safety subsystem generates a virtual representation of the robot using position and / or orientation data associated with the robot, detects an unexpected object within the monitored space using sensor data from a plurality of (external / internal) sensors, detects a virtual halt collision between the virtual robot and the virtual unexpected object based on a halt zone corresponding to the robot or the unexpected object, and generates an instruction to halt the normal operations of the robot. The (dynamic) virtual robot representation may include a robot arm movable with respect to the robot's base. The monitored space is generated by removing ignore zones corresponding to expected objects, including an ignore zone corresponding to the robot itself, from an initial (e.g., spherical) version of the monitored space. Regions having insufficient sensor coverage (i.e., IPZs) are treated as unexpected objects within the monitored space.

The invention discloses a robot navigation method based on reinforcement learning security constraint and danger perception, and relates to the technical field of robots and artificial intelligence, and the robot navigation method comprises the following steps: when a robot is controlled to execute a navigation task in a target area, using a security filter SF to perform security constraint on the execution action of the robot; and when the robot executes the navigation task, the danger perception virtual robot HAVR is utilized to obtain experience in the dangerous area so as to enhance the ability of the robot to explore in the dangerous area. The action generated by reinforcement learning can be dynamically adjusted, so that the action meets the safety constraint, and the safety of the robot in the reinforcement learning training process is further enhanced; by introducing the HAVR with risk-driven rewards, the robot can still obtain experience in a dangerous area while introducing security constraints, and the exploration security of the robot is further enhanced while the performance of the model after convergence is ensured.

A system and method for training a machine learning model are disclosed, designed to determine the traverseability of a real-world environment by a mobile robot based on an image of that environment. The method advantageously generates high-quality synthetic training data for training a traverse detection model to distinguish between traversable and impassable areas in captured images of a real-world environment. The synthetic training data is generated by simulating a virtual robot in a virtual environment. Once the traverse detection model is trained, it can be applied to the mobile robot to predict the traverseability of a real-world environment.

A simulation data creation method, a preview image display method, and a simulation device that enable grasping of the contents of simulation before construction of a virtual space. The simulation data creation method includes the following steps: (a) causing a virtual robot disposed in a virtual work space to act based on an operation instruction by a user, and displaying a simulator image representing the action of the virtual robot in the virtual work space on a display device, the virtual work space being a virtual three-dimensional work space; (b) outputting a snapshot representing a two-dimensional image obtained by a virtual camera capturing a state in which a virtual three-dimensional object including the virtual robot is disposed in the virtual work space; and (c) saving, as simulation data, in a memory, an action program representing a target action of the virtual robot based on the operation instruction, layout data representing positions and postures of objects in the virtual work space, attribute data representing attributes of the objects, and the snapshot.

The application provides a robot process debugging system and method based on three-dimensional visualization, and relates to the technical field of robot processing debugging, and comprises the following steps: a data processing module is used for acquiring saved workstation data and robot data in a robot workstation, and extracting robot trajectory point data and process parameters from the robot data; a three-dimensional display module is used for establishing a virtual workstation model containing a virtual robot model according to the workstation data and the robot data, and establishing a virtual processing trajectory of the virtual robot model according to the trajectory point data and the process parameters; and a debugging module is used for debugging process parameters to obtain process debugging parameters according to externally input debugging instructions, and the three-dimensional display module synchronously displays the adjustment of the virtual processing trajectory according to the process debugging parameters. The beneficial effect is to provide a three-dimensional visual interactive interface, reduce the operation complexity, improve the process debugging efficiency, improve the interactive experience, and make the process debugging more convenient.

Disclosed is a robotic system (104) and a method (600) having a control unit (202). The control unit is configured to generate a virtual environment having a virtual robot (402) based on a received set of first parameters to emulate a real environment. The control unit (202) is configured to actuate the virtual robot (402) to simulate an operation of a robot (108). At least the virtual robot (402) actuates independently, or the robot (108) and the virtual robot (402) actuate in synchrony based on the simulation. Further, at least the virtual robot (402) actuates independently, depending on the set of second parameters or the robot (108) and the virtual robot (402) actuates, depending on the set of second parameters. This configuration ensures multi modal operation of the robotic system (104), thus ensuring ease of operation of the robot (108).

This disclosure provides a robot intelligent visualization method, device, storage medium, and program product. The method includes: acquiring robot pose data, joint data, and point cloud data of a real-world scene; using a coordinate transformation method to convert the position data in the pose and joint data from the robot's body coordinate system to a three-dimensional scene coordinate system; based on the coordinate-transformed pose and joint data, performing real-time posture and skeletal synchronization of the robot to generate a virtual robot model that accurately reproduces the robot's real movements; constructing a three-dimensional scene based on the point cloud data and embedding the virtual robot model into the three-dimensional scene; and triggering a corresponding task mode when the virtual robot model is within the task point sensing range of the three-dimensional scene. This method achieves precise spatiotemporal synchronization and fusion between the physical robot and the virtual scene through multi-source data fusion and coordinate system unification, realistically reproducing robot movements and intelligently triggering tasks.

The invention discloses a crankshaft defect coping method and system based on digital twinning. The method is characterized by comprising the following steps: constructing a digital twin virtual environment consistent with a real environment, wherein virtual camera parameters are consistent with a real multi-view vision system; a crankshaft defect image is collected through a real multi-view vision system; the defect pixel positions are mapped to the surface of the virtual crankshaft to form virtual defects; in the virtual environment, the virtual robot plans a grinding path and generates an instruction; and the instruction is sent to the real robot to execute grinding. The system comprises a real physical system, a digital twin virtual system and a control communication system. According to the method, path planning and verification are completed in the digital twin virtual environment, and then the instruction is issued to the real system for execution, so that the precision, the safety and the automation degree of crankshaft defect grinding are effectively improved, and the collision risk and the debugging cost are reduced.

Provided is a simulation system that simulates an operation of a cell including a robot and a device, wherein the simulation system comprises a virtual device that simulates an output of the device and a a virtual robot controller that simulates, according to an output of the virtual device, an operation of a robot controller that controls the robot, and the virtual device performs an output based on a plurality of output histories of the device.

A teaching device for teaching a robot includes: a storage unit that stores mechanism data for finding a relationship between a tip position of a robot and an angle position at a joint portion of the robot, the mechanism data including a mechanism error parameter of an actual robot; and a virtual robot control unit that controls the operation of a virtual robot on the basis of the mechanism data such that an error in position that occurs in the actual robot due to the mechanism error parameter occurs in the virtual robot.

Embodiments of the present application relate to the field of virtual simulation technologies, and provide a robot offset simulation method and apparatus, a robot, an electronic device, and a storage medium. The robot offset simulation method includes: establishing a reference path for a virtual robot in simulation software, where the reference path is a path along which the virtual robot moves to transfer a reference object from a preset starting point to a destination location and then returns to the preset starting point; receiving location information of a target object that is sent by a PLC; determining an offset of the virtual robot based on the location information, where the offset is a location offset of the target object relative to the reference object; and controlling, based on the reference path and the offset, the virtual robot to move to transfer the target object.

The invention discloses a robot teaching method based on a sphere bounding box and related equipment. The robot teaching method comprises the following steps: acquiring a chemical experiment virtual scene corresponding to a real operation environment; collecting motion data of teaching input equipment, and driving a virtual robot in the chemical experiment virtual scene to perform chemical action drilling based on the motion data; in the moving process of the virtual robot, the space occupation information is updated in real time; obtaining a collision detection result between the sphere bounding boxes according to the updated space occupation information; and generating a control instruction sent to a real robot according to the collision detection result. According to the method, traditional full-grid collision detection is abandoned, collision detection is carried out in a simplified sphere bounding box model mode, low-delay real-time anti-collision early warning is achieved, and the stability and safety of the chemical experiment operation process are guaranteed.

Example implementations may include capturing virtual hot execution, for example by taking a screen recording of a desktop screen of a virtual machine where the execution is taking place. The contents, such as video such as the recording and / or associated data, may be processed, buffered, stored, and / or transmitted before being streamed (e.g., to a user). Thus, these implementations may involve initiating a virtual bot to execute an operation; generating a graphical representation of the virtual bot executing the operation; and transmitting, to a client device, the graphical representation of the operation for display at the client device.

Embodiments of present disclosure provide a method, an apparatus, an electronic device, a computer-readable medium and a computer program product of controlling a virtual robot on a robot training platform. The method comprises receiving, from an extended reality (XR) device, a user operation for controlling the virtual robot on an XR platform. The method further comprises determining, by the XR platform, a control result for the user operation based on the user operation and information associated with the virtual robot. The method further comprises transmitting, to the XR device, the control result to cause the XR device to present a rendered control result of the virtual robot.

The present application specifically relates to the technical field of human-machine collaborative intelligent manufacturing and assembly, and provides a virtual reality and digital twin-based robot teleoperation method and system. The solution comprises: starting up an industrial robot and a wearable device; establishing a bidirectional communication connection between a virtual environment of the wearable device and the industrial robot, and rendering a virtual robot of the wearable device in the virtual environment; and inputting a target pose operation instruction into the virtual robot, and in response to receiving a target motion trajectory planned by the wearable device according to the target pose operation instruction, controlling the industrial robot to perform an operation task according to the motion trajectory. The solution enables information interaction between an operator and a robot, thereby achieving immersive real-time robot teleoperation by means of instant and accurate operation commands and on-site feedback, and significantly improving teleoperation accuracy for complex tasks and flexible manipulation tasks.

The application relates to a collaborative robot interaction control method and device, a storage medium and electronic equipment. The method comprises the following steps: displaying a virtual robot corresponding to a physical robot in a three-dimensional scene; in response to a user triggering operation on a target joint of the virtual robot, displaying a normal line corresponding to the target joint, and displaying a draggable anchor point on the normal line; in response to a user dragging operation on the anchor point, determining motion control data of the target joint according to a dragging direction and a displacement of the dragging operation, controlling the physical robot to move based on the motion control data, and updating a pose of the virtual robot according to a real-time motion state of the physical robot. The application can intuitively display a robot motion displacement and an instantaneous speed in a graphical manner.

Apparatuses, systems, and techniques to perform collision-free motion generation (e.g., to operate a real-world or virtual robot). In at least one embodiment, at least a portion of the collision-free motion generation is performed in parallel.

The invention discloses an equipment virtual-real interaction digital drilling method based on a mixed reality technology, and the method comprises the steps: 1, building a virtual scene three-dimensional space model M and a virtual three-dimensional model m of a robot, marking the original position point, the end position point, the starting position point, the tail end working position point, each working position point and a charging position point of the m in the M, marking a curing interferent, a non-curing interferent and a wave band interference field existing in the M; step 2, calculating the total path time of the robot passing through the position points; 3, robot actions are classified to obtain a work operation action set, a real robot is used for carrying out work simulation drilling on a virtual scene or the virtual robot is used for carrying out work simulation drilling on a real scene, and work simulation drilling time is obtained; and 4, placing the real robots with the same drilling speed in a real scene for simulation drilling, and verifying the drilling action time.

Provided is a simulation system that simulates the operation of a unit including a robot and a device, the simulation system comprising: a virtual device that simulates the output of the device; and a virtual robot controller that simulates an operation of a robot controller that controls the robot on the basis of an output of the virtual device that outputs on the basis of a plurality of output histories of the device.

The application discloses a device virtual-real interaction digitalized drilling method based on a mixed reality technology, and comprises the following steps: 1, a virtual scene three-dimensional space model is built M and a virtual three-dimensional model of a robot m ; original and ending position points, starting and ending working position points, each working position point and a charging position point in m are marked M ; solidified interference, non-solidified interference and wave band interference fields existing in M are marked; 2, the total time of the robot in the path between each position point is calculated; 3, the robot action is classified to obtain a working operation action set, a real robot is used to simulate and drill the virtual scene or a virtual robot is used to simulate and drill the real scene, and a working simulation drilling time is obtained; and 4, the real robot with the same drilling speed is placed in the real scene to simulate and drill, and drilling action time is verified.

Embodiments of the disclosure relate to defining tasks in a scene using variation information for robotic technology systems and applications. In operation, some embodiments first receive or generate scene data. Some embodiments then generate variation amount information indicating one or more changes within the scene data. For example, in response to user input, a particular embodiment generates variation information, such as a variation layer. Generating such variation amount information is useful in various applications (such as robotic technology, simulation, graphics rendering, games, autonomous driving, and the like). For example, with regard to robotic technology, some embodiments store data corresponding to variation amount information as at least a portion of a task definition of a robotic task. Some embodiments then responsively cause or train one or more real-world robot components represented by one or more virtual robot components to perform the task in a real-world scene represented by the virtual scene based on the amount of variation information and the task definition.

The present application relates to the field of information processing. Disclosed are a control method for a robot gimbal guidance task point, and a related apparatus. The method comprises: obtaining an on-site image, wherein the on-site image is an image used for constructing a target three-dimensional scenario; obtaining a target image collection pose of a virtual robot at a corresponding target task point in the target three-dimensional scenario; on the basis of the on-site image, obtaining a target image corresponding to the target image collection pose; and on the basis of the target image, obtaining correction information, wherein the correction information is used for correcting an image collection pose of a real robot when the real robot detects the target task point in a real scenario.

The application discloses a wireless remote operation method and system of an augmented reality robot, relates to the technical field of augmented reality and robot control, and aims to improve the problems of "hand-eye incoordination" of an operator, low remote operation accuracy and efficiency caused by visual and tactile perception time delay and wireless transmission time delay. The method comprises the following steps: acquiring motion information of a multi-degree-of-freedom force feedback controller remote operation, and generating physical robot control instructions; controlling the physical robot to perform remote operation according to the physical robot control instructions, receiving environmental contact force information and displacement increment of the physical robot; calculating virtual additional force information, superimposing the environmental contact force information and the virtual additional force information, and generating tactile feedback information; calculating a displacement correction amount of a virtual robot, superimposing the displacement correction amount and the displacement increment of the virtual robot, obtaining virtual robot control instructions, and determining position information of each joint according to the virtual robot control instructions.

The invention relates to the technical field of remote operation of building robots, and discloses a remote operation method, device and equipment of a building robot and a storage medium, and the method comprises the steps: obtaining a three-dimensional digital twin model, and displaying the three-dimensional digital twin model in a virtual environment; determining an operation strategy based on interaction information of an operator to the virtual robot; and based on the operation strategy, the building robot is controlled to operate in the construction environment. According to the method, the bearing capacity, the environment sensing capacity and the high-precision capacity can be allocated to the building robot, the process experience and the dexterity capacity are allocated to the operator, the operator determines the operation strategy in the virtual environment, and then the building robot is controlled to work in the construction environment based on the operation strategy. The remote operation effect like a finger arm is effectively achieved, the building robot has the capacity of completing various technologies through man-machine interaction, and the working efficiency is greatly improved.

Aspects of the disclosure relate to monitoring, evaluating, and repairing bots in a hashchain-based distributed bot hub that process a workflow. In some embodiments, a computing platform may receive workflow information associated with performing a first workflow that includes executing one or more tasks using a plurality of virtual bots, identify a plurality of bots to process the first workflow, and compute, using a hash function, a hashchain for each identified bot of the plurality of bots. Thereafter, the computing platform may monitor the plurality of bots performing the one or more tasks of the first workflow by verifying tasks of identified bots based on analyzing hashchains of bots performing tasks of the first workflow, and, based on monitoring the plurality of bots, identify a potential anomalous activity by at least one bot.

The invention relates to a teleoperation simulation data acquisition system and method based on VR equipment. The system comprises head-mounted display equipment and a force feedback glove, a data processing module and a real-time synchronization module are arranged in the head-mounted display device; the head-mounted display device integrates an inertial measurement unit and a visual sensor and is used for collecting head and neck postures and hand action data; the force feedback glove serves as a lightweight peripheral and is used for achieving hand action collection and force feedback interaction. The data processing module is used for converting the action data into a mechanical arm control instruction through inverse kinematics solution and is also used for fusing a tactile signal to realize bidirectional control; and the real-time synchronization module is used for driving the virtual robot model to synchronously execute an instruction, and is also used for rendering an interaction state and feeding back force sense through the glove. Compared with a traditional scheme, collection of force data and action data is considered, and the data collection cost is reduced through an immersive virtual environment.

The control system (3) includes: a local controller (100) that causes a robot (2B, 2C, 10) to execute a plurality of tasks included in processing for a workpiece (9) in a real space; a virtual local controller (400) that causes a virtual robot to execute the plurality of tasks in a virtual space; a real information collection section (312) that collects real execution status information indicating an execution status of each of the plurality of tasks by the local controller (100); a virtual information collection section (314) that collects virtual execution status information indicating an execution status of each of the plurality of tasks by the virtual local controller (400); and a task comparison section (316) that extracts one or more deviated tasks in the plurality of tasks in which the real execution status information and the virtual execution status information deviate.