82 results about "Virtual robot" patented technology

The invention discloses an interaction test method for gait simulation of a humanoid robot, which is applied to a test platform. Comprising the following steps: controlling a current gait simulation model corresponding to a virtual robot to perform a gait simulation test in a virtual test environment in a current preset time period; the virtual test data is sent to the physical robot; controlling the physical robot to execute corresponding gait operation according to the virtual test data in the actual test environment, and sending the actual test data to the virtual robot; updating the current gait simulation model based on the virtual test data and the real test data; and continuing to perform the gait simulation test based on the updated gait simulation model, and ending the updating operation until the gait simulation test is ended, so as to generate a complete gait scheme. Therefore, by constructing a closed-loop interaction mechanism between the virtual robot and the physical robot, dynamic adjustment of the gait simulation model is achieved, the gait of the physical robot is restored, and the stability and precision of the gait of the physical robot are improved.

A system and method are disclosed for training a machine learning model configured to determine a traversability of a real-world environment by a mobile robot based on an image of the real-world environment. The method advantageously generates high quality synthetic training data for training a traversability detection model to discriminate between traversable and untraversable regions in images captured of a real-world environment. The synthetic training data is generated through simulation of a virtual robot in a virtual environment. Once the traversability detection model is trained, it can be deployed to the mobile robot for the purpose of predicting traversability of a real-world environment.

Provided is a method for controlling virtual robots. The method includes: starting a plurality of virtual robots constructed by using a finite state machine, wherein the virtual robots are configured to simulate, in a plurality of loops, a user operating an application program, and the finite state machine comprises a plurality of states, a plurality of events occurring at the plurality of states, and actions triggered by the plurality of events; determining, in each loop, a view in which the virtual robots are located in the application program, and determining the view as the state; simulating, in each loop, the user to select a service in the view, and determining the selected service as the event; and driving, in each loop, the virtual robots to execute, in the view, a user operation configured to trigger the service, and determining the user operation as the action.

Provided in the present invention are a virtual-robot and physical-robot collaborative safety control method and apparatus based on mixed reality and a digital twin, which method and apparatus specifically relate to the technical field of smart manufacturing and assembly involving virtual-robot and physical-robot collaboration. In the solution, the method comprises: acquiring first posture information of a physical robot, and acquiring second posture information of a virtual robot; on the basis of the first posture information and the second posture information, performing pose registration on the physical robot and the virtual robot, so as to obtain a registration result; on the basis of the registration result and a preset virtual-robot and physical-robot workspace in combination with deep reinforcement learning, performing path planning to determine a virtual-robot and physical-robot collaborative motion path; and on the basis of the virtual-robot and physical-robot collaborative motion path, executing a virtual-robot and physical-robot collaborative safety control action. The solution uses a mixed-reality device to extract virtual-robot and physical-robot posture information, implements virtual-robot and physical-robot pose registration by means of a virtual-real space mapping method, and designs a bidirectional virtual-robot and physical-robot collaborative safety interaction policy based on deep reinforcement learning, thereby effectively improving the safety and effectiveness of a virtual-robot and physical-robot interaction operation.

The embodiment of the invention provides a virtual robot generation method and device, equipment and a storage medium. Displaying a content input entry; wherein the content input entry comprises a character input entry and / or a content uploading entry; receiving demand information input through the character input entry, and / or receiving training data input through the content uploading entry; generating a virtual robot based on the demand information and / or the training data; wherein the virtual robot is used for performing question and answer interaction with a user. According to the generation method of the virtual robot provided by the embodiment of the invention, the virtual robot is automatically generated based on the input demand information and / or training data, manual configuration of a user is not needed, and not only can the generation efficiency of the virtual robot be improved, but also the interaction capability of the virtual robot can be improved.

For a (stationary or mobile) robot within a monitored space, a safety subsystem generates a virtual representation of the robot using position and / or orientation data associated with the robot, detects an unexpected object within the monitored space using sensor data from a plurality of (external / internal) sensors, detects a virtual halt collision between the virtual robot and the virtual unexpected object based on a halt zone corresponding to the robot or the unexpected object, and generates an instruction to halt the normal operations of the robot. The (dynamic) virtual robot representation may include a robot arm movable with respect to the robot's base. The monitored space is generated by removing ignore zones corresponding to expected objects, including an ignore zone corresponding to the robot itself, from an initial (e.g., spherical) version of the monitored space. Regions having insufficient sensor coverage (i.e., IPZs) are treated as unexpected objects within the monitored space.

The invention discloses a robot navigation method based on reinforcement learning security constraint and danger perception, and relates to the technical field of robots and artificial intelligence, and the robot navigation method comprises the following steps: when a robot is controlled to execute a navigation task in a target area, using a security filter SF to perform security constraint on the execution action of the robot; and when the robot executes the navigation task, the danger perception virtual robot HAVR is utilized to obtain experience in the dangerous area so as to enhance the ability of the robot to explore in the dangerous area. The action generated by reinforcement learning can be dynamically adjusted, so that the action meets the safety constraint, and the safety of the robot in the reinforcement learning training process is further enhanced; by introducing the HAVR with risk-driven rewards, the robot can still obtain experience in a dangerous area while introducing security constraints, and the exploration security of the robot is further enhanced while the performance of the model after convergence is ensured.

A system and method for training a machine learning model are disclosed, designed to determine the traverseability of a real-world environment by a mobile robot based on an image of that environment. The method advantageously generates high-quality synthetic training data for training a traverse detection model to distinguish between traversable and impassable areas in captured images of a real-world environment. The synthetic training data is generated by simulating a virtual robot in a virtual environment. Once the traverse detection model is trained, it can be applied to the mobile robot to predict the traverseability of a real-world environment.

A simulation data creation method, a preview image display method, and a simulation device that enable grasping of the contents of simulation before construction of a virtual space. The simulation data creation method includes the following steps: (a) causing a virtual robot disposed in a virtual work space to act based on an operation instruction by a user, and displaying a simulator image representing the action of the virtual robot in the virtual work space on a display device, the virtual work space being a virtual three-dimensional work space; (b) outputting a snapshot representing a two-dimensional image obtained by a virtual camera capturing a state in which a virtual three-dimensional object including the virtual robot is disposed in the virtual work space; and (c) saving, as simulation data, in a memory, an action program representing a target action of the virtual robot based on the operation instruction, layout data representing positions and postures of objects in the virtual work space, attribute data representing attributes of the objects, and the snapshot.

The application provides an interactive method and device of a virtual robot and a storage medium, and the method comprises the following steps: in response to a virtual robot interactive request triggered by a user, collecting multi-modal data of a live room of the user in real time; then, processing the collected data to convert the multi-modal data into text information in a unified format, wherein the text information is used to uniformly represent feature information of the multi-modal data; then, generating a response text according to the text information; determining interactive information based on the response text, and controlling the virtual robot to perform an interactive operation based on the interactive information. In the above scheme, the multi-modal data of the live room is analyzed and processed to fully mine the live room information, so that the virtual robot can automatically and intelligently interact with the user in combination with various live room information, which not only reduces the operation difficulty of the user, but also improves the live room activity, greatly improving the enthusiasm of the anchor and the experience of the audience.

The present invention discloses a coordinate conversion method, device, and computer-readable medium for a robot, belonging to the field of robot control technology. A specific implementation of the method includes: first, obtaining the self-coordinate system of a virtual robot in a simulation environment, and the coordinate values ​​of the end of the virtual robot relative to the self-coordinate system; then, reading the coordinate values ​​of the end of a physical robot in a real environment relative to the system coordinate system; and finally, determining the system coordinate system of the physical robot in a real environment based on the self-coordinate system of the virtual robot, the coordinate values ​​of the end of the virtual robot, and the coordinate values ​​of the end of the physical robot. Thus, by converting the self-coordinate system of the virtual robot into the system coordinate system, the system coordinates of the physical robot in the real environment can be accurately obtained, so that without knowing the coordinate system referenced by the physical robot's operation, the system coordinate system can be used as a reference to effectively control the operation of the physical robot.

A teaching device constructs, in a virtual space, a virtual robot system in which a virtual 3D model of a robot and a virtual 3D model of a peripheral structure of the robot are arranged, and teaches a moving path of the robot. The teaching device includes an acquisition unit configured to acquire information about a geometric error between the virtual 3D models, and a correction unit configured to correct the moving path of the robot in accordance with the information acquired by the acquisition unit.

The application provides a robot process debugging system and method based on three-dimensional visualization, and relates to the technical field of robot processing debugging, and comprises the following steps: a data processing module is used for acquiring saved workstation data and robot data in a robot workstation, and extracting robot trajectory point data and process parameters from the robot data; a three-dimensional display module is used for establishing a virtual workstation model containing a virtual robot model according to the workstation data and the robot data, and establishing a virtual processing trajectory of the virtual robot model according to the trajectory point data and the process parameters; and a debugging module is used for debugging process parameters to obtain process debugging parameters according to externally input debugging instructions, and the three-dimensional display module synchronously displays the adjustment of the virtual processing trajectory according to the process debugging parameters. The beneficial effect is to provide a three-dimensional visual interactive interface, reduce the operation complexity, improve the process debugging efficiency, improve the interactive experience, and make the process debugging more convenient.

Disclosed is a robotic system (104) and a method (600) having a control unit (202). The control unit is configured to generate a virtual environment having a virtual robot (402) based on a received set of first parameters to emulate a real environment. The control unit (202) is configured to actuate the virtual robot (402) to simulate an operation of a robot (108). At least the virtual robot (402) actuates independently, or the robot (108) and the virtual robot (402) actuate in synchrony based on the simulation. Further, at least the virtual robot (402) actuates independently, depending on the set of second parameters or the robot (108) and the virtual robot (402) actuates, depending on the set of second parameters. This configuration ensures multi modal operation of the robotic system (104), thus ensuring ease of operation of the robot (108).

This disclosure provides a robot intelligent visualization method, device, storage medium, and program product. The method includes: acquiring robot pose data, joint data, and point cloud data of a real-world scene; using a coordinate transformation method to convert the position data in the pose and joint data from the robot's body coordinate system to a three-dimensional scene coordinate system; based on the coordinate-transformed pose and joint data, performing real-time posture and skeletal synchronization of the robot to generate a virtual robot model that accurately reproduces the robot's real movements; constructing a three-dimensional scene based on the point cloud data and embedding the virtual robot model into the three-dimensional scene; and triggering a corresponding task mode when the virtual robot model is within the task point sensing range of the three-dimensional scene. This method achieves precise spatiotemporal synchronization and fusion between the physical robot and the virtual scene through multi-source data fusion and coordinate system unification, realistically reproducing robot movements and intelligently triggering tasks.

The invention discloses a crankshaft defect coping method and system based on digital twinning. The method is characterized by comprising the following steps: constructing a digital twin virtual environment consistent with a real environment, wherein virtual camera parameters are consistent with a real multi-view vision system; a crankshaft defect image is collected through a real multi-view vision system; the defect pixel positions are mapped to the surface of the virtual crankshaft to form virtual defects; in the virtual environment, the virtual robot plans a grinding path and generates an instruction; and the instruction is sent to the real robot to execute grinding. The system comprises a real physical system, a digital twin virtual system and a control communication system. According to the method, path planning and verification are completed in the digital twin virtual environment, and then the instruction is issued to the real system for execution, so that the precision, the safety and the automation degree of crankshaft defect grinding are effectively improved, and the collision risk and the debugging cost are reduced.

Provided is a simulation system that simulates an operation of a cell including a robot and a device, wherein the simulation system comprises a virtual device that simulates an output of the device and a a virtual robot controller that simulates, according to an output of the virtual device, an operation of a robot controller that controls the robot, and the virtual device performs an output based on a plurality of output histories of the device.

A teaching device for teaching a robot includes: a storage unit that stores mechanism data for finding a relationship between a tip position of a robot and an angle position at a joint portion of the robot, the mechanism data including a mechanism error parameter of an actual robot; and a virtual robot control unit that controls the operation of a virtual robot on the basis of the mechanism data such that an error in position that occurs in the actual robot due to the mechanism error parameter occurs in the virtual robot.

Embodiments of the present application relate to the field of virtual simulation technologies, and provide a robot offset simulation method and apparatus, a robot, an electronic device, and a storage medium. The robot offset simulation method includes: establishing a reference path for a virtual robot in simulation software, where the reference path is a path along which the virtual robot moves to transfer a reference object from a preset starting point to a destination location and then returns to the preset starting point; receiving location information of a target object that is sent by a PLC; determining an offset of the virtual robot based on the location information, where the offset is a location offset of the target object relative to the reference object; and controlling, based on the reference path and the offset, the virtual robot to move to transfer the target object.

The invention discloses a robot teaching method based on a sphere bounding box and related equipment. The robot teaching method comprises the following steps: acquiring a chemical experiment virtual scene corresponding to a real operation environment; collecting motion data of teaching input equipment, and driving a virtual robot in the chemical experiment virtual scene to perform chemical action drilling based on the motion data; in the moving process of the virtual robot, the space occupation information is updated in real time; obtaining a collision detection result between the sphere bounding boxes according to the updated space occupation information; and generating a control instruction sent to a real robot according to the collision detection result. According to the method, traditional full-grid collision detection is abandoned, collision detection is carried out in a simplified sphere bounding box model mode, low-delay real-time anti-collision early warning is achieved, and the stability and safety of the chemical experiment operation process are guaranteed.

Systems and methods for human-robot communication. More particularly, some embodiments use augmented reality to facilitate communication of robot intention and teleoperation in human-robot cooperative environments. Various embodiments of the present technology provide a middleware that integrates augmented reality (AR) with novel teleoperation interfaces to increase operation effectiveness, support the user in conducting concurrent work, and decrease stress. Various embodiments provide predictive graphical interfaces such that a teleoperator controls a virtual robot surrogate, rather than directly operating the robot itself, providing the user with foresight regarding where the physical robot will end up and how it will get there. In accordance with various embodiments a user may select between two AR interfaces using such a surrogate: one focused on real-time control and one inspired by waypoint delegation.

Example implementations may include capturing virtual hot execution, for example by taking a screen recording of a desktop screen of a virtual machine where the execution is taking place. The contents, such as video such as the recording and / or associated data, may be processed, buffered, stored, and / or transmitted before being streamed (e.g., to a user). Thus, these implementations may involve initiating a virtual bot to execute an operation; generating a graphical representation of the virtual bot executing the operation; and transmitting, to a client device, the graphical representation of the operation for display at the client device.

An information processing apparatus includes a processor configured to simulate a virtual robot and a virtual image pickup apparatus moving in an interlocked manner with each other in a virtual space. The processor is configured to associate setting information of the virtual image pickup apparatus with a teaching point of the virtual robot.

Embodiments of present disclosure provide a method, an apparatus, an electronic device, a computer-readable medium and a computer program product of controlling a virtual robot on a robot training platform. The method comprises receiving, from an extended reality (XR) device, a user operation for controlling the virtual robot on an XR platform. The method further comprises determining, by the XR platform, a control result for the user operation based on the user operation and information associated with the virtual robot. The method further comprises transmitting, to the XR device, the control result to cause the XR device to present a rendered control result of the virtual robot.

The present application specifically relates to the technical field of human-machine collaborative intelligent manufacturing and assembly, and provides a virtual reality and digital twin-based robot teleoperation method and system. The solution comprises: starting up an industrial robot and a wearable device; establishing a bidirectional communication connection between a virtual environment of the wearable device and the industrial robot, and rendering a virtual robot of the wearable device in the virtual environment; and inputting a target pose operation instruction into the virtual robot, and in response to receiving a target motion trajectory planned by the wearable device according to the target pose operation instruction, controlling the industrial robot to perform an operation task according to the motion trajectory. The solution enables information interaction between an operator and a robot, thereby achieving immersive real-time robot teleoperation by means of instant and accurate operation commands and on-site feedback, and significantly improving teleoperation accuracy for complex tasks and flexible manipulation tasks.

The application relates to a collaborative robot interaction control method and device, a storage medium and electronic equipment. The method comprises the following steps: displaying a virtual robot corresponding to a physical robot in a three-dimensional scene; in response to a user triggering operation on a target joint of the virtual robot, displaying a normal line corresponding to the target joint, and displaying a draggable anchor point on the normal line; in response to a user dragging operation on the anchor point, determining motion control data of the target joint according to a dragging direction and a displacement of the dragging operation, controlling the physical robot to move based on the motion control data, and updating a pose of the virtual robot according to a real-time motion state of the physical robot. The application can intuitively display a robot motion displacement and an instantaneous speed in a graphical manner.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for robot programming. One of the methods comprises generating an interactive user interface that includes an illustration of a virtual robot corresponding to a physical robot; receiving first user input data specifying a first target pose of the virtual robot; causing the physical robot to traverse to the first target pose while updating in real-time the illustration of the virtual robot as the physical robot transitions to the first target pose; receiving a user request to switch from operating in a synchronized mode to operating in an unsynchronized mode; receiving second user input data specifying a second target pose of the virtual robot; and generating an animation of the virtual robot transitioning from the first target pose to the second target pose but withholding causing the physical robot to traverse to the second target pose.

Apparatuses, systems, and techniques to perform collision-free motion generation (e.g., to operate a real-world or virtual robot). In at least one embodiment, at least a portion of the collision-free motion generation is performed in parallel.

This invention relates to the field of industrial internet, and particularly to a method for bidirectional motion synchronization between a robot and its digital twin. The method includes the following steps: constructing a robot data model by analyzing the robot's characteristic information; importing the robot's 3D model into a digital twin system to form a virtual robot model; embedding the robot's kinematic model into the digital twin system as a controller for the virtual robot; constructing OPC UA bidirectional servers on both the physical and virtual robot sides; establishing data acquisition modules on both the physical and virtual robot sides to collect robot motion information and transmit it to the corresponding data nodes of the OPC UA servers; and employing a continuous motion synchronization method based on trapezoidal velocity control. This invention enables bidirectional motion synchronization control between a virtual robot and a physical robot based on the OPC UA protocol, achieving a new, intuitive, and accurate way of operating robots.

The invention discloses an equipment virtual-real interaction digital drilling method based on a mixed reality technology, and the method comprises the steps: 1, building a virtual scene three-dimensional space model M and a virtual three-dimensional model m of a robot, marking the original position point, the end position point, the starting position point, the tail end working position point, each working position point and a charging position point of the m in the M, marking a curing interferent, a non-curing interferent and a wave band interference field existing in the M; step 2, calculating the total path time of the robot passing through the position points; 3, robot actions are classified to obtain a work operation action set, a real robot is used for carrying out work simulation drilling on a virtual scene or the virtual robot is used for carrying out work simulation drilling on a real scene, and work simulation drilling time is obtained; and 4, placing the real robots with the same drilling speed in a real scene for simulation drilling, and verifying the drilling action time.