134 results about "Virtual robot" patented technology

The invention discloses an interaction test method for gait simulation of a humanoid robot, which is applied to a test platform. Comprising the following steps: controlling a current gait simulation model corresponding to a virtual robot to perform a gait simulation test in a virtual test environment in a current preset time period; the virtual test data is sent to the physical robot; controlling the physical robot to execute corresponding gait operation according to the virtual test data in the actual test environment, and sending the actual test data to the virtual robot; updating the current gait simulation model based on the virtual test data and the real test data; and continuing to perform the gait simulation test based on the updated gait simulation model, and ending the updating operation until the gait simulation test is ended, so as to generate a complete gait scheme. Therefore, by constructing a closed-loop interaction mechanism between the virtual robot and the physical robot, dynamic adjustment of the gait simulation model is achieved, the gait of the physical robot is restored, and the stability and precision of the gait of the physical robot are improved.

The invention provides a robot teleoperation method and system based on virtual reality and digital twinning, and particularly relates to the technical field of man-machine collaborative intelligent manufacturing and assemblation.The scheme comprises the steps that an industrial robot and wearable equipment are started; establishing a two-way communication connection between the virtual environment of the wearable device and the industrial robot, and rendering the virtual robot of the wearable device in the virtual environment; and inputting the target pose operation instruction into the virtual robot, responding to a received target motion track planned by the wearable device according to the target pose operation instruction, and controlling the industrial robot to execute an operation task according to the motion track. According to the scheme, information interaction between an operator and the robot can be achieved, so that immersive real-time robot teleoperation is achieved through instant and accurate operation commands and field feedback, and the teleoperation precision of complex tasks and flexible operation tasks can be remarkably improved.

An information processing apparatus includes a graph structure generation unit and a path planning unit. The graph structure generation unit generates a graph where a door on a passage has been replaced by a shunting line of a virtual robot. The path planning unit plans paths of a mobile body and the virtual mobile body such that the mobile body does not conflict with the virtual mobile body at an intersection between the passage and the shunting line when passing through the passage.

A real-world robot control system using artificial intelligence trained on a digital twin, according to the present invention, comprises: an artificial intelligence module that has undergone reinforcement learning by using a virtual robot on the digital twin replicating a real-world robot; one or more sensing units that are provided in the real-world robot and sense physical characteristics and electrical characteristics; a robot data collection unit that collects data from the sensing units; an error calculation unit that calculates an error between the virtual robot and the real-world robot; a disturbance observation unit that monitors an input signal transmitted from the module to the real-world robot, an output signal output from the real-world robot, and the error and estimates disturbance to generate feedback data for compensating for the disturbance when the disturbance is greater than or equal to a threshold value; and a robot driving module that transmits a control command for controlling the real-world robot by reflecting a control signal from the artificial intelligence module and the feedback data generated from the disturbance observation unit.

The invention discloses a data acquisition method for simulation learning, and the method comprises the steps: constructing a virtual robot, enabling the virtual robot to be aligned with a real world coordinate system, and obtaining a virtual robot of the real world coordinate system; performing position and posture matching on the hand tracking equipment and the virtual robot, and performing position and posture feedback on the hand tracking equipment through the position of the target object and the ideal operation path; in the data acquisition process, collision and overrun reminding is carried out through tactile feedback of the hand tracking device and visual feedback of the augmented reality glasses; and storing and preprocessing the collected data, and optimizing the preprocessed data through track consistency, over-limit check and collision frequency check.

A system and method are disclosed for training a machine learning model configured to determine a traversability of a real-world environment by a mobile robot based on an image of the real-world environment. The method advantageously generates high quality synthetic training data for training a traversability detection model to discriminate between traversable and untraversable regions in images captured of a real-world environment. The synthetic training data is generated through simulation of a virtual robot in a virtual environment. Once the traversability detection model is trained, it can be deployed to the mobile robot for the purpose of predicting traversability of a real-world environment.

A method for evaluating robots in simulation, when executed on a computing device, involves the following steps: obtaining a robot description file created based on the physical properties of a robot, obtaining an environment description file created based on an environment, and obtaining an obstacle description file. A physics simulation engine creates a virtual environment based on the environment description file and obstacle description file. A virtual robot is created based on the robot description file. When the virtual robot navigates from the starting area to the ending area within the virtual environment based on navigation information, the physics simulation engine outputs simulation information. The robot navigation program generates and sends navigation information to the physics simulation engine based on the simulation information. When the virtual robot reaches the ending area, the physics simulation engine outputs evaluation information.

A teaching system includes a teaching point generator that generates a teaching point of a robot; an operator that is operated by a user; an image generator that generates a VR image in which a virtual robot corresponding to the robot is placed in a VR space; and a display that displays the VR image. The image generator generates the VR image in which the virtual robot moves in accordance with operation to the operator. The teaching point generator generates a teaching point corresponding to a position of the virtual robot in the VR space generated by the image generator.

To provide a simulation method and simulation device, which enable appropriate simulation of a commercial establishment using service robots.SOLUTION: A simulation device 10 provided herein is configured to output predicted profits Pe of a commercial establishment reflecting a result of an operating simulation involving operating virtual robots R on an establishment layout set by a user, and can display the state of the operating simulation on a display device 90.SELECTED DRAWING: Figure 2

Provided is a method for controlling virtual robots. The method includes: starting a plurality of virtual robots constructed by using a finite state machine, wherein the virtual robots are configured to simulate, in a plurality of loops, a user operating an application program, and the finite state machine comprises a plurality of states, a plurality of events occurring at the plurality of states, and actions triggered by the plurality of events; determining, in each loop, a view in which the virtual robots are located in the application program, and determining the view as the state; simulating, in each loop, the user to select a service in the view, and determining the selected service as the event; and driving, in each loop, the virtual robots to execute, in the view, a user operation configured to trigger the service, and determining the user operation as the action.

A method (200) for adjusting the position of a component of a robot work unit (30) in a virtual robot work unit (40) simulating the robot work unit (30). The method (200) comprises: acquiring relative positioning information (202) between a robot (31) and an object carrier (32) from a controller (10) for the robot (31) in a robot work unit (30); and adjusting the relative position between a virtual robot (41) corresponding to the robot (31) in the virtual robot work unit (40) and a virtual object carrier (42) corresponding to the object carrier (32) in the virtual robot work unit (40) on the basis of the relative position information. Relative positioning information between the robot (31) and the object carrier (32) can thus be automatically obtained and digitally analyzed.

The invention discloses a control method and a related device for guiding task points by a robot holder, and relates to the field of information processing, and the method comprises the steps: obtaining a field image which is an image used for constructing a target three-dimensional scene; obtaining a target image acquisition pose of a virtual robot corresponding to a target task point in the target three-dimensional scene; obtaining a target image corresponding to the target image acquisition pose according to the field image; and correction information is obtained according to the target image, and the correction information is used for correcting an image acquisition pose of a real robot when the real robot detects the target task point in a real scene.

Provided in the present invention are a virtual-robot and physical-robot collaborative safety control method and apparatus based on mixed reality and a digital twin, which method and apparatus specifically relate to the technical field of smart manufacturing and assembly involving virtual-robot and physical-robot collaboration. In the solution, the method comprises: acquiring first posture information of a physical robot, and acquiring second posture information of a virtual robot; on the basis of the first posture information and the second posture information, performing pose registration on the physical robot and the virtual robot, so as to obtain a registration result; on the basis of the registration result and a preset virtual-robot and physical-robot workspace in combination with deep reinforcement learning, performing path planning to determine a virtual-robot and physical-robot collaborative motion path; and on the basis of the virtual-robot and physical-robot collaborative motion path, executing a virtual-robot and physical-robot collaborative safety control action. The solution uses a mixed-reality device to extract virtual-robot and physical-robot posture information, implements virtual-robot and physical-robot pose registration by means of a virtual-real space mapping method, and designs a bidirectional virtual-robot and physical-robot collaborative safety interaction policy based on deep reinforcement learning, thereby effectively improving the safety and effectiveness of a virtual-robot and physical-robot interaction operation.

The invention relates to an interaction method and device of a virtual robot, a storage medium and electronic equipment. The method comprises the following steps: acquiring interaction information which is input by a user and interacts with the virtual robot; the interaction information is input into a control model of the virtual robot, and the control model is obtained through training by taking the interaction information input by a user of a video live broadcast platform and behavior response information of an anchor for the interaction information as model training samples; and according to behavior control information output by the control model based on the interaction information, performing behavior control on the virtual robot. According to the embodiment of the invention, interaction between the virtual robot and the user is realized, the real-time performance, the flexibility and the applicability of the virtual robot are improved, and the requirements of emotion and action communication between the user and the virtual robot are met.

The embodiment of the invention provides a virtual robot generation method and device, equipment and a storage medium. Displaying a content input entry; wherein the content input entry comprises a character input entry and / or a content uploading entry; receiving demand information input through the character input entry, and / or receiving training data input through the content uploading entry; generating a virtual robot based on the demand information and / or the training data; wherein the virtual robot is used for performing question and answer interaction with a user. According to the generation method of the virtual robot provided by the embodiment of the invention, the virtual robot is automatically generated based on the input demand information and / or training data, manual configuration of a user is not needed, and not only can the generation efficiency of the virtual robot be improved, but also the interaction capability of the virtual robot can be improved.

For a (stationary or mobile) robot within a monitored space, a safety subsystem generates a virtual representation of the robot using position and / or orientation data associated with the robot, detects an unexpected object within the monitored space using sensor data from a plurality of (external / internal) sensors, detects a virtual halt collision between the virtual robot and the virtual unexpected object based on a halt zone corresponding to the robot or the unexpected object, and generates an instruction to halt the normal operations of the robot. The (dynamic) virtual robot representation may include a robot arm movable with respect to the robot's base. The monitored space is generated by removing ignore zones corresponding to expected objects, including an ignore zone corresponding to the robot itself, from an initial (e.g., spherical) version of the monitored space. Regions having insufficient sensor coverage (i.e., IPZs) are treated as unexpected objects within the monitored space.

The invention discloses a robot navigation method based on reinforcement learning security constraint and danger perception, and relates to the technical field of robots and artificial intelligence, and the robot navigation method comprises the following steps: when a robot is controlled to execute a navigation task in a target area, using a security filter SF to perform security constraint on the execution action of the robot; and when the robot executes the navigation task, the danger perception virtual robot HAVR is utilized to obtain experience in the dangerous area so as to enhance the ability of the robot to explore in the dangerous area. The action generated by reinforcement learning can be dynamically adjusted, so that the action meets the safety constraint, and the safety of the robot in the reinforcement learning training process is further enhanced; by introducing the HAVR with risk-driven rewards, the robot can still obtain experience in a dangerous area while introducing security constraints, and the exploration security of the robot is further enhanced while the performance of the model after convergence is ensured.

A system and method for training a machine learning model are disclosed, designed to determine the traverseability of a real-world environment by a mobile robot based on an image of that environment. The method advantageously generates high-quality synthetic training data for training a traverse detection model to distinguish between traversable and impassable areas in captured images of a real-world environment. The synthetic training data is generated by simulating a virtual robot in a virtual environment. Once the traverse detection model is trained, it can be applied to the mobile robot to predict the traverseability of a real-world environment.

A simulation data creation method, a preview image display method, and a simulation device that enable grasping of the contents of simulation before construction of a virtual space. The simulation data creation method includes the following steps: (a) causing a virtual robot disposed in a virtual work space to act based on an operation instruction by a user, and displaying a simulator image representing the action of the virtual robot in the virtual work space on a display device, the virtual work space being a virtual three-dimensional work space; (b) outputting a snapshot representing a two-dimensional image obtained by a virtual camera capturing a state in which a virtual three-dimensional object including the virtual robot is disposed in the virtual work space; and (c) saving, as simulation data, in a memory, an action program representing a target action of the virtual robot based on the operation instruction, layout data representing positions and postures of objects in the virtual work space, attribute data representing attributes of the objects, and the snapshot.

The application provides an interactive method and device of a virtual robot and a storage medium, and the method comprises the following steps: in response to a virtual robot interactive request triggered by a user, collecting multi-modal data of a live room of the user in real time; then, processing the collected data to convert the multi-modal data into text information in a unified format, wherein the text information is used to uniformly represent feature information of the multi-modal data; then, generating a response text according to the text information; determining interactive information based on the response text, and controlling the virtual robot to perform an interactive operation based on the interactive information. In the above scheme, the multi-modal data of the live room is analyzed and processed to fully mine the live room information, so that the virtual robot can automatically and intelligently interact with the user in combination with various live room information, which not only reduces the operation difficulty of the user, but also improves the live room activity, greatly improving the enthusiasm of the anchor and the experience of the audience.

The present invention discloses a coordinate conversion method, device, and computer-readable medium for a robot, belonging to the field of robot control technology. A specific implementation of the method includes: first, obtaining the self-coordinate system of a virtual robot in a simulation environment, and the coordinate values ​​of the end of the virtual robot relative to the self-coordinate system; then, reading the coordinate values ​​of the end of a physical robot in a real environment relative to the system coordinate system; and finally, determining the system coordinate system of the physical robot in a real environment based on the self-coordinate system of the virtual robot, the coordinate values ​​of the end of the virtual robot, and the coordinate values ​​of the end of the physical robot. Thus, by converting the self-coordinate system of the virtual robot into the system coordinate system, the system coordinates of the physical robot in the real environment can be accurately obtained, so that without knowing the coordinate system referenced by the physical robot's operation, the system coordinate system can be used as a reference to effectively control the operation of the physical robot.

A control device includes a processor that calculates one or more predetermined command values for one or more robots to undergo synchronous control in predetermined control cycles, an output unit that outputs the one or more predetermined command values in each of the predetermined control cycles, and a generator that generates an output signal for a virtual robot. The virtual robot is virtually defined in relation to the synchronous control. The processor calculates the one or more predetermined command values using the output signal for the virtual robot generated by the generator.

A teaching device constructs, in a virtual space, a virtual robot system in which a virtual 3D model of a robot and a virtual 3D model of a peripheral structure of the robot are arranged, and teaches a moving path of the robot. The teaching device includes an acquisition unit configured to acquire information about a geometric error between the virtual 3D models, and a correction unit configured to correct the moving path of the robot in accordance with the information acquired by the acquisition unit.

The application provides a robot process debugging system and method based on three-dimensional visualization, and relates to the technical field of robot processing debugging, and comprises the following steps: a data processing module is used for acquiring saved workstation data and robot data in a robot workstation, and extracting robot trajectory point data and process parameters from the robot data; a three-dimensional display module is used for establishing a virtual workstation model containing a virtual robot model according to the workstation data and the robot data, and establishing a virtual processing trajectory of the virtual robot model according to the trajectory point data and the process parameters; and a debugging module is used for debugging process parameters to obtain process debugging parameters according to externally input debugging instructions, and the three-dimensional display module synchronously displays the adjustment of the virtual processing trajectory according to the process debugging parameters. The beneficial effect is to provide a three-dimensional visual interactive interface, reduce the operation complexity, improve the process debugging efficiency, improve the interactive experience, and make the process debugging more convenient.

The invention discloses a digital twin system construction method based on Isaac Sim and UDP communication protocols, and belongs to the technical field of robot intelligent control. The system adopts a virtual-real collaborative framework to construct three core modules: a virtual robot model based on a URDF format is established through an Isaac Sim platform, and geometric structure parameters and kinematic characteristics of a real robot are accurately matched; a dynamic control strategy module is constructed by applying an RMPflow algorithm, an optimal motion path is generated through a synergistic effect of a target RMP and a collision avoidance RMP, and accurate target tracking and active obstacle avoidance of an end effector are realized; a bidirectional communication module supporting a UDP protocol is designed, a Socket communication link is constructed based on a local area network, a lightweight data packet is adopted to transmit a control signal and a robot state, and an ROS is used to realize control and state acquisition of a physical robot. The system comprises a virtual control real mode and a real control virtual mode, and supports feedback and synchronization of a virtual environment and a physical entity state. In conclusion, the invention provides the robot digital twin system construction method based on the Isaac Sim and the UDP communication protocol.

Disclosed is a robotic system (104) and a method (600) having a control unit (202). The control unit is configured to generate a virtual environment having a virtual robot (402) based on a received set of first parameters to emulate a real environment. The control unit (202) is configured to actuate the virtual robot (402) to simulate an operation of a robot (108). At least the virtual robot (402) actuates independently, or the robot (108) and the virtual robot (402) actuate in synchrony based on the simulation. Further, at least the virtual robot (402) actuates independently, depending on the set of second parameters or the robot (108) and the virtual robot (402) actuates, depending on the set of second parameters. This configuration ensures multi modal operation of the robotic system (104), thus ensuring ease of operation of the robot (108).

A lab system identifies a set of steps associated with a protocol for a lab meant to be performed by a robot within the lab using equipment and reagents. The lab system renders, within a user interface, a virtual representation of the lab, a virtual robot, and virtual equipment and reagents. Responsive to operating in a first mode, the lab system simulates the identified set of steps identify virtual positions of the virtual robot within the lab as the virtual robot performs the steps and modifies the virtual representation of the lab to mirror the identified positions of the virtual robot in real-time. Responsive to operating in a second mode, the lab system identifies positions of the robot within the lab as the robot performs the identified set of steps and modifies the virtual representation of the lab to mirror the identified positions of the robot in real-time.

This disclosure provides a robot intelligent visualization method, device, storage medium, and program product. The method includes: acquiring robot pose data, joint data, and point cloud data of a real-world scene; using a coordinate transformation method to convert the position data in the pose and joint data from the robot's body coordinate system to a three-dimensional scene coordinate system; based on the coordinate-transformed pose and joint data, performing real-time posture and skeletal synchronization of the robot to generate a virtual robot model that accurately reproduces the robot's real movements; constructing a three-dimensional scene based on the point cloud data and embedding the virtual robot model into the three-dimensional scene; and triggering a corresponding task mode when the virtual robot model is within the task point sensing range of the three-dimensional scene. This method achieves precise spatiotemporal synchronization and fusion between the physical robot and the virtual scene through multi-source data fusion and coordinate system unification, realistically reproducing robot movements and intelligently triggering tasks.

Provided is a teaching device (50) for teaching a robot, the teaching device (50) being provided with: a storage unit (156) for storing mechanism data for obtaining the relationship between the angular position of a joint part of the robot and the tip position of the robot, the mechanism data including a mechanism error parameter of the actual robot; and a virtual robot control unit (153) that, on the basis of the mechanism data, controls the operation of a virtual robot such that a position error generated in the actual robot by the mechanism error parameter is generated in the virtual robot.

The invention discloses a crankshaft defect coping method and system based on digital twinning. The method is characterized by comprising the following steps: constructing a digital twin virtual environment consistent with a real environment, wherein virtual camera parameters are consistent with a real multi-view vision system; a crankshaft defect image is collected through a real multi-view vision system; the defect pixel positions are mapped to the surface of the virtual crankshaft to form virtual defects; in the virtual environment, the virtual robot plans a grinding path and generates an instruction; and the instruction is sent to the real robot to execute grinding. The system comprises a real physical system, a digital twin virtual system and a control communication system. According to the method, path planning and verification are completed in the digital twin virtual environment, and then the instruction is issued to the real system for execution, so that the precision, the safety and the automation degree of crankshaft defect grinding are effectively improved, and the collision risk and the debugging cost are reduced.