113 results about "Virtual robot" patented technology

The invention discloses an interaction test method for gait simulation of a humanoid robot, which is applied to a test platform. Comprising the following steps: controlling a current gait simulation model corresponding to a virtual robot to perform a gait simulation test in a virtual test environment in a current preset time period; the virtual test data is sent to the physical robot; controlling the physical robot to execute corresponding gait operation according to the virtual test data in the actual test environment, and sending the actual test data to the virtual robot; updating the current gait simulation model based on the virtual test data and the real test data; and continuing to perform the gait simulation test based on the updated gait simulation model, and ending the updating operation until the gait simulation test is ended, so as to generate a complete gait scheme. Therefore, by constructing a closed-loop interaction mechanism between the virtual robot and the physical robot, dynamic adjustment of the gait simulation model is achieved, the gait of the physical robot is restored, and the stability and precision of the gait of the physical robot are improved.

A real-world robot control system using artificial intelligence trained on a digital twin, according to the present invention, comprises: an artificial intelligence module that has undergone reinforcement learning by using a virtual robot on the digital twin replicating a real-world robot; one or more sensing units that are provided in the real-world robot and sense physical characteristics and electrical characteristics; a robot data collection unit that collects data from the sensing units; an error calculation unit that calculates an error between the virtual robot and the real-world robot; a disturbance observation unit that monitors an input signal transmitted from the module to the real-world robot, an output signal output from the real-world robot, and the error and estimates disturbance to generate feedback data for compensating for the disturbance when the disturbance is greater than or equal to a threshold value; and a robot driving module that transmits a control command for controlling the real-world robot by reflecting a control signal from the artificial intelligence module and the feedback data generated from the disturbance observation unit.

A system and method are disclosed for training a machine learning model configured to determine a traversability of a real-world environment by a mobile robot based on an image of the real-world environment. The method advantageously generates high quality synthetic training data for training a traversability detection model to discriminate between traversable and untraversable regions in images captured of a real-world environment. The synthetic training data is generated through simulation of a virtual robot in a virtual environment. Once the traversability detection model is trained, it can be deployed to the mobile robot for the purpose of predicting traversability of a real-world environment.

Provided is a method for controlling virtual robots. The method includes: starting a plurality of virtual robots constructed by using a finite state machine, wherein the virtual robots are configured to simulate, in a plurality of loops, a user operating an application program, and the finite state machine comprises a plurality of states, a plurality of events occurring at the plurality of states, and actions triggered by the plurality of events; determining, in each loop, a view in which the virtual robots are located in the application program, and determining the view as the state; simulating, in each loop, the user to select a service in the view, and determining the selected service as the event; and driving, in each loop, the virtual robots to execute, in the view, a user operation configured to trigger the service, and determining the user operation as the action.

Provided in the present invention are a virtual-robot and physical-robot collaborative safety control method and apparatus based on mixed reality and a digital twin, which method and apparatus specifically relate to the technical field of smart manufacturing and assembly involving virtual-robot and physical-robot collaboration. In the solution, the method comprises: acquiring first posture information of a physical robot, and acquiring second posture information of a virtual robot; on the basis of the first posture information and the second posture information, performing pose registration on the physical robot and the virtual robot, so as to obtain a registration result; on the basis of the registration result and a preset virtual-robot and physical-robot workspace in combination with deep reinforcement learning, performing path planning to determine a virtual-robot and physical-robot collaborative motion path; and on the basis of the virtual-robot and physical-robot collaborative motion path, executing a virtual-robot and physical-robot collaborative safety control action. The solution uses a mixed-reality device to extract virtual-robot and physical-robot posture information, implements virtual-robot and physical-robot pose registration by means of a virtual-real space mapping method, and designs a bidirectional virtual-robot and physical-robot collaborative safety interaction policy based on deep reinforcement learning, thereby effectively improving the safety and effectiveness of a virtual-robot and physical-robot interaction operation.

The embodiment of the invention provides a virtual robot generation method and device, equipment and a storage medium. Displaying a content input entry; wherein the content input entry comprises a character input entry and / or a content uploading entry; receiving demand information input through the character input entry, and / or receiving training data input through the content uploading entry; generating a virtual robot based on the demand information and / or the training data; wherein the virtual robot is used for performing question and answer interaction with a user. According to the generation method of the virtual robot provided by the embodiment of the invention, the virtual robot is automatically generated based on the input demand information and / or training data, manual configuration of a user is not needed, and not only can the generation efficiency of the virtual robot be improved, but also the interaction capability of the virtual robot can be improved.

For a (stationary or mobile) robot within a monitored space, a safety subsystem generates a virtual representation of the robot using position and / or orientation data associated with the robot, detects an unexpected object within the monitored space using sensor data from a plurality of (external / internal) sensors, detects a virtual halt collision between the virtual robot and the virtual unexpected object based on a halt zone corresponding to the robot or the unexpected object, and generates an instruction to halt the normal operations of the robot. The (dynamic) virtual robot representation may include a robot arm movable with respect to the robot's base. The monitored space is generated by removing ignore zones corresponding to expected objects, including an ignore zone corresponding to the robot itself, from an initial (e.g., spherical) version of the monitored space. Regions having insufficient sensor coverage (i.e., IPZs) are treated as unexpected objects within the monitored space.

The invention discloses a robot navigation method based on reinforcement learning security constraint and danger perception, and relates to the technical field of robots and artificial intelligence, and the robot navigation method comprises the following steps: when a robot is controlled to execute a navigation task in a target area, using a security filter SF to perform security constraint on the execution action of the robot; and when the robot executes the navigation task, the danger perception virtual robot HAVR is utilized to obtain experience in the dangerous area so as to enhance the ability of the robot to explore in the dangerous area. The action generated by reinforcement learning can be dynamically adjusted, so that the action meets the safety constraint, and the safety of the robot in the reinforcement learning training process is further enhanced; by introducing the HAVR with risk-driven rewards, the robot can still obtain experience in a dangerous area while introducing security constraints, and the exploration security of the robot is further enhanced while the performance of the model after convergence is ensured.

A system and method for training a machine learning model are disclosed, designed to determine the traverseability of a real-world environment by a mobile robot based on an image of that environment. The method advantageously generates high-quality synthetic training data for training a traverse detection model to distinguish between traversable and impassable areas in captured images of a real-world environment. The synthetic training data is generated by simulating a virtual robot in a virtual environment. Once the traverse detection model is trained, it can be applied to the mobile robot to predict the traverseability of a real-world environment.

A simulation data creation method, a preview image display method, and a simulation device that enable grasping of the contents of simulation before construction of a virtual space. The simulation data creation method includes the following steps: (a) causing a virtual robot disposed in a virtual work space to act based on an operation instruction by a user, and displaying a simulator image representing the action of the virtual robot in the virtual work space on a display device, the virtual work space being a virtual three-dimensional work space; (b) outputting a snapshot representing a two-dimensional image obtained by a virtual camera capturing a state in which a virtual three-dimensional object including the virtual robot is disposed in the virtual work space; and (c) saving, as simulation data, in a memory, an action program representing a target action of the virtual robot based on the operation instruction, layout data representing positions and postures of objects in the virtual work space, attribute data representing attributes of the objects, and the snapshot.

The application provides an interactive method and device of a virtual robot and a storage medium, and the method comprises the following steps: in response to a virtual robot interactive request triggered by a user, collecting multi-modal data of a live room of the user in real time; then, processing the collected data to convert the multi-modal data into text information in a unified format, wherein the text information is used to uniformly represent feature information of the multi-modal data; then, generating a response text according to the text information; determining interactive information based on the response text, and controlling the virtual robot to perform an interactive operation based on the interactive information. In the above scheme, the multi-modal data of the live room is analyzed and processed to fully mine the live room information, so that the virtual robot can automatically and intelligently interact with the user in combination with various live room information, which not only reduces the operation difficulty of the user, but also improves the live room activity, greatly improving the enthusiasm of the anchor and the experience of the audience.

The present invention discloses a coordinate conversion method, device, and computer-readable medium for a robot, belonging to the field of robot control technology. A specific implementation of the method includes: first, obtaining the self-coordinate system of a virtual robot in a simulation environment, and the coordinate values ​​of the end of the virtual robot relative to the self-coordinate system; then, reading the coordinate values ​​of the end of a physical robot in a real environment relative to the system coordinate system; and finally, determining the system coordinate system of the physical robot in a real environment based on the self-coordinate system of the virtual robot, the coordinate values ​​of the end of the virtual robot, and the coordinate values ​​of the end of the physical robot. Thus, by converting the self-coordinate system of the virtual robot into the system coordinate system, the system coordinates of the physical robot in the real environment can be accurately obtained, so that without knowing the coordinate system referenced by the physical robot's operation, the system coordinate system can be used as a reference to effectively control the operation of the physical robot.

A control device includes a processor that calculates one or more predetermined command values for one or more robots to undergo synchronous control in predetermined control cycles, an output unit that outputs the one or more predetermined command values in each of the predetermined control cycles, and a generator that generates an output signal for a virtual robot. The virtual robot is virtually defined in relation to the synchronous control. The processor calculates the one or more predetermined command values using the output signal for the virtual robot generated by the generator.

A teaching device constructs, in a virtual space, a virtual robot system in which a virtual 3D model of a robot and a virtual 3D model of a peripheral structure of the robot are arranged, and teaches a moving path of the robot. The teaching device includes an acquisition unit configured to acquire information about a geometric error between the virtual 3D models, and a correction unit configured to correct the moving path of the robot in accordance with the information acquired by the acquisition unit.

The application provides a robot process debugging system and method based on three-dimensional visualization, and relates to the technical field of robot processing debugging, and comprises the following steps: a data processing module is used for acquiring saved workstation data and robot data in a robot workstation, and extracting robot trajectory point data and process parameters from the robot data; a three-dimensional display module is used for establishing a virtual workstation model containing a virtual robot model according to the workstation data and the robot data, and establishing a virtual processing trajectory of the virtual robot model according to the trajectory point data and the process parameters; and a debugging module is used for debugging process parameters to obtain process debugging parameters according to externally input debugging instructions, and the three-dimensional display module synchronously displays the adjustment of the virtual processing trajectory according to the process debugging parameters. The beneficial effect is to provide a three-dimensional visual interactive interface, reduce the operation complexity, improve the process debugging efficiency, improve the interactive experience, and make the process debugging more convenient.

The invention discloses a digital twin system construction method based on Isaac Sim and UDP communication protocols, and belongs to the technical field of robot intelligent control. The system adopts a virtual-real collaborative framework to construct three core modules: a virtual robot model based on a URDF format is established through an Isaac Sim platform, and geometric structure parameters and kinematic characteristics of a real robot are accurately matched; a dynamic control strategy module is constructed by applying an RMPflow algorithm, an optimal motion path is generated through a synergistic effect of a target RMP and a collision avoidance RMP, and accurate target tracking and active obstacle avoidance of an end effector are realized; a bidirectional communication module supporting a UDP protocol is designed, a Socket communication link is constructed based on a local area network, a lightweight data packet is adopted to transmit a control signal and a robot state, and an ROS is used to realize control and state acquisition of a physical robot. The system comprises a virtual control real mode and a real control virtual mode, and supports feedback and synchronization of a virtual environment and a physical entity state. In conclusion, the invention provides the robot digital twin system construction method based on the Isaac Sim and the UDP communication protocol.

Disclosed is a robotic system (104) and a method (600) having a control unit (202). The control unit is configured to generate a virtual environment having a virtual robot (402) based on a received set of first parameters to emulate a real environment. The control unit (202) is configured to actuate the virtual robot (402) to simulate an operation of a robot (108). At least the virtual robot (402) actuates independently, or the robot (108) and the virtual robot (402) actuate in synchrony based on the simulation. Further, at least the virtual robot (402) actuates independently, depending on the set of second parameters or the robot (108) and the virtual robot (402) actuates, depending on the set of second parameters. This configuration ensures multi modal operation of the robotic system (104), thus ensuring ease of operation of the robot (108).

This disclosure provides a robot intelligent visualization method, device, storage medium, and program product. The method includes: acquiring robot pose data, joint data, and point cloud data of a real-world scene; using a coordinate transformation method to convert the position data in the pose and joint data from the robot's body coordinate system to a three-dimensional scene coordinate system; based on the coordinate-transformed pose and joint data, performing real-time posture and skeletal synchronization of the robot to generate a virtual robot model that accurately reproduces the robot's real movements; constructing a three-dimensional scene based on the point cloud data and embedding the virtual robot model into the three-dimensional scene; and triggering a corresponding task mode when the virtual robot model is within the task point sensing range of the three-dimensional scene. This method achieves precise spatiotemporal synchronization and fusion between the physical robot and the virtual scene through multi-source data fusion and coordinate system unification, realistically reproducing robot movements and intelligently triggering tasks.

The invention discloses a crankshaft defect coping method and system based on digital twinning. The method is characterized by comprising the following steps: constructing a digital twin virtual environment consistent with a real environment, wherein virtual camera parameters are consistent with a real multi-view vision system; a crankshaft defect image is collected through a real multi-view vision system; the defect pixel positions are mapped to the surface of the virtual crankshaft to form virtual defects; in the virtual environment, the virtual robot plans a grinding path and generates an instruction; and the instruction is sent to the real robot to execute grinding. The system comprises a real physical system, a digital twin virtual system and a control communication system. According to the method, path planning and verification are completed in the digital twin virtual environment, and then the instruction is issued to the real system for execution, so that the precision, the safety and the automation degree of crankshaft defect grinding are effectively improved, and the collision risk and the debugging cost are reduced.

This control device executes: a physical simulation that simulates a reaction force from a virtual object when a force is applied to the virtual object; and a robot simulation that simulates a virtual robot operation for applying force to the virtual object. On the basis of data obtained during the execution of the physical simulation and the robot simulation, the control device generates a trained model for outputting behavior data of a real robot corresponding to the virtual robot, upon receiving input of state data representing the environment in which the real robot operates.

Disclosed is a robotic system (104) and a method (600) having a control unit (202). The control unit is configured to generate a virtual environment having a virtual robot (402) based on a received set of first parameters to emulate a real environment. The control unit (202) is configured to actuate the virtual robot (402) to simulate an operation of a robot (108). At least the virtual robot (402) actuates independently, or the robot (108) and the virtual robot (402) actuate in synchrony based on the simulation. Further, at least the virtual robot (402) actuates independently, depending on the set of second parameters or the robot (108) and the virtual robot (402) actuates, depending on the set of second parameters. This configuration ensures multi modal operation of the robotic system (104), thus ensuring ease of operation of the robot (108).

Provided is a simulation system that simulates an operation of a cell including a robot and a device, wherein the simulation system comprises a virtual device that simulates an output of the device and a a virtual robot controller that simulates, according to an output of the virtual device, an operation of a robot controller that controls the robot, and the virtual device performs an output based on a plurality of output histories of the device.

A teaching device for teaching a robot includes: a storage unit that stores mechanism data for finding a relationship between a tip position of a robot and an angle position at a joint portion of the robot, the mechanism data including a mechanism error parameter of an actual robot; and a virtual robot control unit that controls the operation of a virtual robot on the basis of the mechanism data such that an error in position that occurs in the actual robot due to the mechanism error parameter occurs in the virtual robot.

Embodiments of the present application relate to the field of virtual simulation technologies, and provide a robot offset simulation method and apparatus, a robot, an electronic device, and a storage medium. The robot offset simulation method includes: establishing a reference path for a virtual robot in simulation software, where the reference path is a path along which the virtual robot moves to transfer a reference object from a preset starting point to a destination location and then returns to the preset starting point; receiving location information of a target object that is sent by a PLC; determining an offset of the virtual robot based on the location information, where the offset is a location offset of the target object relative to the reference object; and controlling, based on the reference path and the offset, the virtual robot to move to transfer the target object.

A simulation device 10 comprises a computation unit 11 that executes a simulation in which a virtual operator for reproducing an operation performed by a worker, and a virtual robot device for reproducing an operation performed by a robot device operating in cooperation with the worker, are caused to operate in a virtual space. The computation unit: acquires basic operation information relating to a basic operation performed by the worker; generates irregular operation information including an irregular operation different from the basic operation by using the generative AI on the basis of the basic operation information; and executes a first simulation in which the virtual worker and the virtual robot device are caused to operate in cooperation in the virtual space on the basis of the basic operation information, the irregular operation information, and a robot-device program for operating the robot device.

The invention discloses a robot teaching method based on a sphere bounding box and related equipment. The robot teaching method comprises the following steps: acquiring a chemical experiment virtual scene corresponding to a real operation environment; collecting motion data of teaching input equipment, and driving a virtual robot in the chemical experiment virtual scene to perform chemical action drilling based on the motion data; in the moving process of the virtual robot, the space occupation information is updated in real time; obtaining a collision detection result between the sphere bounding boxes according to the updated space occupation information; and generating a control instruction sent to a real robot according to the collision detection result. According to the method, traditional full-grid collision detection is abandoned, collision detection is carried out in a simplified sphere bounding box model mode, low-delay real-time anti-collision early warning is achieved, and the stability and safety of the chemical experiment operation process are guaranteed.

Disclosed is a product production system comprising: a planning system; an enterprise resource planning (ERP) system; a digital twin system; and a feedback system provided within the digital twin system, wherein the digital twin system generates a processing model on the basis of a correction value generated by the feedback system and performs a simulation for product production in the future to process virtual data related to a virtual robot operation result through the processing model.

Systems and methods for human-robot communication. More particularly, some embodiments use augmented reality to facilitate communication of robot intention and teleoperation in human-robot cooperative environments. Various embodiments of the present technology provide a middleware that integrates augmented reality (AR) with novel teleoperation interfaces to increase operation effectiveness, support the user in conducting concurrent work, and decrease stress. Various embodiments provide predictive graphical interfaces such that a teleoperator controls a virtual robot surrogate, rather than directly operating the robot itself, providing the user with foresight regarding where the physical robot will end up and how it will get there. In accordance with various embodiments a user may select between two AR interfaces using such a surrogate: one focused on real-time control and one inspired by waypoint delegation.

Example implementations may include capturing virtual hot execution, for example by taking a screen recording of a desktop screen of a virtual machine where the execution is taking place. The contents, such as video such as the recording and / or associated data, may be processed, buffered, stored, and / or transmitted before being streamed (e.g., to a user). Thus, these implementations may involve initiating a virtual bot to execute an operation; generating a graphical representation of the virtual bot executing the operation; and transmitting, to a client device, the graphical representation of the operation for display at the client device.

The invention discloses a pipeline robot remote control method and system, and relates to the technical field of robot control. The method comprises the following steps: acquiring a panoramic image, motion parameters and pose information of the pipeline robot in a pipeline; according to the panoramic image, the motion parameters and the pose information, constructing a digital twinborn scene comprising a virtual robot and a virtual pipeline; the minimum distance between each part of the virtual robot and the pipe wall of the virtual pipeline and / or the obstacle is monitored, and distance information is obtained; monitoring the driving state of each virtual driving wheel of the virtual robot; according to the distance information and the driving state, generating an optimal rotation direction and speed parameter of each virtual driving wheel, and displaying the optimal rotation direction and speed parameter in an interactive interface; receiving a control instruction input by an operator based on the interactive interface, and transmitting the control instruction to the pipeline robot for execution; according to the invention, accurate obstacle avoidance and auxiliary decision making of an operator in a complex pipeline scene are realized, and the physical limitation of traditional wired control and the decision-making bottleneck of single-view monitoring are broken through.