An AR-assisted robot remote operation system based on digital twin technology

Abstract

The present invention discloses an AR-assisted robot remote operation system based on digital twin technology, which includes an AR virtual module, an entity module, and a server terminal; the AR virtual module is used to run the AR-assisted robot remote virtual model to assist in the remote operation of the robot; the entity module includes a robot side and a client side, and is used for information interaction between the robot side and the client side; the server terminal is used to send operation instructions to the robot side and the client side, and at the same time obtain the real-time data of the robot side and the client side; input the real-time data into the AR-assisted robot remote virtual model for calculation, receive the simulation results obtained by the calculation of the AR-assisted robot remote virtual model, and realize the interaction with the industrial robot through a visual interface, flexibly cooperate with other manufacturing robots, make up for the shortcoming that traditional robots can only operate according to the programmed procedures, and the operation method is relatively friendly, which can greatly reduce the workload of operators.

Description

Technical Field

[0001] The present invention relates to the technical field of digital robots, and particularly to an AR-assisted robot remote operation system based on digital twin technology. Background Art

[0002] In today's increasingly competitive market, the manufacturing mode is shifting towards large-scale personalization and marketization, which correspondingly leads to high requirements for the flexibility and automation of manufacturing systems. To achieve large-scale personalization in manufacturing, various industrial robots and operators are incorporated into the manufacturing process. However, most existing robot systems perform pre-programmed tasks in a conventional manner, lacking much intelligence, let alone handling personalized tasks well in a collaborative manner. At this time, a collaborative manufacturing system between fully automated manufacturing and fully manual manufacturing is needed, which can achieve the purpose of collaborating with automated robots or other operators in the digital space.

[0003] The existing patent application No. 202110456817.X discloses a digital twin training method and system for industrial robots, which realizes the equivalent simulation of a physical model by constructing a corresponding digital twin model, obtains the real-time data of a training bench and an industrial robot, inputs the real-time data into a training virtual model for operation; receives the simulation results obtained by the operation of the training virtual model, and finally can complete the coordination between the digital training platform and the physical industrial robot in the virtual space.

[0004] The deficiencies of the existing technology lie in the lack of human intervention by operators, the failure to achieve the interaction effect between humans and robots, and the poor interactivity and flexibility. Summary of the Invention

[0005] The purpose of the present invention is to provide an AR-assisted robot remote operation system based on digital twin technology, which realizes the interaction with industrial robots (including remote control and collaborative cooperation) through a visual interface, flexibly collaborates with other manufacturing robots, makes up for the shortcoming that traditional robots can only operate according to pre-edited programs, and has a friendly operation method, which can greatly reduce the workload of operators.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] An AR-assisted robot remote operation system based on digital twin technology includes an AR virtual module, a physical module, and a server terminal;

[0008] The AR virtual module is used to run an AR-assisted robot remote virtual model to assist in remote operation of the robot;

[0009] The physical module includes a robot side and a client side, and is used for information interaction between the robot side and the client side;

[0010] A server terminal, which is used to send operation instructions to the robot side and the client side, and at the same time obtain the real-time data of the robot side and the client side;

[0011] Input the real-time data into the AR-assisted robot remote virtual model for calculation, and receive the simulation results obtained from the operation of the AR-assisted robot remote virtual model.

[0012] As a further solution of the present invention: The AR-assisted robot remote virtual model is implemented through the following steps:

[0013] Build a multi-robot multi-client communication mechanism to ensure information interaction between AR devices and robots, and between robots and robots;

[0014] Build a physical model of the robot through a game engine and transplant it to the AR glasses. Control the corresponding digital twin robot through the virtual scene on the AR device. The corresponding command information input is obtained through the reinforcement learning algorithm to obtain the corresponding output result and then returned to the physical robot.

[0015] As a further solution of the present invention: The game engine adopts the Unity game engine.

[0016] As a further solution of the present invention: The robots not controlled in the virtual scene are presented in the AR device in their original postures.

[0017] As a further solution of the present invention: The AR-assisted robot remote virtual model uses the reinforcement learning algorithm to calculate the input on the digital twin robot. By associating the state of the task environment with its own motion parameters, the initial robot control strategy is implemented;

[0018] Then, through the returned rewards and continuous trial-and-error interactions with the environment, the corresponding control strategy is improved through the reinforcement learning algorithm to realize the bidirectional mapping of the postures of the physical robot and the digital twin robot

[0019] The beneficial effects of the present invention: The present invention uses AR devices and digital twin technology to realize the function of remotely operating robots. In the virtual space, it can realize the function of remotely controlling robots by humans and collaborating with robots, can make up for the deficiencies in the functions of traditional robots in the intelligent manufacturing process, improve the manufacturing efficiency, and can flexibly adapt to different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The following further describes the present invention with reference to the accompanying drawings.

[0021] Figure 1 It is the flowchart of the present invention;

[0022] Figure 2It is a schematic diagram of the AR-assisted robot remote operation system of the present invention. Detailed implementation manners

[0023] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0024] Please refer to Figure 1 - Figure 2 As shown, the present invention is an AR-assisted robot remote operation system based on digital twin technology;

[0025] It includes an AR virtual module, an entity module, and a server terminal;

[0026] The AR virtual module is used to run the AR-assisted robot remote virtual model to assist in the remote operation of the robot;

[0027] The entity module includes a robot side and a client side, and is used for the interaction between the robot side and the client side;

[0028] The server terminal is used to send operation instructions to the robot side and the client side, and at the same time obtain the real-time data of the robot side and the client side;

[0029] Input the real-time data into the AR-assisted robot remote virtual model for calculation, and receive the simulation results obtained from the calculation of the AR-assisted robot remote virtual model.

[0030] The server terminal serves as a transfer station, including the edge node of the robot side. The edge node of the robot side is used to receive the action and status information of the robot side and send the command information from the client side;

[0031] It further includes an operation node of the robot side, which is used to receive the robot status information from the server terminal and send client commands;

[0032] Among them, the server terminal adopts the sockets communication protocol, allowing the client to control multiple robots simultaneously, and the status of all robots is synchronized with the server terminal (i.e., the main node);

[0033] Main node: Continuously receive the status updates of each robot in the manufacturing system. At the same time, the status of each robot will be distributed to each slave node connected to the same network;

[0034] Edge node: The node connected to the robot client needs to upload the status of the corresponding robot and receive the latest status of other robots sent by the main node.

[0035] The AR-assisted robot remote virtual model is realized through the following steps;

[0036] S1: Physically model the DT of the physical robot through the Unity game engine;

[0037] S2: The DT model of the robot is transplanted onto the AR glasses;

[0038] Among them, in the AR glasses, the DT of the robot synchronized with the physical robot is projected as a hologram in the remote workspace. Through the mapped DT of the physical robot, the movement of the physical robot can be remotely controlled, and at the same time, the state of the physical robot can be monitored and visualized through the robot DT;

[0039] S3: Use the reinforcement learning algorithm to calculate the input on the virtual robot (digital twin robot). By associating the state of the task environment with its own motion parameters, initialize the robot control strategy;

[0040] S4: Through the returned rewards and continuous trial-and-error interactions with the environment, improve the corresponding control strategy through the reinforcement learning algorithm, and realize the two-way mapping of the postures of the physical robot and its digital twin model to achieve the expected performance / target (reach a specific position in the task).

[0041] Specifically, the AR-assisted robot remote virtual model realizes the virtual-physical posture registration of the robot, including two stages, display model alignment and joint alignment:

[0042] In the model alignment stage, the designed virtual 3D robot model and the Vuforia engine are used to align the model targets between the physical and virtual robots to synchronize the displayed robot poses;

[0043] In the joint alignment stage, the joint value alignment is based on the pose alignment model to calculate the joint value transformation matrix, which can transform the joint values of the DT in the AR coordinate system into the joint values of the physical robot in the real-world coordinate system;

[0044] For each iteration of the physical robot pose update loop, the expected position of the physical robot end effector is set by the human in the AR glasses, the feasible joint value solution is calculated by the RL-based motion planning algorithm, and then the joint values of the physical robot are converted into the joint value solution of the virtual robot; by associating the state of the task environment with its own motion parameters, initialize the robot control strategy; then, through the returned rewards and continuous trial-and-error interactions with the environment, improve the corresponding control strategy through this algorithm to achieve the expected performance / target;

[0045] Through this transformation, the joint values of the virtual robot in the AR glasses scenario are mapped to the corresponding joint values of the physical robot, enabling the physical robot to operate synchronously with the DT. Finally, due to the slight differences between the virtual and real robot models and detection accuracies, the joint values of the physical robot will be sent back to the virtual robot after the movement is completed to modify the pose of the virtual robot. Through the above steps, pose synchronization can form a closed-loop process to maintain system stability and avoid errors.

[0046] Among them, the working process of the reinforcement learning algorithm in S3 is as follows:

[0047] In the robot motion planning stage, the planning process is regarded as a sequential decision-making problem and solved as an episodic task through a model-free RL method. In each stage, the interaction trajectory is represented as a Markov decision process (MDP), and the MDP is composed of the following set elements (S, A, P sa , R, γ);

[0048] Among them, the state space S is the set of states of the entire planning process, and the state representation s in this state set consists of four parts: d1, the robot; d2: the target position set based on the robot coordinate system; d3: the position of the end effector based on the robot coordinate system; d4: the distance vector between the target position and the end effector position;

[0049] The robot coordinate system takes the center of the base joint of the robot as the coordinate origin. The positive direction of the y-axis is the direction in which the wire extends, the positive direction of the z-axis is the direction upward of the base node, and the direction of the x-axis is determined by the right-hand screw rule;

[0050] Among them, the action space A is the number of joints that the robot can rotate, a is the selected operation, and the reward R is the total sum of the expected rewards:

[0051]

[0052] Among them, the element r(s, a) is determined by the state representation s where the robot is located and the operation a performed by the robot, and t represents the current time;

[0053] The reward function r includes the following parts: c1: the absolute value of the distance scalar between the target position and the end effector position; c2: the end effector position to the target position (reward period); c3: the joint of the end effector leaving the working area (penalty clause); c4: the sum of the heights of each joint (optional);

[0054] γ ∈ [0, 1] is the discount factor, and P sa is the transition probability distribution of performing the operation a in the state representation s.

[0055]

[0056] With the above settings, the task set starts from the initial state s0, and then the robot samples an operation a ∈ A in each decision interval π(a t |s t ) to adjust the node position of the robot;

[0057] Then, according to the transition probability distribution P sa generates the next state set s of the next decision t+1 , and then the robot obtains the corresponding reward value r(s, a) from the environment;

[0058] The essence of the reinforcement learning algorithm is to optimize the policy π based on performing exploratory actions and reinforcement actions to bring better performance than the robot's expectation, and the robot's expectation is modeled by the state value function V:

[0059]

[0060] where H is the time horizon of the turn-based task.

[0061] For the control of the robot, the classic PPO RL algorithm is adopted. The PPO-based agent performs exploration in the environment and compares the actually obtained reward of each state-action pair with the estimated reward to form the advantage function

[0062] A(s t ,a t ) = R t -V π (s t )

[0063] Meanwhile, the PPO algorithm uses importance sampling to optimize the sampling efficiency, and ρ t is the probability ratio between the updated policy generated by importance sampling and the original policy;

[0064] Meanwhile, under the constraint of the loss function, an intuitive and effective clip function is proposed, where ε is the range of the clip term, and the likelihood function L CLIP (θ) is expressed as follows:

[0065]

[0066] With the help of the clip item, if the policy update offset between the old and new policies exceeds a predefined interval, the clip item will clip the agent's target, thereby restricting the update of the policy function within a certain interval to prevent the policy update from converging too fast or too slow, thus improving the training speed and feasibility of the algorithm. At the same time, the exploration method of the stochastic policy is retained. When the sampled samples meet the maximum likelihood probability, the robot motion planning method will have better exploration and robustness.

[0067] That is, by controlling the corresponding digital twin robot in the virtual scene on the AR device, the input corresponding command information obtains the corresponding output result through the reinforcement learning algorithm and then returns it to the physical robot, while the robots not being controlled will be presented in the AR device with their current postures.

[0068] The core point of the present invention lies in: using the AR device and digital twin technology to realize the function of remotely operating a robot, being able to realize the function of a person remotely controlling a robot and collaborating with the robot on the virtual space, being able to make up for the deficiencies in the functions of traditional robots in the intelligent manufacturing process, improving the manufacturing efficiency, and being able to flexibly adapt to different scenarios.

[0069] The above has described in detail an embodiment of the present invention, but the content described is only the preferred embodiment of the present invention and cannot be considered as used to limit the scope of implementation of the present invention. All equivalent changes and improvements made according to the scope of the application of the present invention should still fall within the scope covered by the patent of the present invention.

Claims

1. An AR-assisted robot remote operation system based on digital twin technology, characterized in that, It includes an AR virtual module, a physical module, and a server terminal; The AR virtual module is used to run the AR-assisted robot remote virtual model to assist in the remote operation of the robot; The physical module includes a robot end and a client end, and is used for information interaction between the robot end and the client end; The server terminal is used to send operation instructions to the robot end and the client end, and at the same time obtain the real-time data of the robot end and the client end; Input the real-time data into the AR-assisted robot remote virtual model for calculation, and receive the simulation results obtained by the operation of the AR-assisted robot remote virtual model; The AR-assisted robot remote virtual model is implemented through the following steps: Construct a multi-robot multi-client communication mechanism to ensure information interaction between the AR device and the robot, and between the robot and the robot; Use the game engine to perform physical modeling on the DT of the physical robot and transplant the DT model of the robot to the AR glasses. Control the corresponding digital twin robot through the virtual scene on the AR device. The input corresponding command information obtains the corresponding output result through the reinforcement learning algorithm and then returns it to the physical robot; The reinforcement learning algorithm is adopted in the AR-assisted robot remote virtual model to calculate the input on the digital twin robot. By associating the state of the task environment with its own motion parameters, the robot control strategy is initialized; Then, through the return of rewards and continuous trial-and-error interaction with the environment, the corresponding control strategy is improved through the reinforcement learning algorithm to achieve the two-way mapping of the postures of the physical robot and the digital twin robot; Among them, the AR-assisted robot remote virtual model realizes the virtual-physical posture registration of the robot, including two stages: display model alignment and joint alignment: In the model alignment stage, use the designed virtual 3D robot model and the Vuforia engine to align the model target between the physical and virtual robots to synchronize the displayed robot poses; In the joint alignment stage, the joint value alignment is to calculate the joint value transformation matrix based on the pose alignment model. This matrix converts the joint values of the DT in the AR coordinate system into the joint values of the physical robot in the real-world coordinate system; For each iteration of the physical robot pose update loop, the expected position of the end effector of the physical robot is set by the human in the AR glasses. The feasible joint value solution is calculated by the RL-based motion planning algorithm, and then the joint values of the physical robot are converted into the joint value solution of the virtual robot; by associating the state of the task environment with its own motion parameters, the robot control strategy is initialized; then, through the return of rewards and continuous trial-and-error interaction with the environment, the corresponding control strategy is improved through this algorithm to achieve the expected performance / goal; Through this conversion, the joint values of the virtual robot in the AR glasses scene are mapped to the corresponding joint values of the physical robot, enabling the physical robot to operate synchronously with the DT. Finally, due to the slight differences between the virtual and real robot models and the detection accuracy, the joint values of the physical robot are sent back to the virtual robot after the movement is completed to modify the pose of the virtual robot.

2. The AR-assisted robot remote operation system based on digital twin technology according to claim 1, wherein The game engine adopts the Unity game engine.

3. The AR-assisted robot remote operation system based on digital twin technology according to claim 1, characterized in that Robots that are not controlled in the virtual scene are presented in the AR device in their original postures.

Images