A remote operation control method and system based on virtual reality to make up time delay

Abstract

The invention discloses a remote operation control method and system for making up time delay based on virtual reality. The method includes: establishing an initial virtual robot model, a virtual camera and a background image according to the RCM mechanism located at the slave end and the actual camera located at the slave end; acquiring position information of the master hand; mapping the position information of the master hand to the joint motion in the initial virtual robot model , to obtain the virtual robot model updated in real time; obtain the feature corners of the slave scene image; calculate the virtual corners of the virtual robot model; correct the virtual simulation model by using the feature corners and virtual corners of the slave scene image to obtain the corrected A virtual simulation model; an updated three-dimensional virtual scene model is obtained from the corrected virtual simulation model, a virtual camera and a background image, so as to realize teleoperation. The invention can realize the construction of a three-dimensional virtual model, and enhance the realism of teleoperation while improving the control precision.

Description

technical field

[0001] The invention relates to the field of teleoperation robot control, in particular to a teleoperation control method and system based on virtual reality to compensate time delay. Background technique

[0002] The teleoperation system is operated by an operator to operate a local robot to control the remote robot through the network to complete the specified work tasks. Teleoperation technology has been widely used in space technology, deep sea exploration, telemedicine and extreme environments. In the application of the teleoperation system, if the actions of the remote robot can be fed back to the main robot in real time, the operator's sense of presence will be greatly improved. Telepresence technology is the core of human-computer interaction teleoperation.

[0003] The virtual environment prediction technology is to create a virtual simulation environment on the computer, mainly including the virtual slave robot and the virtual environment, and the...

Examples

Embodiment 1

[0072] figure 1 It is a flow chart of a teleoperation control method based on virtual reality to make up time delay according to Embodiment 1 of the present invention. see figure 1 , the remote operation control method based on virtual reality to make up for time delay in this embodiment includes:

[0073] Step S1: Establish an initial virtual robot model, a virtual camera and a background image according to the RCM mechanism at the slave end and the actual camera at the slave end; the initial virtual simulation model is established at the master end and used to simulate the RCM mechanism A three-dimensional virtual model; the virtual camera is a virtual camera established on the master end and used to simulate the actual camera; the master end communicates with the slave end.

[0074] In this embodiment, the actual camera is a global camera.

[0075] The step S1 specifically includes:

[0076] 11) Create a virtual RCM mechanism model using the OpenInventor graphics librar...

Embodiment 2

[0104] The teleoperation control method based on virtual reality to make up time delay in this embodiment is mainly divided into the construction of virtual environment and the virtual and real fusion of graphics and images. Based on the kinematics analysis of the slave RCM robot, the virtual simulation model of the master is established by using the OpenInventor graphics library. . Camera parameter calibration and corresponding image feature point recognition are performed on the actual global camera at the slave end. The virtual-real fusion technology is based on the detected image feature points, overlapping the virtual robot model with its actual image, and the rest of the image is used as the background of the model to simulate the surrounding environment, and continuously corrects the distance between the model and the image plane during the movement process. Relative deviation. The specific ideas are as follows:

[0105] First, build a virtual environment, which is a ...

Embodiment 3

[0143] This embodiment provides a teleoperation control system based on virtual reality to make up time delay, including:

[0144] Model construction module, for establishing initial virtual robot model, virtual camera and background image according to the RCM mechanism positioned at slave end and the actual camera positioned at said slave end; Said initial virtual simulation model is set up at master end and is used for simulating said A three-dimensional virtual model of the RCM mechanism; the virtual camera is a virtual camera established on the master end and used to simulate the actual camera; the master end communicates with the slave end.

[0145] The position information acquiring module is used to acquire the position information of the main hand; the position information of the main hand is the position information of the main operator located at the main end.

[0146] A model control module, configured to map the main hand position information to joint motions in th...

technical field

[0001] The invention relates to the field of teleoperation robot control, in particular to a teleoperation control method and system based on virtual reality to compensate time delay. Background technique

[0002] The teleoperation system is operated by an operator to operate a local robot to control the remote robot through the network to complete the specified work tasks. Teleoperation technology has been widely used in space technology, deep sea exploration, telemedicine and extreme environments. In the application of the teleoperation system, if the actions of the remote robot can be fed back to the main robot in real time, the operator's sense of presence will be greatly improved. Telepresence technology is the core of human-computer interaction teleoperation.

[0003] The virtual environment prediction technology is to create a virtual simulation environment on the computer, mainly including the virtual slave robot and the virtual environment, and the...

Examples

Embodiment 1

[0072] figure 1 It is a flow chart of a teleoperation control method based on virtual reality to make up time delay according to Embodiment 1 of the present invention. see figure 1 , the remote operation control method based on virtual reality to make up for time delay in this embodiment includes:

[0073] Step S1: Establish an initial virtual robot model, a virtual camera and a background image according to the RCM mechanism at the slave end and the actual camera at the slave end; the initial virtual simulation model is established at the master end and used to simulate the RCM mechanism A three-dimensional virtual model; the virtual camera is a virtual camera established on the master end and used to simulate the actual camera; the master end communicates with the slave end.

[0074] In this embodiment, the actual camera is a global camera.

[0075] The step S1 specifically includes:

[0076] 11) Create a virtual RCM mechanism model using the OpenInventor graphics librar...

Embodiment 2

[0104] The teleoperation control method based on virtual reality to make up time delay in this embodiment is mainly divided into the construction of virtual environment and the virtual and real fusion of graphics and images. Based on the kinematics analysis of the slave RCM robot, the virtual simulation model of the master is established by using the OpenInventor graphics library. . Camera parameter calibration and corresponding image feature point recognition are performed on the actual global camera at the slave end. The virtual-real fusion technology is based on the detected image feature points, overlapping the virtual robot model with its actual image, and the rest of the image is used as the background of the model to simulate the surrounding environment, and continuously corrects the distance between the model and the image plane during the movement process. Relative deviation. The specific ideas are as follows:

[0105] First, build a virtual environment, which is a ...

Embodiment 3

[0143] This embodiment provides a teleoperation control system based on virtual reality to make up time delay, including:

[0144] Model construction module, for establishing initial virtual robot model, virtual camera and background image according to the RCM mechanism positioned at slave end and the actual camera positioned at said slave end; Said initial virtual simulation model is set up at master end and is used for simulating said A three-dimensional virtual model of the RCM mechanism; the virtual camera is a virtual camera established on the master end and used to simulate the actual camera; the master end communicates with the slave end.

[0145] The position information acquiring module is used to acquire the position information of the main hand; the position information of the main hand is the position information of the main operator located at the main end.

[0146] A model control module, configured to map the main hand position information to joint motions in th...

Images