A Modeling and Control Method Combining Data-Driven and Mechanism Hybrid for Virtual-Reality Fusion Manipulation

Abstract

The present invention discloses a data-driven and mechanism hybrid modeling and control method for virtual-real fusion manipulation, which is oriented to the robot teleoperation process based on virtual simulation. The prediction model generation calculates the accurate prediction state vector of the robot by using a data-driven method according to the collected historical motion data of the robot. Through the digital human-machine interaction process, stable prediction control commands are formed, and then the on-site model is driven to carry out actual task operations. At the same time, the on-site model feeds back the on-site information of the robot to the virtual model for correction and ensures bilateral consistency. The advantage of the present invention is that compared with the traditional bilateral teleoperation process, the operator is not affected by the torque directly feedback by the on-site model, understands the on-site situation through the virtual simulation scenario, and has the characteristics of fast response. It can also control the interaction frequency and reduce the power consumption of the system by designing the update efficiency of the prediction and virtual models.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital collaborative interaction between humans and robots, and particularly relates to a data-driven and mechanism hybrid modeling and control method for virtual-real fusion manipulation. Background Art

[0002] Robot teleoperation is a technology in which the behavior of a local operator determines the movement of a remote robot. During the process of using this technology to perform tasks, the most significant difficulty to overcome is the problem of state uncertainty under the time delay caused by the excessive distance between the local and the remote. Reconstructing the movement of the remote robot using virtual simulation technology to provide behavior reference for local operators is an effective measure to solve the aforementioned problem.

[0003] The robot teleoperation system based on virtual simulation can change the "move - wait - move" of the conventional teleoperation system. There is a situation where the virtual model and the on-site model in space can provide reference behaviors based on the human-machine interaction model, but state synchronization cannot be achieved. In addition, directly predicting data such as the posture and trajectory of a robot through data collected by various sensors of the robot has the technical problem of low prediction state accuracy.

[0004] Therefore, to solve the technical problems existing in the above-mentioned prior art, a new type of robot teleoperation and control method for virtual simulation is to be proposed. Summary of the Invention

[0005] The purpose of the present invention is to provide a data-driven and mechanism hybrid modeling and control method for virtual-real fusion manipulation to solve the problems of inaccurate prediction of robot states and inability to achieve real-time human-machine interaction control in the prior art.

[0006] The present invention adopts the following technical solutions:

[0007] Embodiment 1 of the present invention provides a data-driven and mechanism hybrid modeling and control method for virtual-real fusion manipulation. The method applied to the prediction model includes:

[0008] Taking the joint space position vector and joint space velocity vector of the robot on-site as the input of the prediction model, and generating the prediction state estimation value at the current moment and the prediction state estimation value at the next moment through the rolling horizon state prediction algorithm;

[0009] Generating a predictive control law based on the prediction state estimation value at the current moment, the prediction state estimation value at the next moment, the virtual expected trajectory at the current moment generated by the virtual model, and the virtual expected trajectory at the next moment. The predictive control law is used to control the on-site work of the robot.

[0010] Optionally, the on-site joint space position vector, joint space velocity vector, and the desired trajectory generated by the virtual model are used as the inputs of the prediction model, and the prediction state vector generated by the rolling horizon state prediction algorithm includes:

[0011] Step 1: Process the digital model of the robot operation site using the Euler discretization method, and iteratively calculate the difference of the predicted trajectory at the current moment and the difference of the predicted trajectory at the next moment;

[0012] Step 2: Calculate the predicted state vector at the current moment based on the difference of the predicted trajectory at the current moment and the difference of the predicted trajectory at the next moment;

[0013] Step 3: Obtain the historical data of the predicted state vector through a sliding window, and the historical data of the predicted state vector is calculated based on the joint space position vector and the joint space velocity vector;

[0014] Step 4: Take the minimum loss between the historical data of the predicted state vector and the predicted state vector at the current moment as an optimization problem, and solve for the predicted state estimate value at the current moment;

[0015] Step 5: Perform a nonlinear calculation on the state estimate value at the current moment to obtain the predicted state estimate value at the next moment.

[0016] Optionally, a predictive control law is generated based on the predicted state estimate value at the current moment, the predicted state estimate value at the next moment, the virtual desired trajectory at the current moment generated by the virtual model, and the virtual desired trajectory at the next moment, including:

[0017] The specific calculation method of the predictive control law is:

[0018]

[0019] Where, it is defined that: t k represents the current moment, t k+1 represents the next moment, is the estimation matrix of the prediction model obtained based on the state estimate value of the rolling horizon optimization, is the non-linear function estimation term of the prediction model obtained based on the state estimate value of the rolling horizon optimization, is the predicted state estimate value at the next moment, is the predicted state estimate value at the current moment, q d (t k+1 ) is the virtual desired trajectory at the next moment generated by the virtual model, q d (t k ) is the virtual desired trajectory at the current moment generated by the virtual model, s p (t k ) is for t kThe sliding mode surface at the moment, σ is the time interval between two adjacent samplings, and η p is the approaching gain in the prediction model, and C sp is the linear gain of the sliding mode surface in the prediction model, and τ p (t k ) is the predictive control law at the current moment.

[0020] Optionally, the virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment generated by the virtual model are transmitted to the prediction model in a wired or wireless manner.

[0021] Embodiment II of the present invention provides a data-driven and mechanism hybrid modeling and control method for virtual-real fusion manipulation. The method applied to the virtual model includes:

[0022] Using the Euler discretization method to process the digital model of the robot operation site, and iteratively calculating the virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment;

[0023] Sending the virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment;

[0024] The virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment are used to generate the predictive control law.

[0025] Embodiment III of the present invention provides a data-driven and mechanism hybrid modeling and control method for virtual-real fusion manipulation. The method applied to the on-site model includes:

[0026] Sending the joint space position vector and the joint space velocity vector of the on-site of the robot generated during movement in continuous time;

[0027] Receiving the predictive control law; the generation method of the predictive control law is:

[0028] Taking the joint space position vector and the joint space velocity vector of the on-site of the robot as the input of the prediction model, and generating the predicted state estimate value at the current moment and the predicted state estimate value at the next moment through the rolling horizon state prediction algorithm;

[0029] Generating the predictive control law according to the predicted state estimate value at the current moment, the predicted state estimate value at the next moment, the virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment;

[0030] Controlling the on-site movement of the robot according to the predictive control law.

[0031] Embodiment IV of the present invention provides a data-driven and mechanism hybrid modeling and control method for virtual-real fusion manipulation, including the following steps:

[0032] The joint space position vector and joint space velocity vector of the on-site generated by the on-site model sending robot during continuous movement

[0033] The prediction model receives the joint space position vector and joint space velocity vector of the on-site of the robot sent by the on-site model, inputs them into the prediction model, and generates the prediction state estimation value at the current moment and the prediction state estimation value at the next moment through the rolling horizon state prediction algorithm

[0034] The virtual model iteratively calculates the virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment by using the Euler discretization method

[0035] Send the virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment

[0036] The virtual model generates a predictive control law based on the prediction state estimation value at the current moment, the prediction state estimation value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment, and the predictive control law is used to control the on-site work of the robot

[0037] The virtual model receives the prediction state estimation value at the current moment and the prediction state estimation value at the next moment generated by the prediction model

[0038] Generate the virtual control law at the current moment according to the virtual desired trajectory at the current moment, the virtual desired trajectory at the next moment, the prediction state estimation value at the current moment, and the prediction state estimation value at the next moment

[0039] Embodiment 5 of the present invention provides a data-driven and mechanism hybrid modeling and control system for virtual-real fusion control, including

[0040] An on-site model, which is used to send the joint space position vector and joint space velocity vector of the on-site generated by the robot during continuous movement; receive the predictive control law; the method for generating the predictive control law is

[0041] Take the joint space position vector and joint space velocity vector of the on-site of the robot as the input of the prediction model, and generate the prediction state estimation value at the current moment and the prediction state estimation value at the next moment through the rolling horizon state prediction algorithm

[0042] Generate a predictive control law according to the prediction state estimation value at the current moment, the prediction state estimation value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment

[0043] Control the on-site movement of the robot according to the predictive control law

[0044] A prediction model, which uses the joint space position vector and joint space velocity vector of the robot on-site as the input of the prediction model, and generates the predicted state estimate value at the current moment and the predicted state estimate value at the next moment through the rolling horizon state prediction algorithm;

[0045] Generate a predictive control law based on the predicted state estimate value at the current moment, the predicted state estimate value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment. The predictive control law is used to control the on-site work of the robot;

[0046] A virtual model, which is used to send the joint space position vector and joint space velocity vector of the robot on-site generated by the robot's movement in continuous time;

[0047] Receive the predictive control law; the generation method of the predictive control law is:

[0048] Use the joint space position vector and joint space velocity vector of the robot on-site as the input of the prediction model, and generate the predicted state estimate value at the current moment and the predicted state estimate value at the next moment through the rolling horizon state prediction algorithm;

[0049] Generate a predictive control law based on the predicted state estimate value at the current moment, the predicted state estimate value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment; control the on-site movement of the robot according to the predictive control law.

[0050] The beneficial effects of the present invention are as follows: The present invention is oriented to the robot teleoperation process based on virtual simulation. The prediction model generates accurate predicted state vectors of the robot by using a data-driven method according to the collected historical motion data of the robot. Through the digital human-computer interaction process, stable predictive control instructions are formed, and then the on-site model is driven to carry out actual task operations; at the same time, the on-site model feeds back the on-site information of the robot to the virtual model for correction and ensures bilateral consistency; the advantage of the present invention is that compared with the traditional bilateral teleoperation process, the operator is not affected by the torque directly feedback by the on-site model, understands the on-site situation through the virtual simulation scenario, and has the characteristics of fast response. It can also control the interaction frequency and reduce the power consumption of the system by designing the update efficiency of the prediction and virtual models. Description of the Drawings

[0051] Figure 1 It is a schematic diagram of the steps of a data-driven and mechanism hybrid modeling and control method for virtual fusion control applied to a prediction model provided in Embodiment 1 of the present invention;

[0052] Figure 2 It is a schematic diagram of the steps of a data-driven and mechanism hybrid modeling and control method for virtual fusion control applied to a virtual model provided in Embodiment 2 of the present invention;

[0053] Figure 3 Schematic diagram of steps of a data-driven and mechanism hybrid modeling and control method for virtual fusion control applied to a field model provided in Embodiment 3 of the present invention;

[0054] Figure 4 Schematic diagram of steps of a data-driven and mechanism hybrid modeling and control method for virtual-real fusion control provided in Embodiment 3 of the present invention;

[0055] Figure 5 Schematic diagram of a block diagram of a data-driven and mechanism hybrid modeling and control system for virtual-real fusion control provided in Embodiment 1 of the present invention;

[0056] Figure 6 Simulation diagram of the angular velocity of the prediction model, virtual model, and field robot in the X-axis direction provided in Embodiment 1 of the present invention;

[0057] Figure 7 Simulation diagram of the angular velocity of the prediction model, virtual model, and field robot in the Y-axis direction provided in Embodiment 1 of the present invention;

[0058] Figure 8 Simulation diagram of the angular velocity of the prediction model, virtual model, and field robot in the Z-axis direction provided in Embodiment 1 of the present invention;

[0059] Figure 9 Simulation diagram of the trajectory tracking of the prediction model, virtual model, and field robot in the X-axis direction provided in Embodiment 1 of the present invention;

[0060] Figure 10 Simulation diagram of the trajectory tracking of the prediction model, virtual model, and field robot in the Y-axis direction provided in Embodiment 1 of the present invention;

[0061] Figure 11 Simulation diagram of the trajectory tracking of the prediction model, virtual model, and field robot in the Z-axis direction provided in Embodiment 1 of the present invention. Detailed implementation manner

[0062] The present invention will be described in detail below with reference to the accompanying drawings and specific implementation manners.

[0063] The object of the present invention is to provide a data-driven and mechanism hybrid modeling and control method for virtual-real fusion control, and to solve the technical problems of how to construct a new robot teleoperation system model with decoupled torques and improve the prediction accuracy of the prediction model.

[0064] Combined with Figure 1 , Embodiment 1 of the present invention provides a data-driven and mechanism hybrid modeling and control method for virtual-real fusion control, and the method applied to the prediction model includes:

[0065] Step S101: Use the joint space position vector and joint space velocity vector of the robot on-site as the input of the prediction model, and generate the predicted state estimate value at the current moment and the predicted state estimate value at the next moment through the rolling horizon state prediction algorithm.

[0066] In one embodiment, the present invention adopts the "bilateral isolation and effect synchronization" mode to establish three models of the prediction model, virtual model, and on-site model of the robot. Among them, "bilateral isolation" means that the movement of the remote robot interacting with the environment and the interaction between the operator and the behavior acquisition device do not directly establish a torque closed-loop during the task execution process, and "effect synchronization" means that the prediction model, virtual model, and on-site model separated in space can achieve state synchronization based on the reference behavior provided by the human-machine interaction model.

[0067] Among them, the prediction model is used to receive the motion data of the on-site robot, including the joint space position vector and joint space velocity vector on-site, and generate the predicted state estimate value through the rolling horizon state prediction algorithm according to the historical joint space position vector and joint space velocity vector. The predicted state estimate value includes the predicted state estimate value at the current moment and the predicted state estimate value at the next moment. It should be noted that in this embodiment, the prediction model replaces the prediction of the state estimate value of the robot at the next moment by collecting the motion data of the robot at the current moment with the real-time prediction of the predicted state estimate values of the robot at the current moment and the next moment based on the data-driven algorithm according to the collected historical motion data of the on-site robot, avoiding the "motion - wait - motion" mode of the conventional teleoperation system. And the predicted state estimate value meets the allowable range of the position tracking error on-site, realizing the effect synchronization between the prediction model and the on-site model.

[0068] Step S102: Generate a predictive control law according to the predicted state estimate value at the current moment, the predicted state estimate value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment generated by the virtual model. The predictive control law is used to control the on-site work of the robot.

[0069] In one embodiment, the operator and the robot are simulated and interacted by setting up a virtual model, so that the virtual model can generate the virtual desired trajectory of the robot to replace the operator to reach the on-site for human-machine interaction.

[0070] When the communication between the prediction model and the virtual model is normal, the virtual desired trajectory generated by the virtual model can be obtained in real time, and combined with the predicted state estimate value at the current moment and the predicted state estimate value at the next moment, a predictive control law of the robot is generated to be used for real-time control of the on-site robot movement.

[0071] It should be noted that the advantage of the present invention is that, compared with the traditional bilateral teleoperation process, the operator is not affected by the torque directly feedback by the on-site model. By virtual simulation of the real scene, it replaces the on-site human-machine interaction understanding, has the characteristics of fast response, and can also control the interaction frequency by controlling the update frequency of the prediction model and the virtual model, reducing the power consumption of the system.

[0072] Optionally, the joint space position vector and joint space velocity vector of the on-site and the desired trajectory generated by the virtual model are used as the input of the prediction model, and the predicted state vector is generated by the rolling horizon state prediction algorithm.

[0073] In one embodiment, the human-machine interaction dynamics response in the joint space of the on-site digital human-machine interaction model can be described by the following mathematical expression:

[0074]

[0075] Wherein, represents the artificially set interaction inertia matrix, represents the artificially set interaction damping matrix, is the acceleration vector of the desired joint angle trajectory under human-machine interaction, is the velocity vector of the desired joint angle trajectory under human-machine interaction, is the force exerted by the operator during the human-machine interaction process in the joint space.

[0076] The force exerted by the operator during the human-machine interaction process in the joint space can be accurately obtained through the sensor device. As the task time progresses, the desired trajectory generated by the virtual model can be obtained according to the human-machine interaction dynamics in formula (1).

[0077] When the robot performs a predetermined motion on-site for a period of time, the joint space position vector and joint space velocity vector of the on-site and other motion data of the robot are collected by the sensor as the initialization data of the virtual model. During this process, human-machine interaction is carried out in the virtual model by feeding back the on-site data of the robot, realizing synchronous tracking with a phase difference between the on-site model and the virtual model, and the prediction model can also obtain data such as the joint space position vector and joint space velocity vector of the robot on-site for further motion data estimation.

[0078] The dynamics description of the prediction model corresponding to the on-site digital human-machine interaction model is as follows:

[0079]

[0080] Wherein, is the inertia matrix of the prediction model, is the Coriolis force matrix of the prediction model, is the gravity term vector of the prediction model, is the predicted joint angular acceleration of the prediction model, is the predicted joint angular velocity of the prediction model, is the predicted joint angle of the prediction model, is the control law for tracking the desired trajectory q d of.

[0081] Step 1: Process the digital model of the robot operation site using the Euler discretization method, and iteratively calculate the difference of the predicted trajectory at the current moment and the difference of the predicted trajectory at the next moment.

[0082] In one implementation, the implementation of Step 1 is as follows:

[0083] Using the Euler discretization method, discretize the model in formula (2) to obtain the following mathematical model description:

[0084] Δq p (t k+1 ) - Δq p (t k ) = σ[M p (q p (t k )) -1 f p (t k ) + M p (q p (t k )) -1 τ p (t k )] (3)

[0085] where t k is the current moment, t k+1 is the next moment, Δq p (t k ) is the difference of the predicted trajectory at the current moment, and Δq p (t k+1 ) is the difference of the predicted trajectory at the next moment, and they can be calculated in the following way:

[0086]

[0087] In formulas (3), (4) and (5), σ is the time interval between two adjacent samplings. In formula (3), M p (q p (t k )) -1 represents the inverse matrix of M p (q p ) at time t k , and τp (t k ) is the control sequence τ p at time t k the sampled value, f p (t k ) is a non-linear function, which can be described by the following identity relationship:

[0088] f p (t k ) = -C p (Δq p (t k ), q p (t k ))Δq p (t k ) - G p (t k )(6)

[0089] Step 2: Calculate the predicted state vector at the current moment according to the difference of the predicted trajectory at the current moment and the difference of the predicted trajectory at the next moment;

[0090] In one embodiment, the specific implementation steps of Step 2 are as follows:

[0091] For formula (3), assume that the predicted state vector at the current moment is: x p (t k ) = (q p (t k ), T Δq p (t k ), T ), where q T (t p ) k is the transpose of the predicted trajectory at the previous moment, and Δq T (t p ) k is the transpose of the difference of the predicted trajectory at the previous moment. Thus, according to formula (6), the predicted state vector at the next moment can be deduced as T Among them, since the state quantity is unknown,

[0092]

[0093] where is a non-linear unknown function, expressed as:

[0094]

[0095] Using the linearization method, formula (7) can be linearly described as:

[0096] xp (t l+1 ) = A p x p (t k ) + e p + w(t k ) (8)

[0097] Wherein, x0 represents the state equilibrium point concerned by the linearized system, represents the linearization residual, represents the linearization error.

[0098] Step 3: Obtain the historical data of the predicted state vector at each moment within a preset time length;

[0099] In one embodiment, the specific implementation manner of Step 3 is as follows:

[0100] In the above description, since the state quantity is unknown, therefore, A p , e p and w(t k ) are all unknown, and a data-driven method needs to be used for prediction.

[0101] Define that the historical data relative to the previous moment for the data-driven process obtained from the data interface of the predicted state vector is the data sequence x p[-δ，0] , satisfying x p[-δ，0] = (x p (-δ) T , x p (-δ + σ) T , …, x p (t0) T ) T Considering the moment t k , L = min{t k , t s} is the preset time length, where t s is the artificially set time domain window length, x p (-δ) T is the row vector of the sampling values of the x p sequence at the -δ moment, x p (-δ + σ) T is the row vector of the sampling values of the x p sequence at the -δ + σ moment, x p (t0) T is the row vector of the sampling values of the x p sequence at the t0 moment, the r0 moment is the currently concerned moment, and the -δ moment is the earliest moment for historical available data sampling.

[0102] Step 4 solves for the predicted state estimate value at the current moment based on the optimization problem of minimizing the loss between the historical data of the solved predicted state vector and the predicted state vector at the current moment;

[0103] In one embodiment, the optimization problem to be solved is described as shown in formula (9):

[0104]

[0105] Its constraint conditions are:

[0106]

[0107] In formula (9), λ is a constant value, satisfying λ ∈ (0, 1], represents the predicted state vector at the current moment, is the true value sequence at time t k -L sampling value, α(t k ) is the optimal solution parameter vector at time t k , represents the predicted state estimate value at the current moment corresponding to the termination time t k .

[0108] It should be noted that in formula (10), H L+σ (x p[-δ，0] ) represents the Hankel matrix with a depth of L + σ and an element sequence of x p[-δ，0] , and the expression is:

[0109]

[0110] Among them, represents a column vector with all elements being 1 and a length of (δ + σ - L) / σ, represents the estimated value generated by the rolling horizon algorithm obtained at time t k depending on the historical data (x p (-L) T , x p (-L + σ) T , …, x p (t0) T ), T where x p (-L) T is the row vector of the sampling values of the x p sequence at -L, and x p (-L + σ) T is the x pRow vector of the sampled values of the sequence. Using the prediction algorithm of formulas (3)-(11), it can be ensured that the prediction error is strictly bounded.

[0111] Step 5 performs a non-linear calculation based on the predicted state estimate at the current moment to obtain the predicted state estimate at the next moment.

[0112] In one embodiment, by controlling the movement of the above window, historical data for calculating the predicted state estimate at the next moment is obtained, and further, the predicted state estimate at the next moment is calculated according to the methods of steps 3-4.

[0113] Optionally, a predictive control law is generated based on the predicted state estimate at the current moment, the predicted state estimate at the next moment, the virtual desired trajectory at the current moment generated by the virtual model, and the virtual desired trajectory at the next moment.

[0114] In one embodiment, the predictive control law is calculated by substituting the predicted state estimate at the current moment, the predicted state estimate at the next moment, the virtual desired trajectory at the current moment generated by the virtual model, and the virtual desired trajectory at the next moment into formula (12).

[0115] The specific calculation method of the predictive control law is as follows:

[0116]

[0117] where, it is defined that: t k represents the current moment, t k+1 represents the next moment, is the estimation matrix of the prediction model obtained based on the state estimate of the rolling horizon optimization, is the non-linear function estimation term of the prediction model obtained based on the state estimate of the rolling horizon optimization, is the predicted state estimate at the next moment, is the predicted state estimate at the current moment, q d (t k+1 ) is the virtual desired trajectory at the next moment generated by the virtual model, q d (t k ) is the virtual desired trajectory at the current moment generated by the virtual model, s p (t k ) is the sliding mode surface at time t k , σ is the time interval between two adjacent samplings, η p is the reaching gain in the prediction model, and its value range is greater than 0 and less than 2, C sp is the sliding mode surface linear gain in the prediction model, τ p (t k ) is the predictive control law at the current moment.

[0118] Optionally, the virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment generated by the virtual model are transmitted to the prediction model in a wired or wireless manner.

[0119] In the case of UAV interaction, the continuous-time model of the robot on-site operation dynamics is as follows:

[0120]

[0121] where, is the position vector in the joint space, is the joint space velocity vector, is the joint space acceleration vector, is the inertia matrix of the on-site operation dynamics, is the Coriolis force matrix of the on-site operation dynamics, is the gravity term in the joint space, τ m is the control torque of the robot on-site operation, satisfying:

[0122]

[0123] where, is the designed sliding mode surface, e m is the position tracking error at the operation site, satisfying e m = q m - q d , q d is the position trajectory sequence generated by the prediction model, is the velocity tracking error at the operation site, satisfying is the velocity trajectory sequence generated by the prediction model, C sm is the linear gain parameter of the sliding mode surface, η m is the reaching law gain, satisfying η m > 0, is the acceleration trajectory sequence generated by the prediction model. According to the controller design given by formula (12), it can be ensured that the robot motion control error at the operation site is ultimately uniformly bounded.

[0124] Embodiment 2 of the present invention provides a data-driven and mechanism hybrid modeling and control method for virtual-real fusion control. As Figure 2 shown, the method applied to the virtual model includes:

[0125] Step S201: Process the digital model of the robot operation site by using the Euler discretization method, and iteratively calculate the virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment;

[0126] In one embodiment, the virtual model is an iteratively updated digital model of the robot operation site under the condition that the operator confirms the effectiveness of the existing actual data obtained by the continuous-time model of the robot on-site operation dynamics through the above-mentioned UAV interaction and meets the communication delay condition between the on-site model and the virtual model. It needs to be described by a discrete-time dynamics process, specifically as follows:

[0127] Δq v (t k+1 ) - Δq v (t k ) = σ[M v (q v (t k )) -1 f v (t k ) + M v (q v (t k )) -1 τv(t k )] (15)

[0128] Wherein, Δq v (t k+1 ) is the state difference of the digital model at time t k+1 , and Δq v (t k ) is the state difference of the digital model at time t k . They can be calculated in the following way

[0129]

[0130] In formulas (15), (16) and (17), q v (t k+2 ) represents the virtual desired trajectory at time t k+2 , q v (t k+1 ) represents the virtual desired trajectory at time t k+1 , and q v (t k ) represents the virtual desired trajectory at time t k . In formula (15), M v (q v (t k )) -1 represents the inverse matrix of M v (q v ) at time t k . is the inertia matrix of the virtual model, is the control quantity at time t k , and f v(t k ) is a non - linear function and can be described by the following identity relationship:

[0131] f v (t k ) = - C v (Δq v (t k ), q v (t k ))Δq v (t k ) - G v (t k ) (18)

[0132] Among them, is the inertia matrix of the virtual model at time t k , and is the Coriolis force matrix of the virtual model at time t k .

[0133] Step S202: Send the virtual desired trajectory at the current time and the virtual desired trajectory at the next time;

[0134] The virtual desired trajectory at the current time and the virtual desired trajectory at the next time are used to generate a predictive control law.

[0135] It should be noted that the virtual model also receives the predicted state estimate value at the current time and the predicted state estimate value at the next time generated by the test model;

[0136] Generate the virtual control law at the current time according to the virtual desired trajectory at the current time, the virtual desired trajectory at the next time, the predicted state estimate value at the current time, and the predicted state estimate value at the next time, and the virtual control law is used to control the movement of the robot in the virtual model.

[0137] In one embodiment, further participate the virtual desired trajectory at the current time, the virtual desired trajectory at the next time, the predicted state estimate value at the current time, and the predicted state estimate value at the next time in the calculation of formula (19) to obtain the control law of the virtual model. The calculation method of the control law of the virtual model is specifically as shown in formula (19):

[0138]

[0139] Among them, f v (t k ) is a non - linear function, C sv is the sliding - mode surface linear gain of the virtual model, satisfying C sv > 0, η v is the reaching - law gain, satisfying η v∈(0, 2), s v (t k ) is the sliding mode surface of the virtual model, satisfying σ is the time interval between two adjacent samplings, η p is the reaching gain in the prediction model, q m (t k+1 ) is the sampling value of the prediction model trajectory at time t k+1 , q m (t k ) is the sampling value of the prediction model trajectory at time t k , where τ v (t k ) is the control law of the virtual model at the current moment, used to realize human-machine interaction in the virtual model. By using the algorithm in formula (19), it can ensure that the virtual model tracks the robot operation site, and the error between the two is ultimately uniformly bounded.

[0140] Embodiment 3 of the present invention provides a data-driven and mechanism hybrid modeling and control method for virtual-real fusion control, as Figure 3 shown, the method applied to the field model includes:

[0141] Step S301: Send the joint space position vector and joint space velocity vector of the field generated by the robot's movement in continuous time;

[0142] In one embodiment, the robot of the field model moves according to a preset trajectory for a period of time to generate the joint space vector and the key space velocity vector.

[0143] Step S302: Receive the predictive control law; the generation method of the predictive control law is:

[0144] Take the joint space position vector and joint space velocity vector of the robot's field as the input of the prediction model, and generate the prediction state estimate value at the current moment and the prediction state estimate value at the next moment through the rolling horizon state prediction algorithm;

[0145] Step S303: Generate a predictive control law according to the prediction state estimate value at the current moment, the prediction state estimate value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment;

[0146] Step S304: Control the movement of the robot field according to the predictive control law.

[0147] According to Embodiment 4 of the present invention, a data-driven and mechanism hybrid modeling and control method for virtual-real fusion control is provided, including the following steps:

[0148] Step S401: The on-site model sends the joint space position vector and joint space velocity vector of the on-site generated by the robot's movement in continuous time;

[0149] Step S402: The prediction model receives the joint space position vector and joint space velocity vector of the on-site of the robot sent by the on-site model, inputs them into the prediction model, and generates the predicted state estimate value at the current moment and the predicted state estimate value at the next moment through the rolling horizon state prediction algorithm;

[0150] Step S403: The virtual model iteratively calculates the virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment by using the Euler discretization method;

[0151] Step S404: Send the virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment;

[0152] Step S405: The virtual model generates a predictive control law based on the predicted state estimate value at the current moment, the predicted state estimate value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment, and the predictive control law is used to control the on-site work of the robot;

[0153] Step S406: The virtual model receives the predicted state estimate value at the current moment and the predicted state estimate value at the next moment generated by the prediction model;

[0154] Step S407: Generate the virtual control law at the current moment according to the virtual desired trajectory at the current moment, the virtual desired trajectory at the next moment, the predicted state estimate value at the current moment, and the predicted state estimate value at the next moment.

[0155] Embodiment 5 of the present invention provides a data-driven and mechanism hybrid modeling and control system for virtual-real fusion manipulation, as Figure 5 shown, the system includes:

[0156] An on-site model, which is used to send the joint space position vector and joint space velocity vector of the on-site generated by the robot's movement in continuous time; receive the predictive control law; the generation method of the predictive control law is:

[0157] Taking the joint space position vector and joint space velocity vector of the on-site of the robot as the input of the prediction model, and generating the predicted state estimate value at the current moment and the predicted state estimate value at the next moment through the rolling horizon state prediction algorithm;

[0158] Generating a predictive control law according to the predicted state estimate value at the current moment, the predicted state estimate value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment;

[0159] Control the on-site movement of the robot according to the predicted control law;

[0160] A prediction model, which uses the on-site joint space position vector and joint space velocity vector of the robot as the input of the prediction model, and generates the predicted state estimate value at the current moment and the predicted state estimate value at the next moment through the rolling horizon state prediction algorithm;

[0161] Generate a predicted control law based on the predicted state estimate value at the current moment, the predicted state estimate value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment. The predicted control law is used to control the on-site work of the robot;

[0162] A virtual model, which is used to send the on-site joint space position vector and joint space velocity vector generated by the movement of the robot in continuous time;

[0163] Receive the predicted control law; the generation method of the predicted control law is:

[0164] Use the on-site joint space position vector and joint space velocity vector of the robot as the input of the prediction model, and generate the predicted state estimate value at the current moment and the predicted state estimate value at the next moment through the rolling horizon state prediction algorithm;

[0165] Generate a predicted control law based on the predicted state estimate value at the current moment, the predicted state estimate value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment; control the on-site movement of the robot according to the predicted control law.

[0166] In one embodiment, in combination with Figure 5 The present invention provides a specific interaction method among the on-site model, prediction model, and virtual model in this system. First, the robot moves along a preset trajectory on-site for a period of time. Through the virtual model, the real-time joint space position vector and joint space velocity vector of the on-site robot are received for human-machine interaction operations under simulation. At the same time, the virtual model will, based on the predicted joint space vector at the current moment and the predicted joint space vector at the next moment sent by the prediction model, generate the control law of the virtual model through the sliding mode control module under the condition of satisfying the tracking error, realizing the human-machine interaction without people going to the site and directly simulating the human-machine operation through the virtual model.

[0167] While the robot is moving on-site, the prediction model collects the on-site joint space position vector and joint space velocity vector of the robot, and uses a data-driven method for rolling horizon state prediction in combination with the historical prediction state sequence to obtain the optimal state estimate value. Further, through the prediction control module, a predicted control law for predicting and tracking the desired trajectory is generated based on the state estimate value, predicted joint angular velocity, and predicted joint angle.

[0168] The present invention is directed to the process of robot teleoperation based on virtual simulation. The operator forms stable control commands through a digital human-machine interaction process according to the state of an accurate prediction model, and then drives the on-site model to carry out actual task operations. After the on-site information is fed back, the virtual model is corrected to ensure bilateral consistency. The advantage of the present invention is that compared with the traditional bilateral teleoperation process, the operator is not affected by the torque directly feedback from the on-site model. The operator understands the on-site situation through the virtual simulation scenario, and at the same time has the characteristics of fast response. It can also control the interaction frequency and reduce the power consumption of the system by designing the prediction and the update efficiency of the virtual model.

[0169] Combined with Figures 6 - 11 , it can be seen from the angular velocity changes of the above prediction model, virtual model and on-site model in three different directions that due to the data transmission delay between the on-site robot end and the virtual scenario end, the angular velocity change of the virtual scenario end lags behind that of the on-site robot end as a whole. However, the designed control algorithm can ensure that its angular velocity can smoothly track the angular velocity change of the on-site robot end, that is, it can accurately display the angular velocity-related information of the on-site robot end. In the digital human-machine interaction process, the physical human-machine interaction is directly fed back to the prediction model end. Due to the data transmission delay, the angular velocity change of the on-site robot lags behind the prediction model as a whole. However, the designed data-driven rolling horizon estimation control method can ensure that the trajectory error of the angular velocity is ultimately uniformly stable and bounded.

Claims

1. A modeling and control method that combines data-driven and mechanism-based approaches for virtual-real fusion manipulation, characterized in that The method applied to the prediction model includes: Taking the joint space position vector and joint space velocity vector of the robot on-site as the input of the prediction model, and generating the predicted state estimate value at the current moment and the predicted state estimate value at the next moment through the rolling horizon state prediction algorithm; Generating a predictive control law based on the predicted state estimate value at the current moment, the predicted state estimate value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment, where the predictive control law is used to control the on-site work of the robot; Generating the predicted state estimate value includes: Step 1, iteratively calculating the difference of the predicted trajectory at the current moment and the difference of the predicted trajectory at the next moment by using the Euler discretization method; Step 2, calculating the predicted state vector at the current moment according to the difference of the predicted trajectory at the current moment and the difference of the predicted trajectory at the next moment; Step 3, obtaining the historical data of the predicted state vector through a sliding window, where the historical data is calculated based on the joint space position vector and the joint space velocity vector; Step 4, solving the predicted state estimate value at the current moment with the minimum loss between the historical data and the predicted state vector at the current moment as the optimization problem; Step 5, performing a non-linear calculation on the predicted state estimate value at the current moment to obtain the predicted state estimate value at the next moment; Generating a predictive control law based on the predicted state estimate value at the current moment, the predicted state estimate value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment includes: The specific calculation method of the predictive control law is: Among them, t k represents the current moment, t k+1 represents the next moment, is the estimation matrix of the prediction model obtained based on the state estimation value of the rolling horizon optimization, is the estimated term of the nonlinear function of the prediction model obtained based on the state estimation value of the rolling horizon optimization, is the predicted state estimation value at the next moment, is the predicted state estimation value at the current moment, q d (t k+1 ) is the virtual desired trajectory at the next moment, q d (t k ) is the virtual desired trajectory at the current moment, s p (t k ) is for t k the sliding mode surface at the moment, σ is the time interval between two adjacent samplings, η p is the approaching gain in the prediction model, C sp is the linear gain of the sliding mode surface in the prediction model, τ p (t k ) is the predicted control law at the current moment.

2. A data-driven and mechanism hybrid modeling and control method for virtual-real fusion manipulation as described in claim 1, where the virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment are transmitted to the prediction model in a wired or wireless manner.

3. A data-driven and mechanism hybrid modeling and control method for virtual-real fusion manipulation as claimed in claim 1, characterized in that The method applied to the virtual model includes: Iteratively calculating the virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment by using the Euler discretization method; Sending the virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment; The virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment are used to generate a predictive control law.

4. The data-driven and mechanism hybrid modeling and control method for virtual-real fusion manipulation as claimed in claim 1, wherein, The method applied to the on-site model includes: Sending the joint space position vector and joint space velocity vector of the robot on-site generated by the robot's movement in continuous time; Receiving the predictive control law; the method for generating the predictive control law is: Taking the joint space position vector and joint space velocity vector of the robot on-site as the input of the prediction model, and generating the predicted state estimate value at the current moment and the predicted state estimate value at the next moment through the rolling horizon state prediction algorithm; Generating a predictive control law based on the predicted state estimate value at the current moment, the predicted state estimate value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment; Controlling the on-site movement of the robot according to the predictive control law.

5. A data-driven and mechanism hybrid modeling and control method for virtual-real fusion manipulation as claimed in claim 1, characterized in that, Including the following steps: The on-site model sends the joint space position vector and joint space velocity vector of the robot on-site generated by the robot's movement in continuous time; The prediction model receives the joint space position vector and joint space velocity vector of the robot in the field sent by the on-site model, and inputs them into the prediction model. The prediction state estimate value at the current moment and the prediction state estimate value at the next moment are generated through the rolling horizon state prediction algorithm. The virtual model iteratively calculates the virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment by using the Euler discretization method. Send the virtual desired trajectory at the current moment and the virtual desired trajectory at the next moment. The virtual model generates a predictive control law based on the predicted state estimate value at the current moment, the predicted state estimate value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment. The predictive control law is used to control the on-site work of the robot. The virtual model receives the predicted state estimate value at the current moment and the predicted state estimate value at the next moment generated by the prediction model. Generate the virtual control law at the current moment based on the virtual desired trajectory at the current moment, the virtual desired trajectory at the next moment, the predicted state estimate value at the current moment, and the predicted state estimate value at the next moment.

6. A data-driven and mechanism hybrid modeling and control system for virtual fusion manipulation, characterized in that, A data-driven and mechanism hybrid modeling and control method for virtual-real fusion manipulation according to any one of claims 1-5. Including: An on-site model for sending the joint space position vector and joint space velocity vector of the robot in the field generated by the movement in continuous time; receiving the predictive control law; the method for generating the predictive control law is: Taking the joint space position vector and joint space velocity vector of the robot in the field as the input of the prediction model, and generating the predicted state estimate value at the current moment and the predicted state estimate value at the next moment through the rolling horizon state prediction algorithm. Generating a predictive control law based on the predicted state estimate value at the current moment, the predicted state estimate value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment. Controlling the on-site movement of the robot according to the predictive control law. A prediction model for taking the joint space position vector and joint space velocity vector of the robot in the field as the input of the prediction model, and generating the predicted state estimate value at the current moment and the predicted state estimate value at the next moment through the rolling horizon state prediction algorithm. Generating a predictive control law based on the predicted state estimate value at the current moment, the predicted state estimate value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment. The predictive control law is used to control the on-site work of the robot. A virtual model for sending the joint space position vector and joint space velocity vector of the robot in the field generated by the movement in continuous time. Receiving the predictive control law; the method for generating the predictive control law is: Taking the joint space position vector and joint space velocity vector of the robot in the field as the input of the prediction model, and generating the predicted state estimate value at the current moment and the predicted state estimate value at the next moment through the rolling horizon state prediction algorithm. Generating a predictive control law based on the predicted state estimate value at the current moment, the predicted state estimate value at the next moment, the virtual desired trajectory at the current moment, and the virtual desired trajectory at the next moment; controlling the on-site movement of the robot according to the predictive control law.

Images