A robot virtual modeling and haptic control method and device

Abstract

The application discloses a robot virtual modeling and tactile control method and device, the method comprises the following steps: establishing a virtual robot in a virtual environment according to physical entity data of the robot, and establishing a connection between a force feedback controller and the virtual robot; an operator operates the force feedback controller, and sends end coordinate data of the force feedback controller to the virtual robot; the virtual robot calculates posture data according to the end coordinate data, controls the virtual robot to perform posture transformation according to the posture data, obtains target interaction data obtained by interacting with the virtual environment in the posture transformation process, and sends the target interaction data to the force feedback controller; the force feedback controller executes a force feedback action according to the target interaction data; the method realizes the target of providing two-way immersive tactile interaction for the operator and the virtual robot, improves the authenticity of interaction with the virtual robot, and can be applied to robot control under virtual reality and augmented reality.

Description

Technical Field

[0001] This application relates to the field of robot simulation and control technology, and in particular to a robot virtual modeling and tactile control method and device. Background Technology

[0002] With the continuous development of manufacturing and logistics, robots are widely used in assembly lines, production, and transportation, becoming a core component of smart factory construction. Robots can perform many repetitive, dangerous, and precision-critical tasks, making high-precision control of robots particularly important.

[0003] Virtual robot modeling is a technology that simulates robot motion and control. It uses computer simulations to mimic robot actions and effects, and is applied to robot design, layout, and control. In particular, the emergence of technologies such as virtual reality, augmented reality, and mixed reality allows for the simulation, optimization, and correction of physical robot control through real-time virtual modeling, becoming crucial for supporting human-robot interaction and collaborative control.

[0004] In the process of interacting with traditional robot virtual models, one-way visual and auditory operation can only be performed by relying on head-mounted displays and controllers. This leads to a certain difference between virtual modeling and actual robots. In other words, traditional robot virtual modeling and interaction have low realism and controllability, making it difficult to meet the needs of industrial applications. Summary of the Invention

[0005] In view of this, this application provides a robot virtual modeling and tactile control method and apparatus, which helps to improve the realism of the interaction between the operator and the virtual robot.

[0006] According to one aspect of this application, a method for robot virtual modeling and tactile control is provided, the method comprising:

[0007] A virtual robot is created in a virtual environment based on the physical entity data of the robot, and a connection is established between the force feedback controller and the virtual robot.

[0008] The operator operates the force feedback controller and sends the end coordinate data of the force feedback controller to the virtual robot;

[0009] The virtual robot calculates posture data based on the end-effector coordinate data, controls the virtual robot to perform posture changes based on the posture data, acquires target interaction data that interacts with the virtual environment during the posture change process, and sends the target interaction data to the force feedback controller.

[0010] The force feedback controller performs force feedback actions based on the target interaction data.

[0011] Optionally, the physical entity data includes model data, mechanical dimension data, number of joints data, and range of motion data. The step of building a virtual robot in a virtual environment based on the robot's physical entity data includes:

[0012] Based on the robot's model data, mechanical size data, number of joints data, and range of motion data, virtual models of each component of the virtual robot are established. Based on the actual connection relationship of the robot, the virtual models of each component are combined to form a virtual three-dimensional model of the virtual robot, and the hierarchical relationship of each joint in the virtual three-dimensional model is bound.

[0013] A virtual robot is created based on the robot's physical properties and the bound virtual 3D model. The physical properties include mass, model size, and surface material. The virtual models of each component include a base model, a linkage model, a sensor model, and a front-end tool model.

[0014] Optionally, the binding of the hierarchical relationship between each joint in the virtual 3D model includes:

[0015] Set the base model as the parent node and the first link model as the child node of the base model;

[0016] According to the preset front-to-back order of the actual connection relationship of the robot, each link model is set as a child node of the previous link model in sequence;

[0017] Set the sensor model and the front-end tool model as child nodes of the last link model.

[0018] Optionally, the step of establishing a virtual robot based on the robot's physical properties and the bound virtual 3D model includes:

[0019] Based on the robot's mass, model size, and surface material, rigid body components, collision body components, and surface material components are built for the bound virtual 3D model to create a virtual robot.

[0020] The rigid body component is determined based on the mass of the robot, the collision body component is determined based on the model dimensions of the robot, and the surface material component is determined based on the surface material properties of the robot.

[0021] Optionally, the virtual robot calculates posture data based on the end-effector coordinate data, including:

[0022] Establish the Cartesian coordinate space of the virtual robot, and establish an independent coordinate system for each joint of the virtual robot in the Cartesian coordinate space based on the end-effector coordinate data;

[0023] The transformation matrix of adjacent joints of the virtual robot is determined based on the independent coordinate system of each joint of the virtual robot;

[0024] Based on the transformation matrix of each adjacent joint, the posture information of each joint of the virtual robot is calculated sequentially to obtain posture data.

[0025] Optionally, acquiring the target interaction data that interacts with the virtual environment during the attitude transformation process and sending the target interaction data to the force feedback controller includes:

[0026] The system detects whether the virtual robot comes into contact with the virtual environment during posture transformation. When contact occurs, it acquires collision contact position data and collision virtual attribute data of the contacted virtual environment. The collision contact position data and the collision virtual attribute data are used as collision target interaction data and sent to the force feedback controller. The virtual environment includes virtual objects.

[0027] If the operator presses the grab button while the virtual robot is in contact with the virtual object, the virtual robot is controlled to perform a grabbing action, acquire grabbing contact position data and grabbing virtual attribute data, use the grabbing contact position data and the grabbing virtual attribute data as grabbing target interaction data, and send the grabbing target interaction data to the force feedback controller; if the operator releases the grabbing button, the virtual robot is controlled to perform a release action, wherein the target interaction data includes collision target interaction data and grabbing target interaction data.

[0028] Optionally, the virtual attribute data includes rigid body component parameters, collider component parameters, and surface material component parameters;

[0029] The rigid body component parameters are read from the rigid body component of the contacting virtual object and are determined by the mass of the contacting virtual object; the collider component parameters are read from the collider component of the contacting virtual object and are determined by the model outline of the contacting virtual object; the surface material component parameters are read from the surface material component of the contacting virtual object and are determined by the hardness data, viscosity data, and roughness data of the surface of the contacting virtual object.

[0030] Optionally, the force feedback action includes gravity feedback action, elastic force feedback action, viscous force feedback action, and friction force feedback action, and the force feedback controller executes the force feedback action according to the target interaction data, including:

[0031] The force feedback controller executes the gravity feedback action based on the contact position data and the mass of the virtual object being contacted;

[0032] The force feedback controller executes the elastic force feedback action based on the contact position data and the hardness data of the contacted virtual object;

[0033] The force feedback controller executes the viscous force feedback action based on the contact position data and the viscosity data of the contacted virtual object;

[0034] The force feedback controller executes the friction force feedback action based on the contact position data and the roughness data of the contacted virtual object.

[0035] According to another aspect of this application, a robot virtual modeling and tactile control device is provided, the device comprising:

[0036] A module is used to build a virtual robot in a virtual environment based on the robot's physical entity data, and to establish a connection between the force feedback controller and the virtual robot.

[0037] An operation module is used by the operator to operate the force feedback controller and send the end coordinate data of the force feedback controller to the virtual robot;

[0038] An interaction module is used for the virtual robot to calculate posture data based on the end-effector coordinate data, control the virtual robot to perform posture transformation based on the posture data, acquire target interaction data that interacts with the virtual environment during the posture transformation, and send the target interaction data to the force feedback controller.

[0039] The feedback module is used by the force feedback controller to perform force feedback actions based on the target interaction data.

[0040] Optionally, the building module is further used for:

[0041] Based on the robot's model data, mechanical size data, number of joints data, and range of motion data, virtual models of each component of the virtual robot are established. Based on the actual connection relationship of the robot, the virtual models of each component are combined to form a virtual three-dimensional model of the virtual robot, and the hierarchical relationship of each joint in the virtual three-dimensional model is bound.

[0042] A virtual robot is created based on the robot's physical properties and the bound virtual 3D model. The physical properties include mass, model size, and surface material. The virtual models of each component include a base model, a linkage model, a sensor model, and a front-end tool model.

[0043] Optionally, the building module is further used for:

[0044] Set the base model as the parent node and the first link model as the child node of the base model;

[0045] According to the preset front-to-back order of the actual connection relationship of the robot, each link model is set as a child node of the previous link model in sequence;

[0046] Set the sensor model and the front-end tool model as child nodes of the last link model.

[0047] Optionally, the building module is further used for:

[0048] Based on the robot's mass, model size, and surface material, rigid body components, collision body components, and surface material components are built for the bound virtual 3D model to create a virtual robot.

[0049] The rigid body component is determined based on the mass of the robot, the collision body component is determined based on the model dimensions of the robot, and the surface material component is determined based on the surface material properties of the robot.

[0050] Optionally, the interaction module is further configured to:

[0051] Establish the Cartesian coordinate space of the virtual robot, and establish an independent coordinate system for each joint of the virtual robot in the Cartesian coordinate space based on the end-effector coordinate data;

[0052] The transformation matrix of adjacent joints of the virtual robot is determined based on the independent coordinate system of each joint of the virtual robot;

[0053] Based on the transformation matrix of each adjacent joint, the posture information of each joint of the virtual robot is calculated sequentially to obtain posture data.

[0054] Optionally, the interaction module is further configured to:

[0055] The system detects whether the virtual robot comes into contact with the virtual environment during posture transformation. When contact occurs, it acquires collision contact position data and collision virtual attribute data of the contacted virtual environment. The collision contact position data and the collision virtual attribute data are used as collision target interaction data and sent to the force feedback controller. The virtual environment includes virtual objects.

[0056] If the operator presses the grab button while the virtual robot is in contact with the virtual object, the virtual robot is controlled to perform a grabbing action, acquire grabbing contact position data and grabbing virtual attribute data, use the grabbing contact position data and the grabbing virtual attribute data as grabbing target interaction data, and send the grabbing target interaction data to the force feedback controller; if the operator releases the grabbing button, the virtual robot is controlled to perform a release action, wherein the target interaction data includes collision target interaction data and grabbing target interaction data.

[0057] Optionally, the virtual attribute data includes rigid body component parameters, collider component parameters, and surface material component parameters;

[0058] The rigid body component parameters are read from the rigid body component of the contacting virtual object and are determined by the mass of the contacting virtual object; the collider component parameters are read from the collider component of the contacting virtual object and are determined by the model outline of the contacting virtual object; the surface material component parameters are read from the surface material component of the contacting virtual object and are determined by the hardness data, viscosity data, and roughness data of the surface of the contacting virtual object.

[0059] Optionally, the feedback module is further configured to:

[0060] The force feedback controller executes the gravity feedback action based on the contact position data and the mass of the virtual object being contacted;

[0061] The force feedback controller executes the elastic force feedback action based on the contact position data and the hardness data of the contacted virtual object;

[0062] The force feedback controller executes the viscous force feedback action based on the contact position data and the viscosity data of the contacted virtual object;

[0063] The force feedback controller executes the friction force feedback action based on the contact position data and the roughness data of the contacted virtual object.

[0064] By employing the above technical solutions, this application provides a robot virtual modeling and tactile control method and device. It accurately establishes a virtual robot using the robot's physical entity data. The operator manipulates a force feedback controller to control the virtual robot to perform posture changes based on end-effector coordinate data, acquiring target interaction data with the virtual environment during the posture change process. The force feedback controller executes force feedback actions based on the target interaction data, providing the operator with force tactile perception. This achieves the goal of providing two-way immersive tactile interaction between the operator and the virtual robot, improving the realism of interaction with the virtual robot. It can be applied to robot control in virtual reality and augmented reality environments.

[0065] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0066] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0067] Figure 1 A flowchart illustrating a robot virtual modeling and tactile control method provided in an embodiment of this application is shown.

[0068] Figure 2 A flowchart illustrating another robot virtual modeling and tactile control method provided in an embodiment of this application is shown.

[0069] Figure 3 A schematic diagram of the structure of a robot virtual modeling and tactile control device provided in an embodiment of this application is shown. Detailed Implementation

[0070] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0071] In existing technologies, virtual robots are mainly static representations of physical robots and cannot simulate the dynamic behaviors and attributes of physical robots in the real world, such as collisions and mass. This leads to certain differences between virtual robots and physical robots. Virtual robots cannot accurately simulate a series of behaviors of physical robots in the real world. In the process of interacting with traditional virtual robots, one-way visual and auditory operation can only be performed by relying on head-mounted displays and controllers, making it difficult to perform two-way immersive tactile interaction. In other words, the creation and interaction of traditional virtual robots have low realism and controllability, making it difficult to meet the needs of industrial applications.

[0072] This embodiment provides a robot virtual modeling and tactile control method, such as Figure 1 As shown, the method includes:

[0073] Step 101: Create a virtual robot in the virtual environment based on the robot's physical entity data, and establish a connection between the force feedback controller and the virtual robot.

[0074] This application's embodiments can be applied to the interaction process with virtual robots to achieve tactile interaction with the virtual robot, and can also be applied to robot control in virtual reality and augmented reality. First, a virtual robot is established in the virtual environment based on the robot's physical entity data, and a connection is established between the force feedback controller and the virtual robot. For example, the virtual robot can be established in virtual reality glasses, and the connection between the force feedback controller and the virtual robot can be established through wireless networks, interfaces, etc. The virtual robot can be accurately established using the robot's physical entity data, and the connection between the force feedback controller and the virtual robot can be established, preparing for the subsequent implementation of tactile interaction.

[0075] Step 102: The operator operates the force feedback controller and sends the end coordinate data of the force feedback controller to the virtual robot.

[0076] Next, the operator operates the force feedback controller and sends the end-effector coordinate data of the force feedback controller to the virtual robot, realizing one-way tactile interaction between the operator and the virtual robot, enabling the virtual robot to perform data interaction according to the end-effector coordinate data.

[0077] Step 103: The virtual robot calculates posture data based on the end-effector coordinate data, controls the virtual robot to perform posture transformation based on the posture data, obtains target interaction data of interacting with the virtual environment during the posture transformation, and sends the target interaction data to the force feedback controller.

[0078] Next, to control the virtual robot to simulate the posture corresponding to the aforementioned end-effector coordinate data, posture transformation is required. Therefore, posture data needs to be calculated first using the aforementioned end-effector coordinate data. The virtual robot then performs posture transformation based on the aforementioned posture data, and the target interaction data of the virtual robot interacting with the virtual environment during the aforementioned posture transformation process is obtained. This completes the control and posture change of the virtual robot. The target interaction data is then sent to the force feedback controller to facilitate the subsequent implementation of tactile interaction.

[0079] Step 104: The force feedback controller performs a force feedback action based on the target interaction data.

[0080] Next, the force feedback controller executes force feedback actions based on the target interaction data to complete two-way tactile interaction. The force feedback controller can provide multi-degree-of-freedom force feedback information, giving the operator force tactile perception.

[0081] By applying the technical solution of this embodiment, a virtual robot is accurately established using the physical entity data of the robot. The operator manipulates the force feedback controller to control the virtual robot to change its posture according to the end-effector coordinate data, and obtains the target interaction data with the virtual environment during the posture change process. The force feedback controller executes force feedback actions according to the target interaction data, providing the operator with force tactile perception, thus achieving the goal of providing two-way immersive tactile interaction between the operator and the virtual robot, and improving the realism of interacting with the virtual robot.

[0082] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, and to fully illustrate the specific implementation process of this embodiment, the physical properties include mass, model size, and surface material, and the target interaction data includes collision target interaction data and grasping target interaction data, providing another method for robot virtual modeling and tactile control, such as... Figure 2 As shown, the method includes:

[0083] Step 201: Based on the robot's model data, mechanical size data, number of joints data, and range of motion data, establish virtual models of each component of the virtual robot. Based on the actual connection relationship of the robot, combine the virtual models of each component to form a virtual three-dimensional model of the virtual robot, and bind the hierarchical relationship of each joint in the virtual three-dimensional model.

[0084] In this embodiment, firstly, based on the robot's model data, mechanical size data, number of joints data, and range of motion data, virtual models of each component of the virtual robot are established. These virtual models include a base model, a linkage model, a sensor model, and a front-end tool model. Next, based on the actual connection relationship of the robot, the virtual models of each component are combined to form a virtual three-dimensional model of the virtual robot, and the hierarchical relationship of each joint is bound.

[0085] Step 202: Based on the robot's mass, model size, and surface material, construct rigid body components, collision body components, and surface material components for the bound virtual 3D model to establish a virtual robot; establish a connection between the force feedback controller and the virtual robot.

[0086] Next, the rigid body component is determined based on the robot's mass, the collider component is determined based on the robot's model dimensions, and the surface material component is determined based on the robot's surface material. A rigid body component is built for the virtual 3D model based on the robot's mass, a collider component is built for the virtual 3D model based on the robot's model dimensions, and a surface material component is built for the virtual 3D model based on the robot's surface material. A virtual robot is then established, and a connection is established between the force feedback controller and the virtual robot. By adding the rigid body component, collider component, and surface material component, the virtual robot is given tactile interaction attributes, improving the realism of the interaction.

[0087] Step 203: The operator operates the force feedback controller and sends the end coordinate data of the force feedback controller to the virtual robot.

[0088] Next, the operator operates the force feedback controller and sends the end-effector coordinate data of the force feedback controller to the virtual robot.

[0089] Step 204: Establish the Cartesian coordinate space of the virtual robot; establish an independent coordinate system for each joint of the virtual robot in the Cartesian coordinate space based on the end-effector coordinate data; determine the transformation matrix of adjacent joints of the virtual robot based on the independent coordinate system of each joint; and calculate the posture information of each joint of the virtual robot sequentially based on the transformation matrix of each adjacent joint to obtain posture data.

[0090] Next, the Cartesian coordinate space of the virtual robot is established according to preset creation rules. Based on the end-effector coordinate data, an independent coordinate system for each joint of the virtual robot is established in the Cartesian coordinate space, wherein the right-hand rule is used to establish the independent coordinate systems of each joint. Based on the independent coordinate systems of each joint of the virtual robot, the transformation matrix between adjacent joints is determined as follows:

[0091]

[0092] Where i represents the i-th joint in the virtual robot. Let θ represent the transformation matrix between the (i-1)th joint and the ith joint. i Let l be the joint angle variable of the i-th joint. i-1 d represents the distance along the X-axis between the (i-1)th joint and the ith joint. i α represents the translational distance along the Z-axis between the (i-1)th joint and the ith joint. i-1 This represents the rotation angle around the X-axis between the (i-1)th joint and the ith joint;

[0093] Then, based on the transformation matrix of each adjacent joint, the pose information of each joint of the virtual robot is calculated sequentially. For example, the pose information of the robot's end effector relative to joint 1 can be calculated as follows: Among them, T i i+1 T1 represents the transformation matrix of two adjacent joints, i-th and (i+1)-th. End This represents the transformation matrix between the first joint and the end effector position, obtaining posture data. By calculating the transformation matrices of adjacent joints, the posture data that the virtual robot needs to transform is determined, preparing for the subsequent completion of tactile interaction.

[0094] Step 205: Control the virtual robot to perform posture changes according to the posture data, detect whether the virtual robot comes into contact with the virtual environment during the posture change, and when contact occurs, acquire the collision contact position data and the collision virtual attribute data of the contacted virtual environment, use the collision contact position data and the collision virtual attribute data as collision target interaction data, and send the collision target interaction data to the force feedback controller.

[0095] Next, the target interaction data includes collision target interaction data and grasping target interaction data. Based on the above posture data, the virtual robot is controlled to perform posture changes. It is detected whether the virtual robot comes into contact with the virtual environment during the posture change process. The virtual environment includes virtual objects. The virtual robot may come into contact with the virtual environment or virtual objects in the virtual environment during the posture change process. When contact occurs with the virtual environment or virtual objects, collision contact position data and collision virtual attribute data of the contacted virtual environment or virtual object are acquired. The collision contact position data and the collision virtual attribute data are used as collision target interaction data and sent to the force feedback controller. By acquiring the target interaction data of the virtual robot during the posture change process and sending the target interaction data to the force feedback controller, data interaction with the virtual robot is realized, which facilitates the subsequent implementation of tactile interaction.

[0096] Step 206: If the operator presses the grab button while the virtual robot is in contact with the virtual environment, the operator controls the virtual robot to perform a grabbing action, acquires grabbing contact position data and grabbing virtual attribute data, uses the grabbing contact position data and the grabbing virtual attribute data as grabbing target interaction data, and sends the grabbing target interaction data to the force feedback controller; if the operator releases the grabbing button, the operator controls the virtual robot to perform a release action.

[0097] Next, the virtual environment includes virtual objects. When the operator presses the grasp button while the virtual robot is in contact with the virtual object, the grasp function is triggered, controlling the virtual robot to perform a grasping action, acquiring grasping contact position data and grasping virtual attribute data. This grasping contact position data and grasping virtual attribute data are used as grasping target interaction data and sent to the force feedback controller. When the operator releases the button, the release function is triggered, controlling the virtual robot to perform a release action. The grasping virtual attribute data can be acquired by adding a fixed joint component at the contact position. The fixed joint component connects the virtual robot and the virtual object, simulating the grasping effect. Correspondingly, when the release function is triggered, the fixed joint component and the grasping virtual attribute data are deleted, ending the grasping action. The operator controls the virtual robot to perform corresponding posture changes, acquiring the target interaction data during the change and sending it to the force feedback controller. By establishing a fixed joint component, the grasping effect is simulated, improving the realism of the interaction.

[0098] Step 207: The force feedback controller performs a force feedback action based on the target interaction data.

[0099] Next, the force feedback controller executes force feedback actions based on the target interaction data, providing the operator with tactile force perception and realizing tactile interaction with the virtual robot.

[0100] Optionally, in step 201 above, binding the hierarchical relationship of each joint in the virtual 3D model includes: setting the base model as the parent node and setting the first link model as the child node of the base model; according to the preset front-to-back order of the actual connection relationship of the robot, setting each link model as the child node of the previous link model in sequence; setting the sensor model and the front-end tool model as the child nodes of the last link model.

[0101] In the above embodiments of this application, the linkage model includes multiple linkage models, which are distinguished by a preset front-to-back order. The base model is set as the parent node, and the first linkage model is set as the child node of the base model. According to the preset front-to-back order of the actual connection relationship of the robot, each linkage model is set as the child node of the previous linkage model in sequence, and the sensor model and the front-end tool model are set as the child nodes of the last linkage model, thus completing the binding of the hierarchical relationship of each joint point, which facilitates the realization of virtual robot control.

[0102] Optionally, the force feedback action mentioned above includes gravity feedback action, elastic force feedback action, viscous force feedback action, and friction force feedback action; the step 207 above, "the force feedback controller executes the force feedback action according to the target interaction data," includes: the force feedback controller executes the gravity feedback action according to the contact position data and the mass of the contacted virtual object; the force feedback controller executes the elastic force feedback action according to the contact position data and the hardness data of the contacted virtual object; the force feedback controller executes the viscous force feedback action according to the contact position data and the viscosity data of the contacted virtual object; and the force feedback controller executes the friction force feedback action according to the contact position data and the roughness data of the contacted virtual object.

[0103] In the above embodiments of this application, the force feedback controller performs the gravity feedback action based on the contact position data and the mass of the contacted virtual object. For example, it increases the torque downwards based on the contact position data and the mass of the contacted virtual object to simulate the gravity feedback effect when lifting an object. The force feedback controller also performs the elastic force feedback action based on the contact position data and the hardness data of the contacted virtual object. For example, it increases the torque in the contact direction based on the contact position data and the hardness data of the contacted virtual object to simulate the elastic force effect of the object's surface. Based on the viscosity data of the contacted virtual object, the aforementioned viscous force feedback action is executed. For example, based on the contact position data and the viscosity data of the contacted virtual object, torque is increased in the opposite direction of the contact direction to simulate the elastic force effect of the object surface. Based on the contact position data and the roughness data of the contacted virtual object, the aforementioned force feedback controller executes the aforementioned frictional force feedback action. For example, based on the contact position data and the roughness data of the contacted virtual object, torque is increased in the direction perpendicular to the contact direction to simulate the frictional force effect of the object surface. This allows the operator to perceive the tactile effect through the force feedback controller, improving the realism of the interaction with the robot.

[0104] It should be noted that the aforementioned virtual attribute data includes rigid body component parameters, collider component parameters, and surface material component parameters; the rigid body component parameters are read from the rigid body component of the contacting virtual object and are determined by the mass of the contacting virtual object; the collider component parameters are read from the collider component of the contacting virtual object and are determined by the model outline of the contacting virtual object; the surface material component parameters are read from the surface material component of the contacting virtual object and are determined by the hardness, viscosity, and roughness data of the surface of the contacting virtual object.

[0105] In the above embodiments of this application, the rigid body component parameters are determined by the mass of the contacting virtual object and read from the rigid body component of the contacting virtual object; the collider component parameters are determined by the model outline of the contacting virtual object and read from the collider component of the contacting virtual object; the surface material component parameters are determined by the hardness data, viscosity data, and roughness data of the surface of the contacting virtual object.

[0106] By applying the technical solution of this embodiment, a virtual robot is established through a physical robot. Furthermore, by adding rigid body components, collision body components, and surface material components, the virtual robot is endowed with tactile interaction attributes. When the virtual robot's posture changes in real time with tactile control, target interaction data in the virtual environment is acquired, providing force tactile feedback to the operator. This provides a two-way immersive tactile interaction between the operator and the virtual robot, improving the realism of the interaction.

[0107] Furthermore, as Figure 1 In terms of specific implementation of the method, this application provides a robot virtual modeling and tactile control device, such as... Figure 3 As shown, the device includes:

[0108] A module is used to build a virtual robot in a virtual environment based on the robot's physical entity data, and to establish a connection between the force feedback controller and the virtual robot.

[0109] An operation module is used by the operator to operate the force feedback controller and send the end coordinate data of the force feedback controller to the virtual robot;

[0110] An interaction module is used for the virtual robot to calculate posture data based on the end-effector coordinate data, control the virtual robot to perform posture transformation based on the posture data, acquire target interaction data that interacts with the virtual environment during the posture transformation, and send the target interaction data to the force feedback controller.

[0111] The feedback module is used by the force feedback controller to perform force feedback actions based on the target interaction data.

[0112] Optionally, the building module is further used for:

[0113] Based on the robot's model data, mechanical size data, number of joints data, and range of motion data, virtual models of each component of the virtual robot are established. Based on the actual connection relationship of the robot, the virtual models of each component are combined to form a virtual three-dimensional model of the virtual robot, and the hierarchical relationship of each joint in the virtual three-dimensional model is bound.

[0114] A virtual robot is created based on the robot's physical properties and the bound virtual 3D model. The physical properties include mass, model size, and surface material. The virtual models of each component include a base model, a linkage model, a sensor model, and a front-end tool model.

[0115] Optionally, the building module is further used for:

[0116] Set the base model as the parent node and the first link model as the child node of the base model;

[0117] According to the preset front-to-back order of the actual connection relationship of the robot, each link model is set as a child node of the previous link model in sequence;

[0118] Set the sensor model and the front-end tool model as child nodes of the last link model.

[0119] Optionally, the building module is further used for:

[0120] Based on the robot's mass, model size, and surface material, rigid body components, collision body components, and surface material components are built for the bound virtual 3D model to create a virtual robot.

[0121] The rigid body component is determined based on the mass of the robot, the collision body component is determined based on the model dimensions of the robot, and the surface material component is determined based on the surface material properties of the robot.

[0122] Optionally, the interaction module is further configured to:

[0123] Establish the Cartesian coordinate space of the virtual robot, and establish an independent coordinate system for each joint of the virtual robot in the Cartesian coordinate space based on the end-effector coordinate data;

[0124] The transformation matrix of adjacent joints of the virtual robot is determined based on the independent coordinate system of each joint of the virtual robot;

[0125] Based on the transformation matrix of each adjacent joint, the posture information of each joint of the virtual robot is calculated sequentially to obtain posture data.

[0126] Optionally, the interaction module is further configured to:

[0127] The system detects whether the virtual robot comes into contact with the virtual environment during posture transformation. When contact occurs, it acquires collision contact position data and collision virtual attribute data of the contacted virtual environment. The collision contact position data and the collision virtual attribute data are used as collision target interaction data and sent to the force feedback controller. The virtual environment includes virtual objects.

[0128] If the operator presses the grab button while the virtual robot is in contact with the virtual object, the virtual robot is controlled to perform a grabbing action, acquire grabbing contact position data and grabbing virtual attribute data, use the grabbing contact position data and the grabbing virtual attribute data as grabbing target interaction data, and send the grabbing target interaction data to the force feedback controller; if the operator releases the grabbing button, the virtual robot is controlled to perform a release action.

[0129] Optionally, the virtual attribute data includes rigid body component parameters, collider component parameters, and surface material component parameters;

[0130] The rigid body component parameters are read from the rigid body component of the contacting virtual object and are determined by the mass of the contacting virtual object; the collider component parameters are read from the collider component of the contacting virtual object and are determined by the model outline of the contacting virtual object; the surface material component parameters are read from the surface material component of the contacting virtual object and are determined by the hardness data, viscosity data, and roughness data of the surface of the contacting virtual object.

[0131] Optionally, the feedback module is further configured to:

[0132] The force feedback controller executes the gravity feedback action based on the contact position data and the mass of the virtual object being contacted;

[0133] The force feedback controller executes the elastic force feedback action based on the contact position data and the hardness data of the contacted virtual object;

[0134] The force feedback controller executes the viscous force feedback action based on the contact position data and the viscosity data of the contacted virtual object;

[0135] The force feedback controller executes the friction force feedback action based on the contact position data and the roughness data of the contacted virtual object.

[0136] It should be noted that other corresponding descriptions of the functional units involved in the robot virtual modeling and tactile control device provided in this application embodiment can be found in the following references. Figures 1 to 2 The corresponding descriptions in the method will not be repeated here.

[0137] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platform, or it can be implemented by hardware. A virtual robot is accurately established by the physical entity data of the robot. The operator controls the force feedback controller to control the virtual robot to change posture according to the end coordinate data, and obtains the target interaction data with the virtual environment during the posture change. The force feedback controller executes force feedback actions according to the target interaction data, providing the operator with force tactile perception, and realizing the goal of providing two-way immersive tactile interaction between the operator and the virtual robot, thereby improving the realism of interacting with the virtual robot.

[0138] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the modules in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or can be modified to be located in one or more apparatuses different from this embodiment. The modules of the above-described embodiment can be combined into one module, or further divided into multiple sub-modules.

[0139] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of any particular implementation scenario. The above disclosures are merely a few specific implementation scenarios of this application; however, this application is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. A method for virtual modeling and tactile control of a robot, characterized in that, The method includes: A virtual robot is created in a virtual environment based on the robot's physical entity data, and a connection is established between the force feedback controller and the virtual robot. The physical entity data includes model data, mechanical dimension data, number of joints data, and range of motion data. The operator operates the force feedback controller and sends the end coordinate data of the force feedback controller to the virtual robot; The virtual robot calculates posture data based on the end-effector coordinate data, controls the virtual robot to perform posture changes based on the posture data, acquires target interaction data that interacts with the virtual environment during the posture change process, and sends the target interaction data to the force feedback controller. The force feedback controller executes a force feedback action based on the target interaction data; The virtual robot calculates posture data based on the end-effector coordinate data, including: establishing the Cartesian coordinate space of the virtual robot according to a preset creation rule; establishing an independent coordinate system for each joint of the virtual robot in the Cartesian coordinate space using the right-hand rule based on the end-effector coordinate data; and determining the transformation matrix of adjacent joints of the virtual robot based on the independent coordinate system of each joint of the virtual robot to obtain the posture data. The step of establishing a virtual robot in a virtual environment based on the robot's physical entity data includes: establishing virtual models of each component of the virtual robot based on the robot's model data, mechanical size data, number of joints data, and range of motion data; combining the virtual models of each component according to the actual connection relationship of the robot to form a virtual three-dimensional model of the virtual robot; and binding the hierarchical relationship of each joint in the virtual three-dimensional model; establishing the virtual robot based on the robot's physical attributes and the bound virtual three-dimensional model, wherein the physical attributes include mass, model size, and surface material, and the virtual models of each component include a base model, a link model, a sensor model, and a front-end tool model; binding the hierarchical relationship of each joint in the virtual three-dimensional model includes: setting the base model as the parent node and setting the first link model as the child node of the base model; sequentially setting each link model as the child node of the previous link model according to the preset front-to-back order of the robot's actual connection relationship; and setting the sensor model and the front-end tool model as the child nodes of the last link model.

2. The method according to claim 1, characterized in that, The process of establishing a virtual robot based on the robot's physical properties and the bound virtual 3D model includes: Based on the robot's mass, model size, and surface material, rigid body components, collision body components, and surface material components are built for the bound virtual 3D model to create a virtual robot. The rigid body component is determined based on the mass of the robot, the collision body component is determined based on the model dimensions of the robot, and the surface material component is determined based on the surface material properties of the robot.

3. The method according to claim 1, characterized in that, The step of determining the transformation matrix of adjacent joints of the virtual robot based on the independent coordinate system of each joint to obtain the posture data includes: Based on the transformation matrix of each adjacent joint, the posture information of each joint of the virtual robot is calculated sequentially to obtain posture data.

4. The method according to claim 1, characterized in that, The target interaction data includes collision target interaction data and grasping target interaction data. The step of acquiring the target interaction data that interacts with the virtual environment during the posture transformation process and sending the target interaction data to the force feedback controller includes: The system detects whether the virtual robot comes into contact with the virtual environment during posture transformation. When contact occurs, it acquires collision contact position data and collision virtual attribute data of the contacted virtual environment. The collision contact position data and the collision virtual attribute data are used as collision target interaction data and sent to the force feedback controller. The virtual environment includes virtual objects. If the operator presses the grab button while the virtual robot is in contact with the virtual object, the virtual robot is controlled to perform a grabbing action, acquire grabbing contact position data and grabbing virtual attribute data, use the grabbing contact position data and the grabbing virtual attribute data as grabbing target interaction data, and send the grabbing target interaction data to the force feedback controller; if the operator releases the grabbing button, the virtual robot is controlled to perform a release action.

5. The method according to claim 4, characterized in that, The virtual attribute data includes rigid body component parameters, collider component parameters, and surface material component parameters; The rigid body component parameters are read from the rigid body component of the contacting virtual object and are determined by the mass of the contacting virtual object; the collider component parameters are read from the collider component of the contacting virtual object and are determined by the model outline of the contacting virtual object; the surface material component parameters are read from the surface material component of the contacting virtual object and are determined by the hardness data, viscosity data, and roughness data of the surface of the contacting virtual object.

6. The method according to claim 5, characterized in that, The force feedback action includes gravity feedback action, elastic force feedback action, viscous force feedback action, and friction force feedback action. The force feedback controller executes the force feedback action based on the target interaction data, including: The force feedback controller executes the gravity feedback action based on the contact position data and the mass of the virtual object being contacted; The force feedback controller executes the elastic force feedback action based on the contact position data and the hardness data of the contacted virtual object; The force feedback controller executes the viscous force feedback action based on the contact position data and the viscosity data of the contacted virtual object; The force feedback controller executes the friction force feedback action based on the contact position data and the roughness data of the contacted virtual object.

7. A robot virtual modeling and tactile control device, characterized in that, The device includes: The module is used to build a virtual robot in a virtual environment based on the robot's physical entity data, and to establish a connection between the force feedback controller and the virtual robot. The physical entity data includes model data, mechanical dimension data, number of joints data, and range of motion data. An operation module is used by the operator to operate the force feedback controller and send the end coordinate data of the force feedback controller to the virtual robot; An interaction module is used for the virtual robot to calculate posture data based on the end-effector coordinate data, control the virtual robot to perform posture transformation based on the posture data, acquire target interaction data that interacts with the virtual environment during the posture transformation, and send the target interaction data to the force feedback controller. Feedback module, used by the force feedback controller to perform force feedback actions based on the target interaction data; The interaction module is further configured to establish the Cartesian coordinate space of the virtual robot according to preset creation rules, establish independent coordinate systems for each joint of the virtual robot in the Cartesian coordinate space based on the end-effector coordinate data using the right-hand rule, and determine the transformation matrix of adjacent joints of the virtual robot based on the independent coordinate systems of each joint of the virtual robot to obtain the posture data. The building module is further configured to: establish virtual models of each component of the virtual robot based on the robot's model data, mechanical size data, number of joints data, and range of motion data; combine the virtual models of each component according to the actual connection relationship of the robot to form a virtual 3D model of the virtual robot; and bind the hierarchical relationship of each joint point in the virtual 3D model; build the virtual robot based on the robot's physical properties and the bound virtual 3D model, wherein the physical properties include mass, model size, and surface material; and the virtual models of each component include a base model, a link model, a sensor model, and a front-end tool model. The building module is also configured to: set the base model as the parent node and set the first link model as a child node of the base model; sequentially set each link model as a child node of the previous link model according to the preset front-to-back order of the robot's actual connection relationship; and set the sensor model and the front-end tool model as child nodes of the last link model.

Images