Robot grinding teaching method and system based on force feedback and augmented reality

Abstract

The invention relates to a robot grinding teaching method and system based on force feedback and augmented reality, and relates to the technical field of machining, and the method comprises the steps: firstly, building a virtual workpiece model, and obtaining the surface residual height; registering the virtual robot and the virtual workpiece model to a physical scene by using an augmented reality technology; then, a working space mapping model of the force feedback equipment and the virtual robot is constructed, and a control algorithm is introduced to restrain movement of the virtual robot; and in the AR environment, the force feedback model feeds back the polishing force, the simulation model feeds back the polishing effect of the virtual workpiece, and an operator grinds the part based on the experience of people. And finally, in the AR environment, track simulation is carried out based on the operation data, and an available robot machining track is obtained. According to the method, the augmented reality technology and force sense feedback are combined, and the intelligent perception and dynamic decision-making ability of experts are fully integrated, so that the parts can be ground to the specified shape precision, and the grinding precision and efficiency of the complex parts are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mechanical processing, in particular to a robot grinding teaching method and system based on force feedback and augmented reality. BACKGROUND

[0002] Part surface repair refers to repairing parts with surface defects or damage through various methods and technical means to restore their original functions, strength, precision and other performances. The method for repairing damaged parts is generally to perform welding, deposition and other remanufacturing processing on the damaged parts, and then to perform polishing, grinding and other surface processing on the remanufactured parts according to the standard model of the parts to restore the shape accuracy of the parts.

[0003] Industrial robots have been increasingly applied in the industrial automation process due to their high ease of use and high automation level. Robot polishing has the advantages of high flexibility, sufficient operation space and strong operability, and has gradually become one of the common precision machining methods. The common robot polishing trajectory design methods include online teaching method and offline programming method. The online teaching method generally takes a long time and requires operators to have rich teaching experience. The offline programming method generates robot motion trajectories through algorithm design. This method is relatively complex in the face of some parts with complex structures such as impellers.

[0004] It can be seen that the robot grinding system plays a huge role in manufacturing industry, but the teaching or programming method occupies a large amount of time and affects its efficiency in flexible production. SUMMARY

[0005] The present application provides a robot grinding teaching method and system based on force feedback and augmented reality. The simulation result of polishing teaching is superimposed on the real workpiece through the method of augmented reality, and the polishing effect is observed intuitively, which improves the teaching efficiency, reduces the time occupied by teaching, and further improves the efficiency in flexible production. Moreover, the present application introduces force feedback in augmented reality to adaptively adjust key processing parameters and grind the parts to the specified shape accuracy, so that the grinding trajectory is more accurate, and the grinding accuracy and efficiency of complex parts are improved.

[0006] In a first aspect, the present application provides a robot grinding teaching method based on force feedback and augmented reality, comprising:

[0007] performing three-dimensional reconstruction on the part to be processed to obtain a virtual workpiece model, and determining surface residual height information to be ground based on the virtual workpiece model;

[0008] Augmented reality technology is used to register the constructed virtual robot and the virtual workpiece model to the physical scene, thereby constructing an AR environment;

[0009] A workspace mapping model between the force feedback device and the virtual robot is established, and a rate mode is used to control the virtual polishing tool based on the workspace mapping model. A control algorithm is constructed, wherein the virtual polishing tool is a virtual model built based on the polishing tool.

[0010] A force feedback model is established when the virtual polishing tool processes the virtual workpiece model, and a simulation model that updates the surface residual height information in real time is established based on the virtual workpiece model.

[0011] Based on real-time acquisition of operator actions on the force feedback device, in the AR environment, the force feedback model calculates feedback force, provides force feedback through the force feedback device, and the simulation model provides visual feedback, until operation action data that refines the simulation model to a specified shape accuracy is acquired;

[0012] In the AR environment, based on the operation action data and the control algorithm, the virtual robot controls the virtual polishing tool to perform trajectory simulation and trajectory adjustment on the simulation model, thereby obtaining the robot's processing trajectory.

[0013] Optionally, augmented reality (AR) technology is used to register the constructed virtual robot and the virtual workpiece model to the physical scene, thereby constructing an AR environment, including:

[0014] A virtual simulation scene is constructed based on a pre-built virtual robot and the virtual workpiece model, and augmented reality technology is used to register the virtual simulation scene into the physical scene.

[0015] Determine the reference point;

[0016] Establish the transformation between the coordinate system of the reference object and the world coordinate system of the virtual simulation scene, align the virtual robot and the virtual workpiece model with their corresponding entities, and construct an AR environment;

[0017] The virtual robot and the robot are aligned in position, and the virtual workpiece model is aligned in position with the part to be processed.

[0018] Optionally, a transformation is established between the coordinate system of the reference object and the world coordinate system of the virtual simulation scene, and the virtual robot and the virtual workpiece model are aligned with their corresponding entities to construct an AR environment, including:

[0019] Establish the reference coordinate system of the reference object, the base coordinate system of the robot, and the workpiece coordinate system of the part to be processed, respectively;

[0020] The key positions of the reference object are calibrated by the robot, a first origin position of a reference coordinate system is obtained, and a unit vector of a main axis direction of the reference coordinate system is represented in the basic coordinate system to obtain a first main axis direction unit vector;

[0021] Based on the first origin position and the first main axis direction unit vector, a first transformation matrix of the image coordinate system into the basic coordinate system is constructed

[0022] The positions of the selected points of the workpiece to be processed are calibrated by a mechanical arm of the robot, a second origin position of the workpiece coordinate system in the basic coordinate system is obtained, and a unit vector of a main axis direction of the workpiece coordinate system is represented in the basic coordinate system to obtain a second main axis direction unit vector;

[0023] Based on the second origin position and the second main axis direction unit vector, a second transformation matrix of the workpiece coordinate system into the basic coordinate system is constructed

[0024] The first transformation matrix and the second transformation matrix are taken as inputs, and a transformation matrix of the image coordinate system into a world coordinate system of the virtual simulation scene is obtained by combining an image tracking function According to A virtual workpiece transformation matrix is obtained And a robot transformation matrix

[0025] The virtual workpiece model is mapped to the virtual simulation scene by using the virtual workpiece transformation matrix The virtual robot is mapped to the virtual simulation scene by using the robot transformation matrix

[0026] Optionally, a work space mapping model of the force feedback device and the virtual robot is established, and a speed mode is used to control the virtual polishing tool based on the work space mapping model, and a control algorithm is constructed, including:

[0027] A multi-end operation space is established by using a variable scale mapping method based on an operation space, and a space mapping model of the force feedback device and the virtual robot is constructed based on the multi-end operation space;

[0028] A control proportion is determined according to the distance between the virtual polishing tool and the virtual workpiece model, and the corresponding control proportion is set for each end operation space in the work space mapping model;

[0029] ​The rate mode motion constraint of the virtual polishing tool controlled by the virtual robot and the virtual robot end is constructed by the workspace mapping model.

[0030] Optionally, the rate mode motion constraint of the virtual polishing tool controlled by the virtual robot and the virtual robot end through the workspace mapping model comprises:

[0031] According to v d =v n +v t And The rate mode motion constraint is performed to obtain the expected rate v F ;

[0032] Wherein, when the virtual polishing tool is in contact with the mesh surface of the virtual workpiece model, it is decomposed into a collision point normal component v n And a tangential component v t , v d Is the speed of the force feedback device control end mapped into the virtual simulation scene, k i Is the velocity mapping mapping coefficient, d is the down pressure, v F Is the expected rate of the virtual polishing tool, according to the down pressure d to judge whether the polishing tool can move downward along the collision point normal direction.

[0033] Optionally, the force feedback model of the virtual polishing tool processing the virtual workpiece model is established, comprising:

[0034] The contact depth and polishing feedback force of the virtual polishing tool processing the virtual workpiece model are analyzed, and the force feedback model is constructed, which is based on the real-time detected contact depth d, and the polishing feedback force F is calculated according to F=k i ·d;

[0035] Wherein, d is the contact depth of the virtual polishing tool end on the mesh surface, k i Is the polishing force coefficient, and the direction of the polishing feedback force F is the normal direction of the contact point.

[0036] Optionally, a simulation model for real-time updating the surface residual height information is established based on the virtual workpiece model, comprising:

[0037] Detecting the collision information of the grinding surface when the virtual polishing tool processes the virtual workpiece model, and selecting the index of the grid point with the maximum collision depth from the collision information;

[0038] Based on the index of the grid point, the Hertz contact theory is introduced to analyze the contact information of the grid point, which at least includes the length of the long semi-axis of the contact ellipse, the length of the short semi-axis and the center pressure;

[0039] Based on the contact information, the material removal amount is analyzed in combination with the contact position of the grid vertex in the contact ellipse analyzed by the collision information, and a simulation model is constructed;

[0040] The simulation model stores the residual height value of each grid point through a preset data area, and updates the residual height value in real time according to the material removal amount, so as to determine the surface residual height information.

[0041] Optionally, based on the contact information, the material removal amount is analyzed in combination with the contact position of the grid vertex in the contact ellipse analyzed by the collision information, and includes:

[0042] The long semi-axis length a, the short semi-axis length b and the center pressure p0 of the contact ellipse are taken as inputs, and the pressure distribution p in the contact ellipse is calculated according to The pressure distribution p in the contact ellipse is calculated according to

[0043] The distance from all grid points to the maximum collision point is calculated according to the collision information, and the contact position of the grid vertex in the contact ellipse is determined;

[0044] When the contact position is inside the ellipse, the material removal amount mr of the corresponding grid point is calculated according to

[0045] Wherein, x is the x-axis coordinate value of the grid point in the contact ellipse coordinate system, y is the y-axis coordinate value of the grid point in the contact ellipse coordinate system, k p is a proportional coefficient, and v is the relative sliding speed of the current grid point and the virtual polishing tool.

[0046] Optionally, based on the real-time collected operation of the operator on the force feedback device, in the AR environment, the force feedback model calculates the feedback force, and the force feedback device provides force feedback, and the simulation model provides visual feedback, until the operation action data of grinding the simulation model to the specified shape accuracy is collected, including:

[0047] In the process of adjusting the grinding by the operator through the control force feedback device, the operation information of the force feedback device is continuously acquired in real time;

[0048] In the AR environment, the force feedback device end is analyzed from the operation information;

[0049] Based on the force feedback device end, the inverse solution of the virtual robot is calculated through the robot simulation platform to determine the end data of the virtual robot in the grinding process;

[0050] Based on the end data, the virtual robot drives the virtual polishing tool to perform virtual grinding processing on the simulation model.​

[0051] In the virtual grinding process, the simulation model provides visual feedback based on the contact information of the grid surface, the force feedback model calculates feedback force based on the contact depth of the grid surface and provides force feedback through the force feedback device until it is determined that the operator stops grinding, and operation action data is obtained.

[0052] In a second aspect, the application provides a robot grinding teaching system based on force feedback and augmented reality, comprising:

[0053] A three-dimensional reconstruction and identification module is configured to perform three-dimensional reconstruction on a part to be processed to obtain a virtual workpiece model, and determine surface residual height information of a surface to be ground based on the virtual workpiece model.

[0054] An augmented reality environment construction module is configured to register the constructed virtual robot and the virtual workpiece model to a physical scene using augmented reality technology to construct an AR environment.

[0055] A workspace mapping and motion constraint module is configured to establish a workspace mapping model of a force feedback device and the virtual robot, and control a virtual polishing tool in a speed mode based on the workspace mapping model to construct a control algorithm, wherein the virtual polishing tool is a virtual model established based on a polishing tool.

[0056] A force feedback model and simulation model construction module is configured to establish a force feedback model of the virtual polishing tool when processing the virtual workpiece model, and establish a simulation model based on the virtual workpiece model to update the surface residual height information in real time.

[0057] An operation analysis module is configured to, based on real-time collected operation of an operator on the force feedback device, in the AR environment, the force feedback model calculates feedback force, and the force feedback device provides force feedback, and the simulation model provides visual feedback, until operation action data of grinding the simulation model to a specified shape accuracy is collected.

[0058] A processing trajectory generation module is configured to, in the AR environment, based on the operation action data and the control algorithm, the virtual robot controls the virtual polishing tool to perform trajectory simulation and trajectory adjustment on the simulation model to obtain a processing trajectory of the robot.

[0059] In summary, the embodiment of the present application performs three-dimensional reconstruction on the parts to be processed, and obtains the surface residual height to be ground. Then, the virtual robot and the virtual workpiece model are registered to the physical scene by using the augmented reality technology. Subsequently, the working space mapping model and the control algorithm of the force feedback device and the virtual robot are constructed, and the force feedback model of the polishing tool and the workpiece and the real-time simulation model of the polishing residual height are established. Then, based on the real-time polishing force feedback and the polishing residual height visual feedback provided by the system, the key processing parameters are adaptively adjusted to grind the parts to the specified shape accuracy, and the operation action data is determined. In the AR environment, based on the operation action data and the speed constraint, the virtual robot controls the polishing tool to simulate the trajectory, and the operator adjusts the parameters to obtain the available robot processing trajectory. It can be seen that the present application has important significance for improving the production efficiency and ensuring the machining consistency. Specifically, the embodiment of the present application combines the augmented reality technology and the force feedback, and fully integrates the intelligent perception and dynamic decision-making ability of experts, so that the parts can be ground to the specified shape accuracy, the grinding teaching efficiency is effectively improved, and the grinding accuracy and efficiency of complex parts are improved, and the technical problems existing in the existing teaching or programming method are solved. BRIEF DESCRIPTION OF DRAWINGS

[0060] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the application.

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings required to be used in the embodiment or prior art description will be briefly introduced below. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0062] Figure 1 A flowchart of a robot grinding teaching method based on force feedback and augmented reality provided by an embodiment of the present application;

[0063] Figure 2 A step flowchart of a robot grinding teaching method based on force feedback and augmented reality provided by an optional embodiment of the present application;

[0064] Figure 3 A flowchart of a robot grinding teaching method based on force feedback and augmented reality provided by an example of the present application;

[0065] Figure 4 A virtual simulation scene example diagram provided by an example of the present application;

[0066] Figure 5 A calibration point position example diagram in a virtual-real registration method provided by an example of the present application;

[0067] Figure 6 is an example of a working space mapping principle diagram provided by an example of the present application;

[0068] Figure 7 is an example of a contact ellipse geometry between a polishing ball and a workpiece diagram provided by an example of the present application;

[0069] Figure 8 is an example of an operator controlling a force tactile device to adjust key machining parameters to grind a part diagram provided by an example of the present application;

[0070] Figure 9 is a structure block diagram of a robot grinding teaching system based on force feedback and augmented reality provided by an example of the present application. DETAILED DESCRIPTION

[0071] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0072] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0073] In a specific implementation, with the continuous updating and upgrading of computer software and hardware, augmented reality technology (i.e., AR technology) has been applied to various industrial tasks from maintenance services to industrial robot programming. Using augmented reality technology in robot teaching, a teaching system with high sense of presence, simple and safe operation is developed, which not only can save a lot of cost and space, but also can integrate the experience of the operator into the estimation and planning process to improve the efficiency and accuracy of trajectory planning.

[0074] The embodiments of the present application use augmented reality technology to realize a robot grinding teaching method based on force feedback and augmented reality. Before the detailed embodiments of the present application, the related devices used in the embodiments of the present application are exemplarily described:

[0075] The robot used in the present application is an industrial robot, such as an ABB-IRB4600 industrial robot, which performs surface polishing repair on a workpiece with a complex curved surface, and the workpiece is fixed on a workbench. The end tool used in the present application is divided into two parts: a force control unit and a polishing unit. The force control unit uses an ACF active force control unit, which can provide a constant axial force; the polishing unit is composed of a servo motor and a polishing ball head. The AR device used by the operator is a Hololens2 head-mounted display, and the force touch device used is Omega.7.

[0076] The above-mentioned devices can be used to realize robot grinding teaching based on force feedback and augmented reality.

[0077] It should be noted that the above-mentioned devices are only one example, and those skilled in the art can select different models of devices according to actual needs, and the present application does not limit this.

[0078] Figure 1 A flowchart of a robot grinding teaching method based on force feedback and augmented reality provided by the embodiment of the present application, which can specifically include the following steps:

[0079] Step 110, three-dimensional reconstruction is performed on the part to be machined to obtain a virtual workpiece model, and based on the virtual workpiece model, surface residual height information to be ground is determined.

[0080] In the present embodiment, a 3D (three-dimensional) scanning instrument can be used to scan the part to be machined to reconstruct a surface mesh model of the part to be machined as a virtual workpiece model, and the present embodiment does not limit the specific scanning instrument. Then, the surface mesh model of the standard part can be compared with the virtual workpiece model to calculate the residual height of each corresponding mesh point in the virtual workpiece model that needs to be machined, as the surface residual height information to be ground.

[0081] For example, a portable 3D scanning instrument MetraSCAM3D developed by X Company can be used to scan the part to be machined. The 3D scanning instrument uses optical measurement principle to perform reverse reconstruction on the workpiece, and uses the scanning instrument MetraSCAM3D to obtain the surface mesh model of the part to be machined, and obtains the surface mesh model of the standard part. The standard part can be understood as a part obtained after standardizing grinding / polishing machining of the part to be machined.

[0082] Step 120, the constructed virtual robot and the virtual workpiece model are registered to the physical scene using augmented reality technology to construct an AR environment.

[0083] The physical scene can be understood as a real scene.

[0084] In a specific implementation, the embodiment can construct a virtual robot for a real robot. The virtual robot and the virtual workpiece model are both contents of virtual simulation, and the virtual robot and the virtual workpiece model are superimposed on a physical environment (i.e., a physical scene) by using an augmented reality technology, that is, the virtual model is registered to the physical scene by using a virtual augmented technology. Thus, an operator can observe a polishing simulation effect in an AR environment by using an AR device such as AR glasses.

[0085] In step 130, a force feedback model of the virtual polishing tool machining the virtual workpiece model is established, and a simulation model for updating the surface residual height information in real time is established based on the virtual workpiece model.

[0086] The virtual polishing tool is a virtual model established based on a polishing tool.

[0087] In the embodiment, the physical polishing tool can be understood as a robot end tool.

[0088] Considering that the robot controls the physical polishing tool, the polishing tool performs a grinding process on a machining part, and the force feedback device (or force tactile device) introduced in the embodiment, etc., it is generally required to be constrained in a certain workspace and to adopt a velocity mode motion constraint.

[0089] Therefore, the embodiment constructs a workspace mapping model in an AR environment. The workspace mapping model mainly includes three types of workspaces, i.e., a workspace of the force feedback device, a workspace of a motion mapping of a host robot to the robot end tool, and a workspace required for the polishing tool to machine the entire workpiece. The operation range of each device is constrained by the workspace. The size of each workspace can be set according to actual requirements, and the embodiment does not limit this.

[0090] Then, a velocity mode motion constraint is adopted, that is, a workspace mapping velocity mode motion constraint is adopted, and the embodiment establishes a control algorithm to constrain different virtual devices in the AR environment.

[0091] Since each virtual device model in the AR environment corresponds to a real device, a machining trajectory of a robot generated subsequently can constrain the process of the robot controlling the polishing tool and the polishing tool processing the machining part in the physical environment.

[0092] In step 140, a force feedback model of the virtual polishing tool machining the virtual workpiece model is established, and a simulation model for updating the surface residual height information in real time is established based on the virtual workpiece model.

[0093] For most industrial operation tasks, visual feedback and haptic feedback are usually needed to help users perceive the interaction between the robot and the environment, and the haptic feedback can enhance the immersion of the operator in the robot teaching process.

[0094] The embodiment analyzes relevant data such as contact information when the virtual polishing tool processes the virtual workpiece model in the AR environment, establishes a force feedback model, and the force feedback model can provide force haptic feedback in real time when the operator is teaching.

[0095] The embodiment establishes a simulation model (also referred to as a polishing residual height real-time simulation model) that updates surface residual height information in real time, which can provide real-time feedback of the polishing situation, including visual feedback of the polishing residual height, during the teaching process. The operator can adjust key processing parameters according to the force haptic feedback and the visual feedback.

[0096] In step 150, based on the real-time collected operation of the operator on the force feedback device, the force feedback model calculates feedback force in the AR environment, and the force haptic feedback is provided through the force feedback device, and the simulation model provides visual feedback until the operation action data for grinding the simulation model to the specified shape accuracy is collected.

[0097] In a specific implementation, after the operator wears the AR device, the operator performs operations by controlling the force haptic device (i.e., the force feedback device), and these operation data can be transmitted by a communication connection to the position and posture of the end of the force haptic interaction device to control the virtual robot. The virtual robot controls the virtual polishing tool to grind the simulation model. During the grinding process, the force feedback model calculates feedback force in real time and outputs the feedback force to the force haptic device in real time, and the force haptic device provides force haptic feedback, and the simulation model provides visual feedback, and the polishing simulation effect is provided through the visual feedback.

[0098] The operator can observe the polishing simulation effect of the simulation model to continuously adjust the end posture of the virtual polishing tool, and finally grind the simulation model to the specified shape accuracy. When the operator confirms that the operation is completed, complete operation action data is collected at this time.

[0099] In step 160, based on the operation action data and the control algorithm, the virtual robot controls the virtual polishing tool to perform trajectory simulation and trajectory adjustment on the simulation model in the AR environment, and obtains the processing trajectory of the robot.

[0100] In the AR environment, the virtual robot follows the operation action data, analyzes the recorded tool position trajectory, re-processes the recorded tool position trajectory in the AR environment, and is constrained by the control algorithm during the processing. The operator can observe whether collision, singularity and other conditions occur during the robot processing. If no collision or other safety problems occur during the entire trajectory simulation process through AR, the feasibility of the teaching trajectory can be verified in actual polishing. If the above collision or other safety problems occur, the trajectory can be re-taught and adjusted to obtain a usable processing trajectory.

[0101] It can be seen that the embodiment of the application integrates AR and force touch equipment, superimposes the content of the virtual simulation scene into the physical environment through AR, reduces the cognitive load of robot programming, and improves the programming efficiency; the force touch equipment serves as the interface of man-machine cooperation, fully integrates the intelligent perception and dynamic decision-making ability of experts, combines the advantages of high positioning accuracy of robots, and has important significance for improving the grinding accuracy and efficiency of complex parts.

[0102] Reference Figure 2 , a step flowchart of a robot grinding teaching method based on force feedback and augmented reality provided by an optional embodiment of the application is shown. The method can specifically include the following steps:

[0103] Step 210, three-dimensional reconstruction is performed on the part to be processed to obtain a virtual workpiece model, and surface residual height information to be ground is determined based on the virtual workpiece model.

[0104] Step 220, a virtual simulation scene is constructed based on a pre-constructed virtual robot and the virtual workpiece model, and the virtual simulation scene is registered into a physical scene by using augmented reality technology.

[0105] In the related art, the augmented reality technology is less applied to the robot grinding system. The robot grinding system plays a great role in manufacturing industry, but the teaching or programming method occupies a large amount of time, which affects the efficiency in flexible production.

[0106] It can be seen that integrating augmented reality into the robot grinding system and teaching the robot through the force feedback equipment has important significance for improving production efficiency, ensuring processing consistency and improving labor conditions of workers.

[0107] Reference Figure 3The flowchart shows that, in order to apply the augmented reality technology to the robot polishing, the embodiment builds a virtual simulation scene in Unity, builds a virtual robot based on the entity robot in the virtual simulation scene, and adds a virtual workpiece model to the virtual simulation scene. The virtual simulation scene can be integrated with an AR application, such as an AR HMD application developed using MRTK3 and the Vuforia software development kit.

[0108] Then, the virtual model can be registered to the physical scene by using the virtual augmented technology, such as superimposing the content of the virtual simulation scene to the physical environment by AR, as shown in Figure 4 The operator observes the polishing simulation effect in the built AR environment through the AR device.

[0109] Step 230, determining a reference object.

[0110] Step 240, establishing a conversion between the coordinate system of the reference object and the world coordinate system of the virtual simulation scene, aligning the positions of the virtual robot and the virtual workpiece model with the corresponding entities respectively, and building an AR environment.

[0111] The virtual robot is positionally aligned with the robot, and the virtual workpiece model is positionally aligned with the part to be machined.

[0112] In a specific implementation, after the virtual robot and the virtual workpiece model are registered to the physical environment, the virtual robot and the virtual workpiece model need to be positionally aligned with the original entity respectively, so that the virtual robot or the virtual workpiece model can be reasonably superimposed to the position of the original entity.

[0113] The embodiment can introduce a reference object, use the reference object to build a conversion matrix, and realize the position alignment of the virtual model to the entity through coordinate system conversion.

[0114] For example, a paper image of an entity can be used as a reference object, and an image recognition technology can be introduced, which is used as a target image. When Vuforia is used as the image recognition technology, a target QR Marker for image recognition is created by Vuforia, and the generated image tracking target is placed on the workbench. The operator can determine the position of the target image in the virtual scene through the head-mounted display.

[0115] When the position alignment is performed, first, a coordinate system of a target image and a coordinate system of a robot (which can be referred to as a base coordinate system) are respectively established, and a transformation matrix of the coordinate system of the target image to the base coordinate system is analyzed. Then, a transformation matrix of the image coordinate system of the target to a virtual scene world coordinate system is obtained by using the image tracking function of Vuforia. Finally, the two transformation matrices are combined, and a final transformation matrix of the virtual robot to the virtual scene world coordinate system is obtained. The virtual robot can be mapped to the virtual scene world in a manner of position alignment with the physical robot by using the final transformation matrix.

[0116] Similarly, by using the above manner, the final transformation matrix of the virtual workpiece to the virtual scene world coordinate system can be analyzed by calibrating the coordinate system of the workpiece and combining the coordinate system of the robot. The virtual workpiece model can be mapped to the virtual scene world in a manner of position alignment with the workpiece to be machined by using the final transformation matrix.

[0117] Therefore, the embodiment realizes setting of the virtual robot and the virtual workpiece model in the holographic image.

[0118] In an optional embodiment, the above establishing the transformation of the coordinate system of the reference object to the world coordinate system of the virtual simulation scene, aligning the virtual robot and the virtual workpiece model with the corresponding entities in position respectively, and constructing the AR environment include: establishing a reference coordinate system of the reference object, a base coordinate system of the robot, and a workpiece coordinate system of the workpiece to be machined respectively; calibrating key positions of the reference object by the robot to obtain a first origin position of the reference coordinate system and a representation of a unit vector of a main axis direction of the reference coordinate system in the base coordinate system, and obtain a first main axis direction unit vector; based on the first origin position and the first main axis direction unit vector, constructing a first transformation matrix of the image coordinate system to the base coordinate system calibrating the position of the selected point of the workpiece to be machined by the mechanical arm of the robot to obtain a second origin position of the workpiece coordinate system in the base coordinate system and a representation of a unit vector of a main axis direction of the workpiece coordinate system in the base coordinate system, and obtain a second main axis direction unit vector; based on the second origin position and the second main axis direction unit vector, constructing a second transformation matrix of the workpiece coordinate system to the base coordinate system taking the first transformation matrix and the second transformation matrix as inputs, and combining a transformation matrix of the image coordinate system to the world coordinate system of the virtual simulation scene obtained by using the image tracking function According to obtaining a virtual workpiece transformation matrix and a robot transformation matrix wherein the virtual workpiece transformation matrix The virtual workpiece model is mapped to the virtual simulation scene, utilizing the robot transformation matrix. Map virtual robots onto virtual simulation scenes.

[0119] Reference Figure 5 As shown, taking the QR code image as the reference target image as an example, the specific position alignment process is explained below:

[0120] First, establish a coordinate system, including the image coordinate system of the target image (let's assume it's O). M The base coordinate system of the robot (physical robot) (assumed to be O) B ) and the workpiece coordinate system of the part to be processed (assuming it is O) P ).

[0121] To align the virtual robot with the target robot: the robot calibrates the positions of the four corners of the target image as key locations. For example, the robot's end effector can be equipped with a vision sensor to read the three-dimensional coordinates (including X, Y, and Z coordinates) of each of the four corners of the QR code, obtaining the key locations, let's say: P1, P2, P3, and P4. Using the four coordinates of the key locations as input, according to... Calculate coordinate system O M The first origin position P M That is, to determine the origin position P M In the base coordinate system O B The mapped coordinates below, i.e. (P) MX P MY P MZ ).

[0122] Then calculate each coordinate in the base coordinate system O. B The direction vectors of each axis include: X M Y M and Z M This yields the unit vector along the first principal axis. Wherein, according to... Calculate the X-axis direction vector according to Calculate the Y-axis direction vector according to Calculate the Z-axis direction vector Based on the direction vectors of each axis, calculate the direction of each principal axis in the base coordinate system O. B Unit vectors under the following conditions include: based on get based on get based on get

[0123] Finally, using the parameters obtained above, the transformation matrix is ​​obtained.

[0124] The transformation matrix from the image target coordinate system to the virtual scene world coordinate system is obtained through Vuforia's image tracking function. By combining the two transformation matrices, calculate the transformation matrix from the virtual robot to the world coordinate system of the virtual simulation scene. This allows for the mapping of virtual robots onto virtual simulation scenes, achieving positional alignment.

[0125] Similarly, for aligning the virtual workpiece model with the workpiece: use a robotic arm to calibrate the position of selected points on the part to be processed, and obtain the workpiece coordinate system O. P The position of the origin P P That is, the position of the second origin. In the workpiece coordinate system O P Select coordinate positions (P) in each axis direction X ,P Y ,P Z Determine the origin P. P The coordinates of each point in the base coordinate system O B The mapped coordinates below are obtained (P) PX ,P PY ,P PZ ).

[0126] Calculate each coordinate in the base coordinate system O B The direction vectors of each of the lower axes are used to obtain the unit vector of the second principal axis direction. Wherein, according to... Calculate the X-axis direction vector according to Calculate the Y-axis direction vector according to Calculate the Z-axis direction vector Based on the direction vectors of each axis, calculate the direction of each principal axis in the base coordinate system O. B The unit vector below is obtained (For details, please refer to the component calculation process above; this example will not repeat it here).

[0127] Finally, using the parameters obtained above, the transformation matrix is ​​obtained. The transformation matrix from the image target coordinate system to the virtual scene world coordinate system is obtained through Vuforia's image tracking function. By combining the two transformation matrices, calculate the transformation matrix from the virtual workpiece model to the world coordinate system of the virtual simulation scene. This allows for the mapping of virtual workpiece models onto virtual simulation scenes, achieving positional alignment.

[0128] Step 250, a force feedback device and the virtual robot workspace mapping model is established, and the virtual polishing tool is controlled based on the workspace mapping model to build a control algorithm.

[0129] The virtual polishing tool is a virtual model established based on the polishing tool.

[0130] Optionally, the above-mentioned establishment of the force feedback device and the virtual robot workspace mapping model, and the control of the virtual polishing tool based on the workspace mapping model to build a control algorithm can include the following sub-steps:

[0131] Sub-step 2501, a multi-end operation space is established based on an operation space variable scale mapping method, and a space mapping model of the force feedback device and the virtual robot is built based on the multi-end operation space.

[0132] Sub-step 2502, according to the distance between the virtual polishing tool and the virtual workpiece model, the control proportion is determined, and the corresponding control proportion is set for each end operation space in the workspace mapping model.

[0133] Sub-step 2503, the virtual robot and the virtual polishing tool controlled by the virtual robot at the end are subjected to rate mode motion constraint through the workspace mapping model to build a control algorithm.

[0134] The sub-steps 2501-2503 are uniformly described as follows:

[0135] Referring to Figure 6 The embodiment adopts a multi-scale mapping method based on an operation space to build a workspace mapping model. A multi-end operation space (or multi-end workspace) is established by multi-scale mapping, and the multi-end operation space includes: a master end workspace as a workspace of a force touch device; a slave end workspace as a workspace of a master robot workspace motion mapped to a robot end tool; and a target workspace as a workspace required for polishing tool processing of the entire workpiece. The workspace can be used as a spatial constraint for each device

[0136] When establishing the workspace, a variable scale workspace mapping method is adopted to control the robot, and by setting a rate mode control proportion, the proportion of the slave end workspace mapping can be adjusted to adjust the size of the slave end workspace. When the polishing tool does not contact the curved surface to be ground, a larger control proportion is set to cover the entire target workspace, which can quickly adjust the polishing tool to the vicinity of the workpiece; when the polishing tool contacts the curved surface, the control proportion is automatically switched to a given value, which is the same as the size of the master robot workspace, to ensure control accuracy.

[0137] In addition, when the polishing tool contacts the curved surface, the subsequent control algorithm will modify the speed of the end mapping to the end, which will cause the deviation of the end workspace. Therefore, the button input can be set to control the polishing tool position to the position above the nearest polishing point, and if the operator-controlled force tactile device does not have enough movement space, the button can be clicked to reset the mapping range of the workspace.

[0138] In an optional embodiment, the embodiment performs rate mode motion constraint on the virtual polishing tool controlled by the virtual robot and the virtual robot end through the workspace mapping model, which can specifically include: according to v d = v n + v t and the rate mode motion constraint is performed to obtain the expected rate v F ; wherein, when the virtual polishing tool contacts the mesh surface of the virtual workpiece model, it is decomposed into a collision point normal component v n and a tangential component v t , v d is the speed of the end mapping to the virtual simulation scene controlled by the force feedback device, k i is the velocity mapping coefficient, d is the depression amount, v F is the expected rate of the virtual polishing tool, and according to the depression amount d, it is judged whether the polishing tool can move downward along the collision point normal direction.

[0139] In a specific implementation, a control algorithm is introduced, which is a workspace mapping rate mode motion constraint, which introduces a depression force d as a parameter for dynamically adjusting the rate, analyzes the situation of the virtual tool contacting the virtual workpiece model, and thus adjusts the speed of the virtual polishing tool.

[0140] Step 260, establishing a force feedback model when the virtual polishing tool processes the virtual workpiece model, and establishing a simulation model for real-time updating of the surface residual height information based on the virtual workpiece model.

[0141] In an optional embodiment, the above-mentioned establishment of the force feedback model when the virtual polishing tool processes the virtual workpiece model can specifically include: analyzing the contact depth and polishing feedback force when the virtual polishing tool processes the virtual workpiece model, and constructing a force feedback model, the force feedback model is based on the real-time detected contact depth d, and the polishing feedback force F is calculated according to F=k i ·d; wherein, d is the contact depth of the virtual polishing tool end on the mesh surface, k i is the polishing force coefficient, and the direction of the polishing feedback force F is the normal direction of the contact point.

[0142] In the embodiment, when the polishing tool of the entity processes the part to be processed, the contact depth and the polishing feedback force are analyzed and mapped to the polishing processing of the virtual polishing tool on the virtual workpiece model.

[0143] In an optional embodiment, the embodiment establishes a simulation model for real-time updating of the surface residual height information based on the virtual workpiece model, which can include: detecting the collision information of the grinding surface when the virtual polishing tool processes the virtual workpiece model, selecting the index of the grid point with the maximum collision depth from the collision information; based on the index of the grid point, introducing the Hertz contact theory to analyze the contact information of the grid point, the contact information at least including the length of the major axis, the length of the minor axis and the center pressure of the contact ellipse; based on the contact information, analyzing the contact position of the grid vertex in the contact ellipse in combination with the collision information, analyzing the material removal amount, and constructing a simulation model; wherein the simulation model stores the residual height value of each grid point through a preset data area, and updates the residual height value in real time according to the material removal amount to determine the surface residual height information.

[0144] Referring to Figure 7 Fig. 1, when the virtual polishing tool (a polishing ball head as shown in Figure 7 Fig. 1) contacts the virtual workpiece model, the free surface of the virtual workpiece model deforms when it is contacted. By detecting the collision of the virtual polishing tool and the surface to be ground, the index of the grid point with the maximum collision depth is found from the contact area. Since the contact area of the virtual tool and the surface grid of the model forms an ellipse, the Hertz contact theory is introduced to analyze the length of the major axis, the length of the minor axis and the center pressure of the contact ellipse of the grid point. Finally, the material removal amount is calculated, and the simulation model is obtained by updating the original virtual workpiece model.

[0145] In a specific implementation, the collision of the virtual polishing tool and the surface to be ground can be detected by using a GPU. Specifically, based on the GPU method, the distance from all grid points to the center of the polishing tool is calculated, and if the distance is less than the radius of the polishing tool, it means that a collision occurs. The GPU writes the calculation results to the corresponding storage location, and uses atomic operations to find the index of the grid point with the maximum collision depth value.

[0146] The material removal amount of the corresponding grid point inside the contact ellipse is calculated by using the preston formula. First, the collision information is analyzed, the distance from all grid points to the maximum collision point is calculated, and it is judged whether the grid vertex is inside the contact ellipse. If it is inside the contact ellipse, the material removal amount of the corresponding point is calculated by using the preston formula.

[0147] In the process of calculating the material removal amount, the changes of the workpiece model during polishing are recorded in real time by the GPU. The distance from each polishing point (i.e. grid point) to the center of the tool is calculated, and the calculation result is written to the corresponding storage location, i.e. the residual height corresponding to each grid point is stored in the GPU cache. Then, the material removal during polishing is displayed in real time by the surface shader.

[0148] The surface mesh information of the standard part model is extracted, including the mesh vertex coordinates, normal direction, maximum principal curvature A and minimum principal curvature B and direction. The surface residual height to be ground is obtained by comparing the standard part model with the three-dimensionally reconstructed part model.

[0149] It should be noted that the various models established in the above steps, including the virtual tool model, the space mapping model, and various algorithms, can be updated to the AR environment.

[0150] Optionally, the analysis of the material removal amount based on the contact information and the contact position of the grid vertex on the contact ellipse analyzed based on the collision information can include: taking the length of the major axis a, the length of the minor axis b and the center pressure p0 of the contact ellipse as inputs, and calculating the pressure distribution p in the contact ellipse according to , calculating the distance from all grid points to the maximum collision point according to the collision information, determining the contact position of the grid vertex on the contact ellipse, and when the contact position is inside the ellipse, calculating the material removal amount mr of the corresponding grid point according to ; wherein x is the x-axis coordinate value of the grid point in the contact ellipse coordinate system, y is the y-axis coordinate value of the grid point in the contact ellipse coordinate system, k p is a proportional coefficient, and v is the relative sliding speed of the current grid point and the virtual polishing tool. The proportional coefficient is affected by external factors such as material, temperature and shape.

[0151] Referring to Figure 7 , when the polishing tool contacts the workpiece model, the contact wheel contacts the polishing tool at point O under the action of normal force F n . The contact area of the polishing tool is an ellipse, and the coordinate system is OX c Y c Z c , wherein the OX c axis and the OY c axis are the minimum and maximum curvature directions of the workpiece at the polishing contact point O.

[0152] In the collision detection process, the index of the grid point with the maximum collision depth is found, and based on the Hertz contact theory, the length of the major axis of the contact ellipse of the point is calculated according to , and the contact position of the grid vertex on the contact ellipse is determined according to Calculate the length of the minor semi-axis of the contact ellipse at that point, and, according to Calculate the contact center pressure at that point.

[0153] The key parameters involved in the above formulas can be calculated using the following formulas:

[0154] according to and Calculate the total deformation δ of the workpiece and the polishing ball head. * ;according to Calculate the equivalent elastic modulus E of the polishing tool tip and the workpiece. c ;according to Calculate the normal force F n In the above formulas, ε(κ) is a complete elliptic integral of the second kind, ε(κ) is a complete elliptic integral of the first kind, E1 and E2 are the elastic moduli of the solid workpiece and the solid polished ball head, respectively, and v1 and v2 are the Poisson's ratios of the workpiece and the polished ball head, respectively. The elastic modulus and Poisson's ratio can be set in the corresponding virtual model.

[0155] Step 270: Based on real-time acquisition of operator actions on the force feedback device, in the AR environment, the force feedback model calculates the feedback force, provides force feedback through the force feedback device, and the simulation model provides visual feedback, until operational action data that refines the simulation model to the specified shape accuracy is acquired.

[0156] In one optional embodiment, during the operator's adaptive adjustment of the grinding process via the force feedback device, the operator continuously acquires real-time operational information from the force feedback device. In the AR environment, the end effector of the force feedback device is analyzed from the operational information. Based on the end effector, the virtual robot is subjected to inverse kinematics calculations via a robot simulation platform to determine the end effector data of the virtual robot during the grinding process. Based on the end effector data, the virtual robot drives the virtual polishing tool to perform virtual grinding on the simulation model. During the virtual grinding process, the simulation model provides visual feedback based on the contact information of the mesh surface, and the force feedback model calculates the feedback force based on the contact depth of the mesh surface and provides force feedback through the force feedback device until the operator stops grinding, thus obtaining the operational action data.

[0157] Reference Figure 8In actual implementation, the operator adjusts the key machining parameters to grind the part to the specified shape accuracy by adaptively adjusting the force tactile device, transmits the position and posture of the force tactile interaction device end using network communication, and controls the virtual robot according to the data. The PC end uses the unity editor as a server, connects and controls the Omega.7, obtains the operator's hand posture, and calculates the feedback force to provide tactile feedback for the operator. Through the CoppeliaSim robot simulation platform, the inverse solution operation of the robot ABB4600 joint angle is performed, and the PC end and the virtual robot end posture and joint angle data in the AR HMD are synchronized through Socket.

[0158] The AR HMD as a client receives the information sent by the PC end, including the position and posture of the force tactile interaction device end, and sets the joint angle of the virtual robot according to the received information.

[0159] The operator observes the polishing simulation effect and continuously adjusts the polishing tool end posture, finally grinds the part to the specified shape accuracy, and obtains the operation action data.

[0160] Step 280, in the AR environment, based on the operation action data and the control algorithm, the virtual robot controls the virtual polishing tool to perform trajectory simulation and trajectory adjustment on the simulation model, and obtains the machining trajectory of the robot.

[0161] In summary, the embodiment of the application first constructs a model of the processed part, and obtains the surface residual height to be ground based on the model. Then, a virtual robot is constructed based on a physical robot, a virtual simulation scene is constructed by using the virtual robot and the virtual workpiece model, the simulation scene is registered to the physical scene, so as to construct an AR environment, and the position alignment of the virtual model and the entity is realized in the AR environment. Subsequently, a multi-end operation space is established by using a variable scale mapping method, so as to construct a space mapping model, the control proportion of the work space is set, so that each virtual model can move in the corresponding work space, and a control algorithm is introduced to constrain the speed. Next, a force feedback model of the polishing tool and the workpiece is constructed, mainly in the AR environment, the force feedback model when the virtual polishing tool processes the virtual workpiece model is established, and the virtual workpiece model is updated, so as to establish a simulation model for real-time updating of the surface residual height, so that when the operator operates the force tactile device, the force feedback model can feed back the force sense in real time, and the simulation model can feed back the vision in real time. Based on the operation of the operator on the force tactile device, in the AR environment, the virtual robot responds to the operation and drives the virtual polishing tool to virtually grind the simulation model, the operator adjusts the operation according to the real-time visual feedback until the part is ground to the specified shape accuracy, at which time the operation action data is collected. Finally, in the AR environment, the virtual robot controls the virtual polishing tool to perform trajectory simulation on the simulation model based on the operation action data, in the trajectory simulation process, the work space of each virtual device is constrained by the work space mapping model, and the movement speed of each virtual device is constrained by the control algorithm, and the operator can continuously adjust the end of the virtual polishing tool according to the observed polishing simulation effect, and finally the processing trajectory of the robot is obtained.

[0162] It can be seen that the embodiment of the application integrates AR and force tactile device, superimposes the content of the virtual simulation scene into the physical environment through AR, reduces the cognitive load of robot programming, and improves the programming efficiency; the force tactile device serves as the interface of man-machine cooperation, fully integrates the intelligent perception and dynamic decision-making ability of experts, combines the advantages of high positioning accuracy of the robot, and is of great significance to improve the grinding accuracy and efficiency of complex parts.

[0163] It should be noted that, for the method embodiment, in order to simply describe, it is expressed as a series of action combinations, but those skilled in the art should know that the embodiment of the application is not limited by the described action sequence, because according to the embodiment of the application, certain steps can be performed in other order or simultaneously.

[0164] As shown in Figure 9 , the embodiment of the application also provides a robot grinding teaching system based on force feedback and augmented reality 900, which comprises:

[0165] The three-dimensional reconstruction and recognition module 910 is configured to perform three-dimensional reconstruction on the part to be processed, to obtain a virtual workpiece model, and to determine surface residual height information to be ground based on the virtual workpiece model.

[0166] The augmented reality environment construction module 920 is configured to register the constructed virtual robot and the virtual workpiece model to a physical scene by using augmented reality technology, and to construct an AR environment.

[0167] The workspace mapping and motion constraint module 930 is configured to establish a workspace mapping model of the force feedback device and the virtual robot, to control a virtual polishing tool in a speed mode based on the workspace mapping model, and to construct a control algorithm, the virtual polishing tool being a virtual model established based on a polishing tool.

[0168] The force feedback model and simulation model construction module 940 is configured to establish a force feedback model of the virtual polishing tool when processing the virtual workpiece model, and to establish a simulation model based on the virtual workpiece model, the simulation model being capable of updating the surface residual height information in real time.

[0169] The operation analysis module 950 is configured to, based on real-time collected operation of an operator on the force feedback device, calculate a feedback force by the force feedback model in the AR environment, provide force feedback by the force feedback device, and provide visual feedback by the simulation model, until operation action data of grinding the simulation model to a specified shape accuracy is collected.

[0170] The processing trajectory generation module 960 is configured to, based on the operation action data and the control algorithm in the AR environment, control the virtual robot to perform trajectory simulation and trajectory adjustment on the virtual polishing tool to the simulation model, to obtain a processing trajectory of the robot.

[0171] It should be noted that the robot grinding teaching system based on force feedback and augmented reality provided in the embodiments of the present application can perform the robot grinding teaching method based on force feedback and augmented reality provided in any embodiment of the present application, and has the corresponding functions and advantages of the performing method.

[0172] It has to be noted that, in the present document, relational terms are intended only to convey a possible relationship between elements or

[0173] The above description is merely that of a specific implementation of the application and as such is not to be taken in a limiting sense. Various modifications and changes can be made by those skilled in the art to which this application pertains without departing from the spirit and scope of the application as defined by the appended claims. The description is thus to be interpreted in the broadest sense and is intended to include all modifications and equivalents thereof.

Claims

1. A robot dressing teach method based on force feedback and augmented reality, characterized by, The method comprises the following steps: reconstructing a three-dimensional model of a part to be processed to obtain a virtual workpiece model, and determining surface residual height information to be ground based on the virtual workpiece model; registering the constructed virtual robot and the virtual workpiece model to a physical scene by using augmented reality technology to construct an AR environment; establishing a workspace mapping model of a force feedback device and the virtual robot, and controlling a virtual polishing tool in a speed mode based on the workspace mapping model to construct a control algorithm, wherein the virtual polishing tool is a virtual model established based on a polishing tool; establishing a force feedback model of the virtual polishing tool when processing the virtual workpiece model, and establishing a simulation model for updating the surface residual height information in real time based on the virtual workpiece model; based on real-time collected operation of an operator on the force feedback device, the force feedback model calculates feedback force in the AR environment, force sensation feedback is provided through the force feedback device, and visual feedback is provided through the simulation model until operation action data for grinding the simulation model to a specified shape accuracy is collected; based on the operation action data and the control algorithm in the AR environment, the virtual robot controls the virtual polishing tool to perform trajectory simulation and trajectory adjustment on the simulation model to obtain a processing trajectory of the robot.

2. The method of claim 1, wherein, The method comprises the following steps: constructing a virtual simulation scene based on a pre-constructed virtual robot and the virtual workpiece model, and registering the virtual simulation scene to a physical scene by using augmented reality technology; determining a reference object; establishing a conversion between a coordinate system of the reference object and a world coordinate system of the virtual simulation scene, aligning positions of the virtual robot and the virtual workpiece model with corresponding entities respectively, and constructing an AR environment; wherein the virtual robot is aligned with the robot in position, and the virtual workpiece model is aligned with the part to be processed in position.

3. The method of claim 2, wherein, The method comprises the following steps: respectively establishing a reference coordinate system of the reference object, a base coordinate system of the robot, and a workpiece coordinate system of the part to be processed; calibrating a key position of the reference object by the robot to obtain a first origin position of the reference coordinate system and a representation of a unit vector of a main axis direction of the reference coordinate system in the base coordinate system, and obtaining a first main axis direction unit vector; constructing a first transformation matrix of the image coordinate system into the base coordinate system based on the first origin position and the first principal axis direction unit vector calibrating a selected point of the part to be processed by a mechanical arm of the robot to obtain a second origin position of the workpiece coordinate system in the base coordinate system and a representation of a unit vector of a main axis direction of the workpiece coordinate system in the base coordinate system, and obtaining a second main axis direction unit vector; constructing a second transformation matrix of the workpiece coordinate system into the base coordinate system based on the second origin position and the second principal axis direction unit vector inputting the first transformation matrix and the second transformation matrix, and combining a transformation matrix from an image coordinate system obtained by image tracking to a world coordinate system of the virtual simulation scene According to obtaining a virtual workpiece transformation matrix and a robot transformation matrix wherein the virtual workpiece transformation matrix is utilized mapping the virtual workpiece model to the virtual simulation scene, utilizing the robot transformation matrix mapping the virtual robot to the virtual simulation scene.

4. The method of claim 1, wherein, establishing a workspace mapping model of a force feedback device and the virtual robot, and controlling a virtual polishing tool in a speed mode based on the workspace mapping model to construct a control algorithm, comprising: A multi-end operation space is established by using a variable scale mapping method based on an operation space, and a space mapping model of a force feedback device and the virtual robot is constructed based on the multi-end operation space; According to the distance between the virtual polishing tool and the virtual workpiece model, a control ratio is determined, and a corresponding control ratio is set for each end operation space in the work space mapping model; Through the work space mapping model, the virtual robot and the virtual polishing tool controlled by the virtual robot are subjected to rate mode motion constraint, and a control algorithm is constructed.

5. The method of claim 4, wherein, Through the work space mapping model, the virtual robot and the virtual polishing tool controlled by the virtual robot are subjected to rate mode motion constraint, including: According to v d = v n + v t and motion-constrained rate mode, the desired rate v F ; where, when the virtual polishing tool is in contact with the mesh surface of the virtual workpiece model, the decomposition is into a collision point normal component v n and a tangential component v t v d is the speed of the force feedback device control end mapped into the virtual simulation scene, k i is a speed mapping mapping coefficient, d is a down pressure amount, v F is the final output to the virtual polishing tool, and the polishing tool is judged whether it can move downward along the collision point normal direction according to the down pressure amount d.

6. The method of claim 1, wherein, A force feedback model of the virtual polishing tool machining the virtual workpiece model is established, including: analyzing the contact depth and the polishing feedback force when the virtual polishing tool processes the virtual workpiece model, constructing a force feedback model, the force feedback model calculating the polishing feedback force F according to F=k i ·d based on the real-time detected contact depth d; where d is the contact depth of the virtual polishing tool tip on the grid surface, k i is the polishing force coefficient, and the direction of the polishing feedback force F is the normal direction of the contact point.

7. The method of claim 1, wherein, Based on the virtual workpiece model, a simulation model for updating the surface residual height information in real time is established, including: When the virtual polishing tool machines the virtual workpiece model, collision information of the ground surface is detected, and an index of a grid point with the maximum collision depth is selected from the collision information; Based on the index of the grid point, Hertz contact theory is introduced to analyze the contact information of the grid point, and the contact information at least includes the length of the major axis, the length of the minor axis and the center pressure of the contact ellipse; Based on the contact information, the contact position of the grid vertex on the contact ellipse is analyzed in combination with the collision information, the material removal amount is analyzed, and a simulation model is constructed; The simulation model stores the residual height value of each grid point through a preset data area, and updates the residual height value in real time according to the material removal amount to determine the surface residual height information.

8. The method of claim 7, wherein, Based on the contact information, the contact position of the grid vertex on the contact ellipse is analyzed in combination with the collision information, and the material removal amount is analyzed, including: With the length of the major axis a, the length of the minor axis b and the center pressure p0 of the contact ellipse as inputs, the pressure distribution p within the contact ellipse is calculated according to p = p0 * (1 - (x2 / a2 + y2 / b2)1 / 2) According to the collision information, the distance from all grid points to the maximum collision point is calculated to determine the contact position of the grid vertex on the contact ellipse; In the case that the contact position is inside the ellipse, according to calculating the material removal amount mr of the corresponding grid point; wherein x is the x-axis coordinate value of the grid point in the contact ellipse coordinate system, y is the y-axis coordinate value of the grid point in the contact ellipse coordinate system, k p is a proportional coefficient, and v is the relative sliding speed of the current grid point and the virtual polishing tool.

9. The method of claim 1, wherein, Based on the real-time collected operation of the operator on the force feedback device, the force feedback model calculates the feedback force in the AR environment, the force feedback device provides force feedback, and the simulation model provides visual feedback until the operation action data of grinding the simulation model to the specified shape accuracy is collected, including: During the process of the operator adjusting the grinding by controlling the force feedback device, the real-time feedback operation information of the force feedback device is continuously acquired; In the AR environment, the force feedback device end is analyzed from the operation information; Based on the force feedback device end, inverse kinematics of the virtual robot is performed on the robot simulation platform to determine the end data of the virtual robot in the grinding process; Based on the end data, the virtual robot drives the virtual polishing tool to perform virtual grinding processing on the simulation model; In the virtual grinding process, the simulation model provides visual feedback based on the contact information of the grid surface, and the force feedback model calculates the feedback force based on the contact depth of the grid surface and provides force feedback through the force feedback device until it is determined that the operator stops grinding to obtain the operation action data.

10. A force feedback and augmented reality based robot dressing teach system, characterized by, Including: a three-dimensional reconstruction and recognition module, configured to perform three-dimensional reconstruction on a workpiece to be processed to obtain a virtual workpiece model, and determine surface residual height information to be ground based on the virtual workpiece model; an augmented reality environment construction module, configured to register the constructed virtual robot and the virtual workpiece model to a physical scene by using augmented reality technology, and construct an AR environment; a workspace mapping and motion constraint module, configured to establish a workspace mapping model of a force feedback device and the virtual robot, and control a virtual polishing tool in a speed mode based on the workspace mapping model to construct a control algorithm, the virtual polishing tool being a virtual model established based on a polishing tool; a force feedback model and simulation model construction module, configured to establish a force feedback model of the virtual polishing tool when processing the virtual workpiece model, and establish a simulation model based on the virtual workpiece model, the simulation model being capable of updating the surface residual height information in real time; an operation analysis module, configured to, based on real-time collected operation of an operator on the force feedback device, calculate a feedback force by the force feedback model in the AR environment, provide force sense feedback through the force feedback device, and provide visual feedback by the simulation model, until operation action data of grinding the simulation model to a specified shape accuracy is collected; a processing trajectory generation module, configured to, based on the operation action data and the control algorithm in the AR environment, control the virtual robot to perform trajectory simulation and trajectory adjustment on the virtual polishing tool to the simulation model, and obtain a processing trajectory of the robot.

Images