Implementation method of fast binding and real-time animation of robotic arm based on web platform

Abstract

The present invention discloses a method for realizing rapid binding and real-time animation of a mechanical arm based on a web platform, which belongs to the technical field of mechanical arm control technology. In terms of the production and production of mechanical arm animation, the present invention saves huge labor costs on the web platform, is highly compatible with major mainstream graphics engines on the web platform, and uses an analytical solution as the motion algorithm instead of an approximate solution of a traditional iterative method. In comparison, the result accuracy and computational efficiency are greatly improved, making it scalable with rich application scenarios and strong practicality, including but not limited to application scenarios such as reverse control of the working posture of the mechanical arm at the application end. Through the above-mentioned method, the steps of the present invention can all be implemented on the client side of the web platform, and the entire production process can be completed efficiently without professional training, including highly optimized production operation processes and interactive forms and efficient and accurate mechanical arm animation motion algorithms.

Description

Technical Field

[0001] The present invention relates to the technical field of mechanical arm control, and in particular to a method for realizing rapid binding and real-time animation of a mechanical arm based on a web platform. Background Art

[0002] At present, the realization and production methods of robot animation on web platform applications are mostly based on the pre-made bone binding in modeling software, and the pre-made fixed animation actions drive its bones. The general production process is: after retopological modeling of the robot in the modeling software, manually add or adjust the position of the bone anchor point, manually define the weight relationship between the bone and the model vertex, adjust the parameters by adding effectors or controllers based on experience, make the movement restrictions and control methods of the bone, manually make the animation of the controller and bake the action to the bone, import the model skeleton and animation into the web platform, and realize the robot animation by selecting and switching animations.

[0003] For example, Chinese patent CN118269095A discloses a multi-degree-of-freedom control method for a robotic arm based on reinforcement learning. First, the estimated position information and direction posture information of the target hole in the current environment in the world coordinate system are obtained through a visual sensor, and the robotic arm is controlled to move to the vicinity of the central axis above the target hole and align with the target hole according to the obtained information; a depth image is obtained through a depth camera set at the end of the robotic arm, and then the posture and position of the end of the robotic arm are adjusted using a reinforcement learning strategy, and the socket adjustment information for the current state is given; according to the socket adjustment information, the robotic arm socket process is decomposed into multiple sub-actions between the starting end point and the target end point, and the robotic arm completes the socket task by scheduling the visual sensor cycle after executing one sub-action.

[0004] However, in the current production process of robotic arm animation, the steps of adding and adjusting bones, defining weight relationships, making controllers and animations, and exporting and importing animations to the web platform are cumbersome and require professionals to carry out the entire production process, with huge manpower costs. In addition, the produced fixed animations are only applicable to established scenes and established actions, and cannot be changed to specific postures and action states in real time. The animation baking and exporting process involves data compression, which will cause animation jitter and inaccuracy. All of the above disadvantages make it impossible to control the robotic arm on the application side to complete production tasks.

[0005] Based on this, the present invention designs a method for realizing rapid binding and real-time animation of a robotic arm based on a web platform to solve the above problems. Summary of the invention

[0006] In view of the above-mentioned shortcomings of the prior art, the present invention provides a method for realizing rapid binding and real-time animation of a robotic arm based on a web platform.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0008] A method for implementing rapid binding and real-time animation of a robotic arm based on a web platform, the method comprising the following steps:

[0009] Step 1: The six-axis robot arm includes a base, a rotating seat, a lower arm, an upper arm, an upper arm rotating axis, a swing arm and a fixture. Before the user or program operates the robot arm to move, a controller is first defined. The controller directly controls the final posture of the fixture, and the rotation angle of all joints of the robot arm is inferred from the posture.

[0010] Step 2: The controller editing mode is entered through the interactive part. The engine built-in control axis is used through the engine interface. The user operates the control axis to define the position and orientation of one or more points. The interactive part includes a preprocessing module, a classification operation module, a defined axis rotation module and a defined axis restriction module.

[0011] Step 3: After the user defines the required points, they are transformed with respect to the posture of the fixture skeleton, and the maximum and minimum positions of all vertices on each axis after the transformation are taken. Then, an AABB box based on the fixture skeleton posture is generated to achieve efficient physical collision detection, which is used to indicate the current controller posture and facilitate the user to click the controller for operation. Among them, after defining the controller AABB box, the axis point position of the swinging hand should be kept relatively still with the AABB box during the posture transformation. It can be understood that a controller point hidden from the user is automatically defined at the axis point position, and the swinging hand axis point always follows the position of this point during the movement.

[0012] Step 4: When the user uses the motion algorithm in the controller to control the transformation operation of the robot arm, activate a point among the above defined points as the axis, and the robot arm rotates with the axis as the origin. The axis switches when the user clicks, which makes it easy for the user to define complex robot arm movements.

[0013] Furthermore, the preprocessing module includes the following implementation steps:

[0014] Step 1: Guide the user to upload the robot model to be used to the web platform in the form of UI guidance;

[0015] Step 2: Call the web platform to use the default rendering method and rendering engine interface to obtain the point, line, and surface structure information of the robotic arm model;

[0016] Step 3: Search for all models based on point, line and surface information, and summarize the surface IDs of each separated part in the form of a two-dimensional array. The first dimension of the array is the summarized model block, and the second dimension is the surface ID under the model block.

[0017] Furthermore, the classification operation module includes the following implementation steps:

[0018] Step 1: Prepare schematic diagrams and interactive buttons of each component in advance in the UI interface. The user must first click the interactive button to select the component to be classified, and then click the 3D model to classify the selected part;

[0019] Step 2: Use ray detection to let the user pick up the model. The ray collision returns the face ID. When the user performs component classification operations, the model block to which the face ID belongs is mapped to the classification of each component;

[0020] Step 3: After the user clicks the component interactive button, the model rendering method will be changed to the rendering method in the classification mode to change the transparency and color of the model block to distinguish whether the model block is classified or selected;

[0021] Step 4: When the user has finished setting all the categories and decides to end the classification step, if there are unclassified models, the user will be prompted whether to abandon them. The abandoned models will be removed from the rendering layer in the classification mode and will not be rendered;

[0022] Step 5: The hierarchical relationship between the components is defined according to the motion inheritance relationship of the robot arm mechanical structure. The order from parent to subset is: base, rotating seat, lower arm, upper arm, upper arm rotating axis, swing arm and fixture;

[0023] Furthermore, the rendering method in the classification mode includes the following implementation steps:

[0024] Step 1: After the user enters the selection state of a certain component by operating the component interactive button, ray picking, i.e. ray detection operation, is performed at a lower frequency to obtain the vertices of the part of the model pointed by the user's mouse, and the model state is distinguished in the fragment shader by changing the model vertex color, so as to perform differentiated rendering;

[0025] Step 2: Before the user clicks, change the vertex color of the model vertex pointed by the user's mouse, and determine the vertex color difference in the fragment shader, and render these vertices as the color to be selected;

[0026] Step 3: After the user clicks, the vertex color of the model vertex pointed by the user's mouse is changed, and the vertex color difference is determined in the fragment shader, and these vertices are rendered as the selected color;

[0027] Step 4: After the user clicks and confirms the selection multiple times, exit the mode and return to the default rendering method;

[0028] The rendering method in the default state is: pure intrinsic color rendering and PBR rendering are selected according to the support level of the rendering engine, wherein the culled model will be culled, and in the fragment shader, the rendering transparency of the culled model is set to 0, and the transparency clipping is set to a value greater than 0.

[0029] Furthermore, defining the axis rotation module includes the following implementation steps:

[0030] Step 1: After the user clicks the interactive button representing a component, the model rendering method will be changed to the rendering method in the defined rotation axis mode, and other classified model blocks that do not belong to the component will be hidden. The model rendering method in the defined rotation axis mode is as follows: After the user selects a component to define the rotation axis, the model rendering method in the defined rotation axis mode is similar to the default state rendering method, except that the model to be eliminated is different. After the model is eliminated here, the detection pool of the ray detection should be adjusted so that the eliminated model does not participate in the ray detection;

[0031] Step 2: The algorithm for finding the intersection of the three perpendicular lines of a triangle is an axis-finding algorithm. The algorithm is used to find the center of the circumscribed circle of the triangle. The center of the circumscribed circle of the triangle is called the axis. The axis-finding algorithm simultaneously calculates the positions of the three points of the triangle and the current camera orientation, and finds the vector perpendicular to the center of the circle. The vector is the axis.

[0032] Step 3: When the user moves the mouse, the ray detection algorithm will be executed first to obtain the triangle under the current mouse, and the axis center of the triangle will be calculated using the axis center calculation algorithm. The axis center of the triangle can be obtained in real time. After the computer obtains these data, it will continue to calculate the radius of the circle.

[0033] In each logical frame, ray detection is used to return the triangle face hit by the ray, and the axis center and axial direction of the triangle face are obtained using the axis center algorithm. The rendering method is changed to the rendering method of the axis center and axial direction graphically illustrated. Before the user clicks to confirm, the execution and rendering are repeated until the user confirms the axis center and axial direction. Among the various components, except for the base and the fixture, only one set of axis center and axial direction need to be defined, and the remaining components need to define two sets of axis center and axial direction;

[0034] The rendering method of the axis and axial graphic diagram is as follows: pre-make assets, a transparent base the size of a unit circle, and place the center of the disk perpendicular to the disk surface with an arrow pointing outward; after knowing the axis position and vector in the step of determining the axis, then determine the size of the disk by the distance between the axis and a certain point of the triangle, and apply the size, direction and position parameters to the pre-made assets in real time, so that the position of the axis point to be defined can be fed back to the user in real time;

[0035] Step 4: After defining the rotation axis, call the rendering engine interface and place bones that are compatible with the rendering engine.

[0036] Step 5: According to the engine characteristics, in the default posture of the robot arm, select appropriate bone axes as the rotation axis, the orientation axis and the tangential axis. The rotation axis must be oriented toward the axial direction of the axis point, the orientation axis is always oriented toward the point where the axis of the subcomponent is mapped to the axis plane and is perpendicular to the rotation axis, and the tangential axis is perpendicular to the orientation axis and the rotation axis. The tangential axis should be oriented toward the underground direction of the plane where the robot arm is located as much as possible;

[0037] Step 6: Call the engine interface to bind the component classification model in the above steps to the corresponding robotic arm components, so that the skeleton changes the robotic arm posture. Place a skeleton for each defined axis point, place two skeletons for the same component, and define the skeleton hierarchy based on the parent-child hierarchy of the component model and the motion relationship of the mechanical structure.

[0038] Furthermore, defining the axis limit module includes the following implementation steps:

[0039] Step 1: Place all defined axis points in a semi-transparent sphere for interactive judgment, and match the sphere to the axis one by one;

[0040] Step 2: After the user selects one of the axes, a plane with its normal direction in the direction of the axis is generated at the axis point. The plane is used to indicate the axis range set by the user, while weakening the models of other components to reduce information interference of the user;

[0041] Step 3: The user selects one of the axes and moves the mouse. The intersection of the mouse position and the plane is obtained through the ray detection algorithm. The vector of the point relative to the axis is obtained through the intersection position.

[0042] Step 4: On two different component models, use two bones for the two axes. The rotation limit of the axis should be relative to the bone at the axis of its parent component model. The spatial parameters of the axis inherit the parent component model.

[0043] Step 5: Based on the above axis vector, calculate the rotation angle relative to the skeleton, rotate the skeleton by the angle, and provide real-time feedback to the user on the feasibility of the rotation angle;

[0044] Step 6: When the user clicks and holds the mouse, the rotation angle is defined as one of the maximum or minimum angles. The direction of the mouse movement after this moment is recorded, and the axis-limited graphic rendering method is used to render the current graphic representation of the passed angle. When the user releases the mouse, the rotation angle is defined as another point of the maximum or minimum angle. Combined with the movement direction, the two values ​​can be defined as the maximum or minimum angle value;

[0045] The rendering method of the axis limit graphic diagram includes the following specific implementation steps:

[0046] Step 1: Among the pre-made assets, the base needs to be specially made in the design software. First, generate a cylinder with a circumference of about 1 segment;

[0047] Step 2: Then delete the top and bottom of the cylinder, and make UV cuts on the edges facing a certain world axis;

[0048] Step 3: Unfold the UV evenly and scale the UV along the axis to fill all UV space;

[0049] Step 4: Scale the edge of the ring on the cylinder to a point, do not merge the points;

[0050] Step 5: Scale the cylinder into a circle and center it at the origin;

[0051] Step 6: After the asset is created, in the fragment shader, a certain UV axis is passed. The axis depends on the state of the cylinder after UV expansion. The rendering of a certain interval value is changed to indicate the user's angle state. The rendering interval value is input by the program instruction. Two floating-point numbers and a Boolean value of whether to reverse are obtained. The rendering interval is converted from the angle, and the Boolean value is determined by the above-mentioned user mouse movement.

[0052] Furthermore, the motion algorithm includes the following implementation steps:

[0053] Step 1: Pre-calculate the distance between each axis point, and then subtract the starting point coordinate from the end point coordinate to get the modulus;

[0054] Step 2: Take the axis point of the swinging hand as the target and make the rotating seat face this target;

[0055] Step 3: Take the pivot point of the swinging hand as the target and the pivot point of the lower arm as the starting point, and use the simplified TwoBoneIK algorithm to calculate the rotation angle between the upper arm and the upper arm rotation axis;

[0056] Step 4: Calculate the rotation angle of the upper arm rotation axis and the swinging hand according to the fixture posture and the lower direction of the fixture;

[0057] Step 5: The axis point position of the swinging hand is the relative position of the fixture, and the axis point position of the swinging hand moves with the position of the fixture.

[0058] Furthermore, the axis point of the swinging hand is taken as the target, and the rotating seat is directed toward this target. The specific calculation method is:

[0059] Step 1: Get the three-axis vectors of the rotating seat and correspond them to the world coordinates in the default posture;

[0060] Step 2: Multiply the world position of the target point by the three-axis vector of the rotating seat;

[0061] Step 3: Remove the z vector from the dot product result, take the square root of the sum of the squares of the x and y values, and get the rotation angle of the rotating seat;

[0062] Step 4: Determine whether the rotation angle is between the maximum angle and the minimum angle defined above, and define the value between the maximum angle and the minimum angle as the allowable angle. If it is between the allowable angles, rotate the angle so that the rotating seat faces the target point. If it is not between the allowable angles, the rotating seat does not move.

[0063] Furthermore, taking the pivot point of the swinging hand as the target and the pivot point of the lower arm as the starting point, the simplified TwoBoneIK algorithm is used to obtain the rotation angle between the upper arm and the upper arm rotation axis. The specific calculation method is:

[0064] Step 1: Use the TwoBoneIK algorithm to calculate the distance between the target and the starting point;

[0065] Step 2: Substitute the known length of the upper arm bone, the length of the upper arm rotation axis bone, and the distance between the target and the starting point into the cosine function to obtain the angles of each angle;

[0066] Step 3: Apply the angle to the lower arm and upper arm. The angle of the lower arm is based on the angle from the current axis starting point to the target point. When applying the angle, determine whether it is within the allowed angle. If the angle is not allowed, use the maximum allowed angle.

[0067] Furthermore, the rotation angle of the upper arm rotation axis and the swinging hand is calculated according to the fixture posture and the lower direction of the fixture. The specific calculation method is:

[0068] Step 1: The lower direction of the fixture is known in the above step of determining the skeleton, and the position of the fixture moving a small distance in the lower direction is calculated according to the fixture posture;

[0069] Step 2: Calculate the position of the tiny vector relative to the upper arm rotation axis plane according to the upper arm bone posture, and then use the dot multiplication algorithm to obtain it;

[0070] Step 3: After obtaining the position of the small vector relative to the upper arm rotation axis plane, calculate the angle required to rotate from the lower part of the upper arm rotation axis toward the relative plane position;

[0071] Step 4: Use the dot multiplication algorithm to find the projection position of the point on the waving hand according to the position of the fixture, and find the required rotation angle;

[0072] Step 5: Determine whether the angle is within the allowed angle. If it is within the allowed angle, the hand will rotate to that angle. If it is not within the allowed angle, the hand will not move.

[0073] Compared with the prior art, the present invention has the following beneficial effects:

[0074] In terms of the production of robotic arm animation, the present invention saves huge manpower costs on the web platform, is highly compatible with major mainstream graphics engines on the web platform, and the motion algorithm is an analytical solution rather than an approximate solution of the traditional iterative method. In comparison, the result accuracy and calculation efficiency are greatly improved, making its expandable application scenarios rich and practical, including but not limited to application scenarios such as reverse control of the working posture of the robotic arm at the application end. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0076] Figure 1 A three-dimensional image of a rendered model of the robotic arm of the present invention;

[0077] Figure 2 A split diagram of the robot arm rendering model of the present invention;

[0078] Figure 3 The lower arm screen of the present invention displays the picture;

[0079] Figure 4 It is the lower arm program space screen of the present invention;

[0080] Figure 5 A screen for drawing graphics for the lower arm program of the present invention.

[0081] The numbers in the figure represent:

[0082] 1. Base 2. Rotating seat 3. Lower arm 4. Upper arm 5. Upper arm rotating axis 6. Swinging hand 7. Clamp. DETAILED DESCRIPTION

[0083] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0084] Embodiment 1

[0085] In some embodiments, please refer to the attached instructions. Figure 1-5, a method for realizing rapid binding and real-time animation of a robotic arm based on a web platform, the method comprising the following steps:

[0086] Step 1: If Figure 1 As shown, the six-axis robot arm includes a base 1, a rotating base 2, a lower arm 3, an upper arm 4, an upper arm rotating shaft 5, a swing hand 6 and a clamp 7. The upper end of the base 1 is rotatably connected to the rotating base 2, the lower arm 3 is hinged to the rotating base 2, the lower arm 3 is hinged to the upper arm 4, the upper arm 4 is hinged to the upper arm rotating shaft 5, the upper arm rotating shaft 5 is hinged to the swing hand 6, and the swing hand 6 is hinged to the clamp 7;

[0087] Before the user or program operates the robot arm to move, a controller is first defined. The controller directly controls the final posture of the gripper 7, and uses the posture to reversely infer the rotation angles of all joints of the robot arm.

[0088] Step 2: The controller editing mode is entered through the interactive part. The engine built-in control axis is used through the engine interface. The user operates the control axis to define the position and orientation of one or more points. The interactive part includes a preprocessing module, a classification operation module, a defined axis rotation module and a defined axis restriction module.

[0089] Step 3: After the user defines the required points, they are transformed relative to the posture of the fixture 7 skeleton, and the maximum and minimum positions of all vertices on each axis after the transformation are taken. Then, an AABB box based on the posture of the fixture 7 skeleton is generated to achieve efficient physical collision detection. The AABB box is a simple hexahedron, each side of which is parallel to a coordinate plane. The rectangular bounding box is not necessarily a cube, and its length, width, and height can be different from each other. It is used to indicate the current controller posture and to facilitate the user to click the controller to operate. Among them, the axis point position of the waving hand 6, after defining the controller AABB box, the axis point position and the AABB box must remain relatively still during the posture transformation. It can be understood that a controller point hidden from the user is automatically defined at the axis point position, and the axis point of the waving hand 6 always follows the position of this point during movement;

[0090] Step 4: When the user uses the motion algorithm in the controller to control the transformation operation of the robot arm, activate a point among the above defined points as the axis, and the robot arm rotates with the axis as the origin. The axis switches when the user clicks, which makes it easy for the user to define complex robot arm movements.

[0091] Embodiment 2

[0092] In some embodiments, Figure 1-5 As shown, as a preferred embodiment of the present invention, the preprocessing module includes the following implementation steps:

[0093] Step 1: Guide the user to upload the robot model to be used to the web platform in the form of UI guidance;

[0094] Step 2: Call the web platform to use the default rendering method and rendering engine interface to obtain the point, line, and surface structure information of the robotic arm model;

[0095] Step 3: Search for all models based on point, line and surface information, and summarize the surface IDs of each separated part in the form of a two-dimensional array. The first dimension of the array is the summarized model block, and the second dimension is the surface ID under the model block.

[0096] The classification operation module includes the following implementation steps:

[0097] Step 1: Prepare schematic diagrams and interactive buttons of each component in advance in the UI interface. The user must first click the interactive button to select the component to be classified, and then click the 3D model to classify the selected part;

[0098] Step 2: Use ray detection to let the user pick up the model. The ray collision returns the face ID. When the user performs component classification operations, the model block to which the face ID belongs is mapped to the classification of each component. Ray detection is a general technology. Its significance is to detect whether a straight line starting from a point and heading in a certain direction touches a certain part of the model, such as Figure 3 , Figure 4 As shown, the ray detection in the present invention specifically refers to emitting rays from the user's perspective toward the direction of the mouse to detect whether a certain part of the model is touched, which is ultimately manifested as detecting that the user clicks the model on the screen with the mouse;

[0099] Step 3: After the user clicks the component interactive button, the model rendering method will be changed to the rendering method in the classification mode to change the transparency and color of the model block to distinguish whether the model block is classified or selected;

[0100] Step 4: When the user has finished setting all the categories and decides to end the classification step, if there are unclassified models, the user will be prompted whether to abandon them. The abandoned models will be removed from the rendering layer in the classification mode and will not be rendered;

[0101] Step 5: The hierarchical relationship between the components is defined according to the motion inheritance relationship of the mechanical structure of the robot arm, and the order from parent to subset is: base 1, rotating seat 2, lower arm 3, upper arm 4, upper arm rotating axis 5, swing hand 6 and fixture 7;

[0102] The rendering method in the classification mode includes the following implementation steps:

[0103] Step 1: After the user enters the selection state of a certain component by operating the component interactive button, ray picking, i.e. ray detection operation, is performed at a lower frequency to obtain the vertices of the part of the model pointed by the user's mouse, and the model state is distinguished in the fragment shader by changing the model vertex color, so as to perform differentiated rendering;

[0104] Step 2: Before the user clicks, change the vertex color of the model vertex pointed by the user's mouse, and determine the vertex color difference in the fragment shader, and render these vertices as the color to be selected;

[0105] Step 3: After the user clicks, the vertex color of the model vertex pointed by the user's mouse is changed, and the vertex color difference is determined in the fragment shader, and these vertices are rendered as the selected color;

[0106] Step 4: After the user clicks and confirms the selection multiple times, exit the mode and return to the default rendering method;

[0107] The rendering method in the default state is: pure intrinsic color rendering and PBR rendering are selected according to the support level of the rendering engine, wherein the culled model will be culled, and in the fragment shader, the rendering transparency of the culled model is set to 0, and the transparency clipping is set to a value greater than 0.

[0108] Defining the axis rotation module includes the following implementation steps:

[0109] Step 1: After the user clicks the interactive button representing a component, the model rendering method will be changed to the rendering method in the defined rotation axis mode, and other classified model blocks that do not belong to the component will be hidden. The model rendering method in the defined rotation axis mode is as follows: After the user selects a component to define the rotation axis, the model rendering method in the defined rotation axis mode is similar to the default state rendering method, except that the model to be eliminated is different. After the model is eliminated here, the detection pool of the ray detection should be adjusted so that the eliminated model does not participate in the ray detection;

[0110] Step 2: The algorithm for finding the intersection of the three perpendicular lines of a triangle is an axis-finding algorithm. The algorithm is used to find the center of the circumscribed circle of the triangle. The center of the circumscribed circle of the triangle is called the axis. The axis-finding algorithm simultaneously calculates the positions of the three points of the triangle and the current camera orientation, and finds the vector perpendicular to the center of the circle. The vector is the axis.

[0111] Step 3: When the user moves the mouse, the ray detection algorithm will be executed first to obtain the triangle under the current mouse, and the axis center of the triangle will be calculated using the axis center calculation algorithm. The axis center of the triangle can be obtained in real time. After the computer obtains these data, it will continue to calculate the radius of the circle.

[0112] like Figure 5As shown, in each logical frame, ray detection is used to return the triangular face hit by the ray, and the axis center and axial direction of the triangular face are obtained using the axis center algorithm. The rendering method is changed to the rendering method illustrated by the axis center and axial direction graphics. Before the user clicks to confirm, the execution and rendering are repeated until the user confirms the axis center and axial direction. Among the various components, except for the base 1 and the fixture 7, only one set of axis center and axial direction need to be defined, and the remaining components need to define two sets of axis center and axial direction;

[0113] The rendering method of the axis and axial graphic diagram is as follows: pre-make assets, a transparent base the size of a unit circle, and place the center of the disk perpendicular to the disk surface with an arrow pointing outward; after knowing the axis position and vector in the step of determining the axis, then determine the size of the disk by the distance between the axis and a certain point of the triangle, and apply the size, direction and position parameters to the pre-made assets in real time, so that the position of the axis point to be defined can be fed back to the user in real time;

[0114] Step 4: After defining the rotation axis, call the rendering engine interface and place the bones compatible with the rendering engine. The bones in the rendering engine represent a coordinate system for calculating the spatial position transformation.

[0115] Step 5: According to the engine characteristics, in the default posture of the robot arm, select appropriate bone axes as the rotation axis, the orientation axis and the tangential axis. The rotation axis must be oriented toward the axial direction of the axis point, the orientation axis is always oriented toward the point where the axis of the subcomponent is mapped to the axis plane and is perpendicular to the rotation axis, and the tangential axis is perpendicular to the orientation axis and the rotation axis. The tangential axis should be oriented toward the underground direction of the plane where the robot arm is located as much as possible;

[0116] like Figure 2 As shown, the figure shows the positions of the axis points of each axis of the robot arm in principle, and the lines starting from base 1 represent the inheritance relationship between these axis points, that is, the parent-child relationship.

[0117] Step 6: Call the engine interface to bind the component classification model in the above steps to the corresponding robotic arm components, so that the skeleton changes the robotic arm posture. Place a skeleton for each defined axis point, place two skeletons for the same component, and define the skeleton hierarchy based on the parent-child hierarchy of the component model and the motion relationship of the mechanical structure.

[0118] Defining an axis limit module includes the following implementation steps:

[0119] Step 1: Place all defined axis points in a semi-transparent sphere for interactive judgment, and match the sphere to the axis one by one;

[0120] Step 2: After the user selects one of the axes, a plane with its normal direction in the direction of the axis is generated at the axis point. The plane is used to indicate the axis range set by the user, while weakening the models of other components to reduce information interference of the user;

[0121] Step 3: The user selects one of the axes and moves the mouse. The intersection of the mouse position and the plane is obtained through the ray detection algorithm. The vector of the point relative to the axis is obtained through the intersection position.

[0122] Step 4: On two different component models, use two bones for the two axes. The rotation limit of the axis should be relative to the bone at the axis of its parent component model. The spatial parameters of the axis inherit the parent component model.

[0123] Step 5: Based on the above axis vector, calculate the rotation angle relative to the skeleton, rotate the skeleton by the angle, and provide real-time feedback to the user on the feasibility of the rotation angle;

[0124] Step 6: When the user clicks and holds the mouse, the rotation angle is defined as one of the maximum or minimum angles. The direction of the mouse movement after this moment is recorded, and the axis-limited graphic rendering method is used to render the current graphic representation of the passed angle. When the user releases the mouse, the rotation angle is defined as another point of the maximum or minimum angle. Combined with the movement direction, the two values ​​can be defined as the maximum or minimum angle value;

[0125] The rendering method of the axis limit graphic diagram includes the following specific implementation steps:

[0126] Step 1: Among the pre-made assets, the base 1 needs to be specially made in the design software. First, a cylinder with a circumference of about 64 segments is generated;

[0127] Step 2: Then delete the top and bottom of the cylinder, and make UV cuts on the edges facing a certain world axis;

[0128] Step 3: Unfold the UV evenly and scale the UV along the axis to fill all UV space;

[0129] Step 4: Scale the edge of the ring on the cylinder to a point, do not merge the points;

[0130] Step 5: Scale the cylinder into a circle and center it at the origin;

[0131] Step 6: After the asset is created, in the fragment shader, a certain UV axis is passed. The axis depends on the state of the cylinder after UV expansion. The rendering of a certain interval value is changed to indicate the user's angle state. The rendering interval value is input by the program instruction with two floating-point numbers from 0 to 1 and a Boolean value indicating whether it is reversed. The rendering interval is converted from the angle, and the Boolean value is determined by the above-mentioned user mouse movement.

[0132] Embodiment 3

[0133] In some embodiments, Figure 1-5 As shown, as a preferred embodiment of the present invention, the motion algorithm includes the following implementation steps:

[0134] Step 1: Pre-calculate the distance between each axis point, and then subtract the starting point coordinate from the end point coordinate to get the modulus;

[0135] Step 2: Take the axis point of the swinging hand 6 as the target and make the rotating seat 2 face this target;

[0136] Step 3: Take the pivot point of the swinging hand 6 as the target and the pivot point of the lower arm 3 as the starting point, and use the simplified TwoBoneIK algorithm. The TwoBoneIK algorithm is a method of applying IK to a 3-joint chain using a Two Bone IK controller, and then calculate the rotation angle between the upper arm 4 and the upper arm rotation axis 5;

[0137] Step 4: Calculate the rotation angles of the upper arm rotation axis 5 and the swinging hand 6 according to the position of the fixture 7 and the lower direction of the fixture 7;

[0138] Step 5: The axis point position of the swinging hand 6 is the relative position of the fixture 7, and the axis point position of the swinging hand 6 moves with the posture of the fixture 7.

[0139] Take the axis point of the swing hand 6 as the target and make the rotating seat 2 face this target. The specific calculation method is:

[0140] Step 1: Get the three-axis vectors of the rotating seat 2, which correspond to the world coordinates in the default posture. The default posture is the default posture when the system creates the corresponding skeleton;

[0141] Step 2: Multiply the world position of the target point by the three-axis vector of the rotation seat 2;

[0142] Step 3: Remove the z vector from the dot product result, take the square root of the sum of the squares of the x and y values, and get the rotation angle of the rotating seat 2;

[0143] Step 4: Determine whether the rotation angle is between the maximum angle and the minimum angle defined above, and define the value between the maximum angle and the minimum angle as the allowable angle. If it is between the allowable angles, rotate the angle so that the rotating seat 2 faces the target point. If it is not between the allowable angles, the rotating seat 2 does not move.

[0144] Taking the pivot point of the swinging hand 6 as the target and the pivot point of the lower arm 3 as the starting point, the simplified TwoBoneIK algorithm is used to obtain the rotation angle between the upper arm 4 and the upper arm rotation axis 5. The specific calculation method is:

[0145] Step 1: Use the TwoBoneIK algorithm to calculate the distance between the target and the starting point;

[0146] Step 2: Substitute the known length of the upper arm 4 bone, the length of the upper arm rotation axis 5 bone, and the distance between the target and the starting point into the cosine function to obtain the angle of each angle;

[0147] Step 3: Apply the angle to lower arm 3 and upper arm 4. The angle of lower arm 3 is applied based on the angle from the current axis starting point to the target point. When applying the angle, determine whether it is within the allowed angle. If the angle is not allowed, use the maximum allowed angle.

[0148] The rotation angles of the upper arm rotation axis 5 and the swinging hand 6 are calculated according to the position and lower direction of the fixture 7. The specific calculation method is:

[0149] Step 1: The lower direction of the fixture 7 is known in the above-mentioned step of determining the skeleton, and the position of the fixture 7 moving a small distance in the lower direction is calculated according to the posture of the fixture 7;

[0150] Step 2: Calculate the position of the tiny vector relative to the plane of the upper arm rotation axis 5 according to the upper arm skeletal posture 4, and then use the dot multiplication algorithm to obtain it;

[0151] Step 3: After obtaining the position of the small vector relative to the plane of the upper arm rotation axis 5, calculate the angle required to rotate from the lower part of the upper arm rotation axis 5 toward the relative plane position;

[0152] Step 4: Use the dot multiplication algorithm to find the projection position of the point on the swinging hand 6 according to the position of the fixture 7, and find the required rotation angle;

[0153] Step 5: Determine whether the angle is within the allowed angle. If it is within the allowed angle, the hand waving 6 rotates to the angle. If it is not within the allowed angle, the hand waving 6 does not move.

[0154] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for implementing rapid binding and real-time animation of a robotic arm based on a web platform, characterized in that: The method comprises the following steps: Step 1: The six-axis robotic arm comprises a base (1), a rotating base (2), a lower arm (3), an upper arm (4), an upper arm rotating shaft (5), a swing arm (6) and a clamp (7). Before a user or a program operates the movement of the robotic arm, a controller is first defined. The controller directly controls the final posture of the clamp (7), and the rotation angles of all joints of the robotic arm are inferred from the final posture. Step 2: The controller editing mode is entered through the interactive part. The engine built-in control axis is used through the engine interface. The user operates the control axis to define the position and orientation of one or more points. The interactive part includes a preprocessing module, a classification operation module, a defined axis rotation module and a defined axis restriction module. Step 3: After the user defines the required point, it is transformed with respect to the posture of the skeleton of the fixture (7), and the maximum and minimum positions of all vertices on each axis after the transformation are taken. Then, an AABB box based on the skeleton posture of the fixture (7) is generated to achieve efficient physical collision detection, so as to indicate the current controller posture and facilitate the user to click the controller to operate. Among them, after defining the controller AABB box, the axis point position of the waving hand (6) should be kept relatively still with the AABB box when the posture is transformed. It can be understood that a controller point hidden from the user is automatically defined at the axis point position, and the axis point of the waving hand (6) always follows the position of this point during movement; Step 4: When the user uses the motion algorithm in the controller to control the transformation operation of the robot arm, he activates a point among the above defined points as the axis, and the robot arm rotates with the axis as the origin. The axis switches when the user clicks, which makes it easier for the user to define complex robot arm movements; The motion algorithm includes the following implementation steps: Step 1: Pre-calculate the distance between each axis point, and then subtract the starting point coordinate from the end point coordinate to get the modulus; Step 2: Taking the axis point of the swinging hand (6) as the target, make the rotating seat (2) face this target; Step 3: Using the pivot point of the swinging hand (6) as the target and the pivot point of the lower arm (3) as the starting point, use the simplified TwoBoneIK algorithm to calculate the rotation angle between the upper arm (4) and the upper arm rotation axis (5); Step 4: Calculate the rotation angles of the upper arm rotation axis (5) and the swinging hand (6) based on the position of the fixture (7) and the lower direction of the fixture (7); Step 5: The axis point position of the swinging hand (6) is the relative position of the fixture (7), and the axis point position of the swinging hand (6) moves with the posture of the fixture (7).

2. The method for realizing rapid binding and real-time animation of a robotic arm based on a web platform according to claim 1, characterized in that: The preprocessing module includes the following implementation steps: Step 1: Guide the user to upload the robot model to be used to the web platform in the form of UI guidance; Step 2: Call the web platform to use the default rendering method and rendering engine interface to obtain the point, line, and surface structure information of the robotic arm model; Step 3: Search for all models based on point, line and surface information, and summarize the surface IDs of each separated part in the form of a two-dimensional array. The first dimension of the array is the summarized model block, and the second dimension is the surface ID under the model block.

3. The method for realizing rapid binding and real-time animation of a robotic arm based on a web platform according to claim 2 is characterized in that: The classification operation module includes the following implementation steps: Step 1: Prepare schematic diagrams and interactive buttons of each component in advance in the UI interface. The user must first click the interactive button to select the component to be classified, and then click the 3D model to classify the selected part; Step 2: Use ray detection to let the user pick up the model. The ray collision returns the face ID. When the user performs the component classification operation, the model block to which the face ID belongs is mapped to the classification of each component; Step 3: After the user clicks the component interactive button, the model rendering method will be changed to the rendering method in the classification mode to change the transparency and color of the model block to distinguish whether the model block is classified or selected; Step 4: When the user has finished setting all the categories and decides to end the classification step, if there are unclassified models, the user will be prompted whether to abandon them. The abandoned models will be removed from the rendering layer in the classification mode and will not be rendered; Step 5: The hierarchical relationship between the components is defined according to the motion inheritance relationship of the robot arm's mechanical structure. The order from parent to subset is: base (1), rotating base (2), lower arm (3), upper arm (4), upper arm rotating axis (5), swing arm (6) and fixture (7).

4. The method for realizing rapid binding and real-time animation of a robotic arm based on a web platform according to claim 3 is characterized in that: The rendering method in the classification mode includes the following implementation steps: Step 1: After the user enters the selection state of a certain component by operating the component interactive button, ray picking, i.e. ray detection operation, is performed at a lower frequency to obtain the vertices of the part of the model pointed by the user's mouse, and the model state is distinguished in the fragment shader by changing the model vertex color, so as to perform differentiated rendering; Step 2: Before the user clicks, change the vertex color of the model vertex pointed by the user's mouse, and determine the vertex color difference in the fragment shader, and render these vertices as the color to be selected; Step 3: After the user clicks, the vertex color of the model vertex pointed by the user's mouse is changed, and the vertex color difference is determined in the fragment shader, and these vertices are rendered as the selected color; Step 4: After the user clicks and confirms the selection multiple times, the rendering method in the classification mode is exited and the rendering method in the default state is returned; The rendering method in the default state is: pure intrinsic color rendering and PBR rendering are selected according to the support level of the rendering engine, wherein the culled model will be culled, and in the fragment shader, the rendering transparency of the culled model is set to 0, and the transparency clipping is set to a value greater than 0.

5. The method for realizing rapid binding and real-time animation of a robotic arm based on a web platform according to claim 4 is characterized in that: Defining the axis rotation module includes the following implementation steps: Step 1: After the user clicks the interactive button representing a component, the model rendering method will be changed to the rendering method in the defined rotation axis mode, and other classified model blocks that do not belong to the component will be hidden; the model rendering method in the defined rotation axis mode is: after the user selects a component to define the rotation axis, the model rendering method in the defined rotation axis mode is similar to the default state rendering method, the difference is that the model to be eliminated is different. After the model is eliminated here, the detection pool of the ray detection should be adjusted so that the eliminated model does not participate in the ray detection; Step 2: The algorithm for finding the intersection of the three perpendicular lines of a triangle is an axis-finding algorithm. The algorithm is used to find the center of the circumscribed circle of the triangle. The center of the circumscribed circle of the triangle is called the axis. The axis-finding algorithm simultaneously calculates the positions of the three points of the triangle and the current camera orientation, and finds the vector perpendicular to the center of the circle. The vector is the axis. Step 3: When the user moves the mouse, the ray detection algorithm will be executed first to obtain the triangle under the current mouse, and the axis center of the triangle will be calculated using the axis center calculation algorithm. The axis center of the triangle can be obtained in real time. After the computer obtains these data, it will continue to calculate the radius of the circle. In each logical frame, ray detection is used to return the triangular face hit by the ray, and the axis center and axial direction of the triangular face are obtained using the axis center algorithm. The rendering method is changed to a rendering method that shows the axis center and axial direction graphically. Before the user clicks to confirm, the execution and rendering are repeated until the user confirms the axis center and axial direction. Among the various components, except for the base (1) and the fixture (7), only one set of axis center and axial direction need to be defined, and the remaining components need to define two sets of axis center and axial direction; The rendering method of the axis and axial graphic diagram is as follows: pre-make assets, a transparent base the size of a unit circle, and place the center of the disk perpendicular to the disk surface with an arrow pointing outward; after knowing the axis position and vector in the step of determining the axis, then determine the size of the disk by the distance between the axis and a certain point of the triangle, and apply the size, direction and position parameters to the pre-made assets in real time, so that the position of the axis point to be defined can be fed back to the user in real time; Step 4: After defining the rotation axis, call the rendering engine interface and place the bones that are compatible with the rendering engine; Step 5: According to the engine characteristics, in the default posture of the robot arm, select appropriate bone axes as the rotation axis, the orientation axis and the tangential axis. The rotation axis must be oriented toward the axial direction of the axis point, the orientation axis is always oriented toward the point where the axis of the subcomponent is mapped to the axis plane and is perpendicular to the rotation axis, and the tangential axis is perpendicular to the orientation axis and the rotation axis. The tangential axis should be oriented toward the underground direction of the plane where the robot arm is located as much as possible; Step 6: Call the engine interface to bind the component classification model in the above steps to the corresponding robotic arm components, so that the skeleton changes the robotic arm posture. Place a skeleton for each defined axis point, place two skeletons for the same component, and define the skeleton hierarchy based on the parent-child hierarchy of the component model and the motion relationship of the mechanical structure.

6. The method for realizing rapid binding and real-time animation of a robotic arm based on a web platform according to claim 5, characterized in that: Defining an axis limit module includes the following implementation steps: Step 1: Place all defined axis points in a semi-transparent sphere for interactive judgment, and match the sphere to the axis one by one; Step 2: After the user selects one of the axes, a plane with its normal direction in the direction of the axis is generated at the axis point. The plane is used to indicate the axis range set by the user, while weakening the models of other components to reduce information interference of the user; Step 3: The user selects one of the axes and moves the mouse. The intersection of the mouse position and the plane is obtained through the ray detection algorithm. The vector of the point relative to the axis is obtained through the intersection position. Step 4: On two different component models, use two bones for the two axes. The rotation limit of the axis should be relative to the bone at the axis of its parent component model. The spatial parameters of the axis inherit the parent component model. Step 5: Based on the above axis vector, calculate the rotation angle relative to the skeleton, rotate the skeleton by the angle, and provide real-time feedback to the user on the feasibility of the rotation angle; Step 6: When the user clicks and holds the mouse, the rotation angle is defined as one of the maximum or minimum angles. The direction of the mouse movement after this moment is recorded, and the axis-limited graphic rendering method is used to render the current graphic representation of the passed angle. When the user releases the mouse, the rotation angle is defined as another point of the maximum or minimum angle. Combined with the movement direction, the two values ​​can be defined as the maximum or minimum angle value; The rendering method of the axis limit graphic diagram includes the following specific implementation steps: Step 1: Among the pre-made assets, the base (1) needs to be specially made in the design software. First, generate a cylinder with 64 segments in one circle; Step 2: Then delete the top and bottom of the cylinder, and make UV cuts on the edges facing a certain world axis; Step 3: Unfold the UV evenly and scale the UV along the axis to fill all UV space; Step 4: Scale the edge of the ring on the cylinder to a point, do not merge the points; Step 5: Scale the cylinder into a circle and center it at the origin; Step 6: After the asset is created, in the fragment shader, a certain UV axis is passed. The axis depends on the state of the cylinder after UV expansion. The rendering of a certain interval value is changed to indicate the user's angle state. The rendering interval value is input by the program instruction with two floating-point numbers from 0 to 1 and a Boolean value indicating whether it is reversed. The rendering interval is converted from the angle, and the Boolean value is determined by the above-mentioned user mouse movement.

7. The method for realizing rapid binding and real-time animation of a robotic arm based on a web platform according to claim 6, characterized in that: Taking the axis point of the swinging hand (6) as the target, the rotating seat (2) is directed toward this target. The specific calculation method is: Step 1: Obtain the three-axis vectors of the rotating seat (2), which correspond to the world coordinates in the default posture; Step 2: Multiply the world position of the target point by the three-axis vector of the rotation seat (2); Step 3: Remove the z vector from the dot product result, take the square root of the sum of the squares of the x and y values, and get the rotation angle of the rotating seat (2); Step 4: Determine whether the rotation angle is between the maximum angle and the minimum angle defined above, and define the value between the maximum angle and the minimum angle as the allowable angle. If it is between the allowable angles, rotate the angle so that the rotating seat (2) faces the target point. If it is not between the allowable angles, the rotating seat (2) does not move.

8. The method for realizing rapid binding and real-time animation of a robotic arm based on a web platform according to claim 7, characterized in that: Taking the pivot point of the swinging hand (6) as the target and the pivot point of the lower arm (3) as the starting point, the simplified TwoBoneIK algorithm is used to obtain the rotation angle between the upper arm (4) and the upper arm rotation axis (5). The specific calculation method is: Step 1: Use the TwoBoneIK algorithm to calculate the distance between the target and the starting point; Step 2: Substitute the known length of the upper arm (4) bone, the length of the upper arm rotation axis (5) bone, and the distance between the target and the starting point into the cosine function to obtain the angle of each angle; Step 3: Apply the angle to the lower arm (3) and upper arm (4). The angle of the lower arm (3) is based on the angle from the current axis start point to the target point. When applying the angle, determine whether it is within the allowed angle. If the angle is not allowed, use the maximum allowed angle.

9. The method for realizing rapid binding and real-time animation of a robotic arm based on a web platform according to claim 8, characterized in that: The rotation angles of the upper arm rotation axis (5) and the swinging hand (6) are calculated based on the position of the fixture (7) and the lower direction of the fixture (7). The specific calculation method is: Step 1: after the lower direction of the fixture (7) is known in the step of determining the skeleton, the position of the fixture (7) moving a small distance in the lower direction is calculated according to the posture of the fixture (7); Step 2: Calculate the position of the microvector relative to the plane of the upper arm rotation axis (5) based on the upper arm (4) bone posture, and then use the dot multiplication algorithm to obtain the position; Step 3: After obtaining the position of the small vector relative to the plane of the upper arm rotation axis (5), calculate the angle required to rotate the lower part of the upper arm rotation axis (5) toward the relative plane position; Step 4: According to the position of the fixture (7), use the dot multiplication algorithm to obtain the projection position of the point on the swinging hand (6) to obtain the required rotation angle; Step 5: Determine whether the angle is within the allowed angle. If it is within the allowed angle, the hand (6) will rotate to the angle. If it is not within the allowed angle, the hand (6) will not move.

Images