Mechanical arm control method based on rocker cumulative offset and terminal device
By using a control method based on joystick cumulative offset, the problem of the robotic arm being unable to extend when the human hand is fully outstretched is solved, thereby improving the continuity and smoothness of the robotic arm and providing a flexible operation method and an efficient teleoperation solution.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHONGKE FIFTH CENTURY (HANGZHOU) INTELLIGENT TECHNOLOGY CO LTD
- Filing Date
- 2026-06-25
- Publication Date
- 2026-07-31
AI Technical Summary
In existing robotic arm control technology, the robotic arm cannot continue to extend when the human hand is fully extended, resulting in poor continuity and smoothness of operation.
A control method based on joystick cumulative offset is adopted. The reference position and attitude are determined by the input state of the VR handle. The local displacement and yaw angle offset are calculated by combining the axial input of the joystick, so as to realize the target position and attitude adjustment of the end effector of the robotic arm.
With the hand position locked, the position of the robotic arm's end effector can be adjusted via a joystick, improving the continuity and smoothness of robotic arm control, freeing up the hand, reducing operator fatigue, and providing flexible operating strategies and efficient teleoperation solutions.
Smart Images
Figure CN122480988A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of robotic arm control technology, and specifically discloses a robotic arm control method and terminal device based on joystick cumulative offset. Background Technology
[0002] Hand-following robotic arm control is an intelligent control technology that captures human hand movements in real time and maps them onto a robotic arm, enabling it to synchronously mimic human gestures or operational intentions. It makes human-robot interaction more natural and intuitive, and is widely used in remote operations, intelligent manufacturing, and rehabilitation assistance.
[0003] Due to the physical length of the human hand, the robotic arm often cannot extend further when the operator's arm is fully extended. The operator must frequently interrupt the operation, retract the hand, and re-establish the mapping (similar to repeatedly moving a mouse), which severely disrupts the continuity and smoothness of the operation. Summary of the Invention
[0004] The purpose of this invention is to provide a robotic arm control method and terminal device based on joystick cumulative offset, so as to solve the technical problems of poor continuity and lack of smoothness in existing robotic arm control.
[0005] A first aspect of the present invention provides a robotic arm control method based on joystick cumulative offset, comprising:
[0006] Step 1: Determine the reference position and reference posture of the VR controller based on the input state of the VR controller controlling the robotic arm;
[0007] Step 2: Determine the axial input of the joystick on the VR controller, and determine the local displacement offset and yaw angle offset based on the axial input;
[0008] Step 3: Determine the mixed position based on the reference position and the local displacement offset, and determine the mixed attitude based on the reference attitude and the yaw angle offset;
[0009] Step 4: Determine the target position and target posture of the robotic arm end effector based on the mixed position and the mixed posture.
[0010] Preferably, step 2 specifically includes:
[0011] Determine whether a joystick switching command has been received; if so, determine the yaw angle offset based on the X-axis input of the joystick on the VR controller, and determine the first local displacement offset based on the Y-axis input of the joystick on the VR controller.
[0012] If not, the second local displacement offset is determined based on the X-axis and Y-axis inputs of the joystick on the VR controller.
[0013] Preferably, the yaw angle offset is determined based on the X-axis input of the joystick on the VR controller, specifically as follows:
[0014] The yaw increment is determined based on the angular displacement scaling factor and the X-axis input of the joystick on the VR controller;
[0015] The yaw increment is cumulatively updated to obtain the yaw angle offset.
[0016] Preferably, the first local displacement offset is determined based on the Y-axis input of the joystick on the VR controller, specifically as follows:
[0017] The Z-axis increment is determined based on the linear displacement scaling factor and the Y-axis input of the joystick on the VR controller;
[0018] The Z-axis increment is cumulatively updated to obtain the Z-axis cumulative input;
[0019] The first local displacement offset is determined based on the cumulative input along the Z-axis.
[0020] Preferably, step 3 specifically includes:
[0021] When a joystick switching command is received, the mixed position is determined based on the reference position and the first local displacement offset, and the mixed attitude is determined based on the reference attitude and the yaw angle offset.
[0022] Otherwise, the mixed position is determined based on the reference position and the second local displacement offset, and the reference attitude is recorded as the mixed attitude.
[0023] Preferably, the mixing position is determined based on the reference position and the first local displacement offset, specifically as follows:
[0024] The position offset is determined by the product of the VR controller's reference posture and the first local displacement offset.
[0025] The mixed position is determined based on the sum of the position offset and the reference position.
[0026] Preferably, step 4 specifically includes:
[0027] The local displacement increment is determined based on the reference position, reference posture, and the hybrid position of the VR controller, and the target position of the robotic arm end is determined based on the local displacement increment and the reference position of the robotic arm end.
[0028] The relative pose transformation amount is determined based on the reference pose of the VR controller and the hybrid pose, and the target pose of the robotic arm end effector is determined based on the relative pose transformation amount and the reference pose of the robotic arm end effector.
[0029] Preferably, the local displacement increment is determined based on the VR controller's reference position, reference posture, and the hybrid position, specifically as follows:
[0030] The difference between the hybrid position and the reference position of the VR controller is determined and denoted as the first value;
[0031] The local displacement increment is determined based on the first value and the reference posture of the VR controller.
[0032] Preferably, step 1 specifically includes:
[0033] When the input state of the VR controller is paused, the position of the VR controller when paused is recorded as the reference position, and the posture of the VR controller when paused is recorded as the reference posture.
[0034] When the input state of the VR controller is control state, the current position of the VR controller is recorded as the reference position, and the current posture of the VR controller is recorded as the reference posture.
[0035] A second aspect of the present invention provides a terminal device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the above-described robotic arm control method based on joystick cumulative offset.
[0036] The robotic arm control method and terminal device based on joystick cumulative offset of the present invention have the following advantages compared with the prior art:
[0037] This invention enables the robotic arm to continue extending toward the target point by adjusting the end-effector's pose via a joystick when the operator's hand position is locked, thereby improving the continuity and smoothness of robotic arm control and providing an efficient, flexible, and easy-to-use solution for the application of VR controllers in the field of robot teleoperation. Attached Figure Description
[0038] Figure 1 This is a flowchart of a robotic arm control method based on joystick cumulative offset, according to an embodiment of the present invention. Detailed Implementation
[0039] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.
[0040] A first aspect of this invention provides a robotic arm control method based on joystick cumulative offset, such as... Figure 1 As shown, it includes:
[0041] Step 1: Determine the reference position and reference posture of the VR controller based on the input state of the VR controller controlling the robotic arm, specifically as follows:
[0042] When the input state of the VR controller is paused, the position of the VR controller at the time of pause is obtained through the VR pose topic. and posture and the position of the VR controller when paused. Record as reference position ,Right now Simultaneously, the posture of the VR controller during pause will be adjusted. Recorded as reference posture ,Right now This invention utilizes the position of the VR controller when paused. and posture Use as a reference position and reference posture This allows the joystick to be offset over this reference after the operator puts their hand down, improving the continuity and smoothness of subsequent operations.
[0043] When the input state of the VR controller is control state, the current position of the VR controller will be... Record as reference position ,Right now ; The current position of the VR controller Recorded as reference posture ,Right now This allows the real-time updated controller pose to serve as a reference for overlaying joystick mapping values.
[0044] In this embodiment of the invention, the position and posture of the VR controller are obtained through the WebXR / OpenXR standard protocol.
[0045] Step 2: Determine the axial input of the joystick on the VR controller, and determine the local displacement offset and yaw angle offset based on the axial input, specifically:
[0046] Determine if a joystick switching command has been received; if so, determine the yaw angle offset based on the X-axis input of the joystick on the VR controller, and determine the first local displacement offset based on the Y-axis input of the joystick on the VR controller. In this embodiment of the invention, the Y-axis of the joystick corresponds to the local forward / backward direction, and the X-axis corresponds to the local left / right direction.
[0047] Specifically, the yaw angle offset is determined based on the X-axis input of the joystick on the VR controller, as follows:
[0048] Based on the angular displacement scaling factor X-axis input of the joystick on the VR controller Determine yaw increment That is, the yaw rotation around the Z-axis corresponding to the X-axis, as shown in formula (1):
[0049] (1)
[0050] yaw increment After cumulative updating, the yaw angle offset is obtained as shown in formula (2):
[0051] (2)
[0052] In the formula, for Yaw angle offset at any given time; for Yaw angle offset at any given time; for Yaw increment at any given moment.
[0053] Specifically, the first local displacement offset is determined based on the Y-axis input of the joystick on the VR controller, as follows:
[0054] Based on the linear displacement scaling factor Y-axis input of the joystick on the VR controller Determine the Z-axis increment As shown in formula (3):
[0055] (3)
[0056] Z-axis increment Perform cumulative updates to obtain the cumulative input along the Z-axis. As shown in formula (4):
[0057] (4)
[0058] In the formula, for Accumulated input along the Z-axis at any given time; for Accumulated input along the Z-axis at any given time; for The Z-axis increment at time t.
[0059] Further based on the cumulative input along the Z-axis Determine the first local displacement offset That is, .
[0060] If no joystick switching command is received, the second local displacement offset is determined based on the X-axis and Y-axis inputs of the joystick on the VR controller, specifically:
[0061] Based on the linear displacement scaling factor Y-axis input of the joystick on the VR controller Determine the X-axis increment As shown in formula (5):
[0062] (5)
[0063] Based on the linear displacement scaling factor X-axis input of the joystick on the VR controller Determine the Y-axis increment As shown in formula (6):
[0064] (6)
[0065] Increment along the X-axis respectively and Y-axis increment Perform cumulative updates to obtain the cumulative input along the X-axis. and Y-axis cumulative input As shown in formula (7):
[0066] (7)
[0067] In the formula, for Accumulated input along the X-axis at any given time; for Accumulated input along the X-axis at any given time; for Accumulated input along the Y-axis at any given time; for Accumulated input along the Y-axis at any given time; for The increment of the X-axis at time t; for The increment along the Y-axis at time t.
[0068] Further based on and Determine the second local displacement offset ,Right now (in the) (Represented in the defined local coordinate system).
[0069] The embodiments of the present invention adopt The reason for the symbolic conventions lies in the difference in symbols between the axis reported by the virtual reality device and the operator's intuitive direction: for example, pushing the joystick forward often corresponds to... A negative value usually corresponds to pushing the joystick to the left. The result is negative. By accumulating through subtraction, this negative input can be mapped to a target motion direction that conforms to the operator's habits, thereby ensuring the consistency and usability of the control direction.
[0070] In this embodiment of the invention, preprocessing is required when using the joystick axial input. Specifically, this involves obtaining the joystick X-axis input returned by the left and right handles at the current moment. (Indicates left and right axis values) and joystick Y-axis input (Representing the forward and backward axis values), and performs dead-zone processing (threshold of 0.1) on them to suppress minor jitter; where, and This represents the current two-dimensional axis value of the joystick. The processed left and right joystick axis values are output in a unified message structure: the first set of two-dimensional components corresponds to the left joystick, and the second set of two-dimensional components corresponds to the right joystick.
[0071] Step 3: Based on the reference position The mixed position is determined by the local displacement offset and based on the reference attitude. and yaw angle offset The hybrid attitude is determined as follows:
[0072] When a joystick switching command is received, based on the reference position and the first local displacement offset Determine the mixing position based on the reference attitude. and yaw angle offset Determine the hybrid attitude.
[0073] In this embodiment of the invention, the first local displacement offset is first... Transform to the world coordinate system, and then use the transformation to the world coordinate system. Determine the mixing location Specifically, based on the baseline posture of the VR controller. offset from the first local displacement The position offset is determined by the product of the position offset and the reference position. The sum determines the mixing position. As shown in formula (8):
[0074] (8)
[0075] The hybrid attitude is determined according to the following formula (9) in the embodiments of the present invention. :
[0076] (9)
[0077] In the formula Indicates rotation about the Z-axis The rotation matrix.
[0078] If no joystick switching command is received, the embodiments of the present invention will proceed according to the reference position. Second local displacement offset Determine the mixing location and will refer to attitude This is denoted as a hybrid attitude. As shown in formulas (10) and (11) respectively:
[0079] (10)
[0080] (11)
[0081] Step 4: Based on the mixing position and hybrid posture Determine the target position at the end of the robotic arm and target posture Specifically:
[0082] Step 4.1: Based on the reference position of the VR controller Reference attitude and mixed position Determine the local displacement increment And based on the local displacement increment and the reference position of the end effector of the robotic arm Determine the target position at the end of the robotic arm .
[0083] Step 4.1 of this embodiment of the invention can map the position from VR space to robot space.
[0084] Among them, determining the local displacement increment Specifically, this involves determining the mixing location. Reference position of VR controller The difference is recorded as the first value; based on the first value and the reference posture of the VR controller... The local displacement increment is determined as shown in formula (12):
[0085] (12)
[0086] Among them, determining the target position of the robotic arm's end effector. As shown in formula (13):
[0087] (13)
[0088] Step 4.2: Based on the reference posture of the VR controller and hybrid posture Determine the relative attitude transformation amount And based on the relative attitude change amount and the reference posture of the robotic arm end effector Determine the target posture of the robotic arm's end effector .
[0089] Step 4.2 of this embodiment of the invention can map the posture from VR space to robot space.
[0090] Among them, the relative attitude transformation amount is determined. As shown in formula (14):
[0091] (14)
[0092] Among them, determining the target posture of the robotic arm's end effector. As shown in formula (15):
[0093] (15)
[0094] The target pose of the robotic arm end effector in this embodiment of the invention is: It can be output through a preset interface (such as ROS2 topic posting) for use by motion control or planning modules.
[0095] A second aspect of the present invention provides a terminal device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described robotic arm control method based on joystick cumulative offset.
[0096] The beneficial effects of this invention are as follows:
[0097] (1) Breaking through the physical range of motion limitations of the human hand: Achieving deep decoupling between "hand following" and "joystick adjustment". In the state of single-arm pause (hand following frozen), the joystick adjustment function is still active, and the operator can adjust the position of the end of the robotic arm by means of the joystick while the hand position is locked. When the human hand reaches the limit of its range of motion, the combination of "locking the hand + joystick propulsion" can drive the robotic arm to continue to extend towards the target point without changing the human posture.
[0098] (2) Free your hands and relieve fatigue: The operator can pause hand following and put down the physical arm to rest, and make position correction only by using the joystick, which solves the common pain point of "arm fatigue" in VR teleoperation.
[0099] (3) Flexible operation strategy: The operator can choose the control method flexibly. He can first use the joystick to move the robotic arm to the vicinity of the target position and then operate it by hand; or he can move the joystick while controlling it by hand to achieve a large-scale continuous workspace.
[0100] (4) Single-arm independent control: One arm can be paused for joystick adjustment. After the arm is lowered, the end can still be adjusted by the joystick. At the same time, the other arm can still perform normal hand following and joystick assistance, realizing the decoupling of the two arms with one side paused and the other side continued to operate. This improves the flexibility of operation and provides an efficient, flexible and easy-to-use solution for the application of VR devices in the field of robot teleoperation.
[0101] The above descriptions are merely a few embodiments of the present invention and are not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any modifications or alterations made by those skilled in the art without departing from the scope of the technical solution of the present invention using the disclosed technical content are equivalent to equivalent implementation cases and fall within the scope of the technical solution.
Claims
1. A robotic arm control method based on joystick cumulative offset, characterized in that, include: Step 1: Determine the reference position and reference posture of the VR controller based on the input state of the VR controller controlling the robotic arm; Step 2: Determine the axial input of the joystick on the VR controller, and determine the local displacement offset and yaw angle offset based on the axial input; Step 3: Determine the mixed position based on the reference position and the local displacement offset, and determine the mixed attitude based on the reference attitude and the yaw angle offset; Step 4: Determine the target position and target posture of the robotic arm end effector based on the mixed position and the mixed posture.
2. The robotic arm control method based on joystick cumulative offset according to claim 1, characterized in that, Step 2 is as follows: Determine whether a joystick switching command has been received; if so, determine the yaw angle offset based on the X-axis input of the joystick on the VR controller, and determine the first local displacement offset based on the Y-axis input of the joystick on the VR controller. If not, the second local displacement offset is determined based on the X-axis and Y-axis inputs of the joystick on the VR controller.
3. The robotic arm control method based on joystick cumulative offset according to claim 2, characterized in that, The yaw angle offset is determined based on the X-axis input of the joystick on the VR controller, specifically as follows: The yaw increment is determined based on the angular displacement scaling factor and the X-axis input of the joystick on the VR controller; The yaw increment is cumulatively updated to obtain the yaw angle offset.
4. The robotic arm control method based on joystick cumulative offset according to claim 2, characterized in that, The first local displacement offset is determined based on the Y-axis input of the joystick on the VR controller, specifically as follows: The Z-axis increment is determined based on the linear displacement scaling factor and the Y-axis input of the joystick on the VR controller; The Z-axis increment is cumulatively updated to obtain the Z-axis cumulative input; The first local displacement offset is determined based on the cumulative input along the Z-axis.
5. The robotic arm control method based on joystick cumulative offset according to claim 2, characterized in that, Step 3 specifically involves: When a joystick switching command is received, the mixed position is determined based on the reference position and the first local displacement offset, and the mixed attitude is determined based on the reference attitude and the yaw angle offset. Otherwise, the mixed position is determined based on the reference position and the second local displacement offset, and the reference attitude is recorded as the mixed attitude.
6. The robotic arm control method based on joystick cumulative offset according to claim 5, characterized in that, The mixing position is determined based on the reference position and the first local displacement offset, specifically as follows: The position offset is determined by the product of the VR controller's reference posture and the first local displacement offset. The mixed position is determined based on the sum of the position offset and the reference position.
7. The robotic arm control method based on joystick cumulative offset according to claim 1, characterized in that, Step 4 specifically involves: The local displacement increment is determined based on the reference position, reference posture, and the hybrid position of the VR controller, and the target position of the robotic arm end is determined based on the local displacement increment and the reference position of the robotic arm end. The relative pose transformation amount is determined based on the reference pose of the VR controller and the hybrid pose, and the target pose of the robotic arm end effector is determined based on the relative pose transformation amount and the reference pose of the robotic arm end effector.
8. The robotic arm control method based on joystick cumulative offset according to claim 7, characterized in that, The local displacement increment is determined based on the VR controller's reference position, reference posture, and the hybrid position, specifically as follows: The difference between the hybrid position and the reference position of the VR controller is determined and denoted as the first value; The local displacement increment is determined based on the first value and the reference posture of the VR controller.
9. The robotic arm control method based on joystick cumulative offset according to claim 1, characterized in that, Step 1 is as follows: When the input state of the VR controller is paused, the position of the VR controller when paused is recorded as the reference position, and the posture of the VR controller when paused is recorded as the reference posture. When the input state of the VR controller is control state, the current position of the VR controller is recorded as the reference position, and the current posture of the VR controller is recorded as the reference posture.
10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the robotic arm control method based on the cumulative offset of the joystick as described in any one of claims 1 to 9.