Teleoperation control method and apparatus based on grip force adjustment, and computer readable medium

Abstract

The application discloses a kind of remote control method, device and computer readable medium based on grip adjustment, including main terminal equipment and the slave terminal equipment being connected with main terminal equipment communication;Slave terminal equipment includes mechanical arm;The method is applied to main terminal equipment: including: at current sampling time, the current grip corresponding to user operation intention and the current direction corresponding to user operation intention are collected;Current direction is normalized, and current normalized direction vector is generated;Based on current grip, current normalized direction vector, and the maximum movement speed of mechanical arm, current mechanical arm movement vector is generated according to grip adaptive model;Current mechanical arm movement vector is converted into movement control instruction, and is sent to slave terminal equipment;Receive the mechanical arm operation result fed back by slave terminal equipment, and feedback to user.Thereby establish a kind of mapping relationship in line with human natural feeling and robot movement expectation, improve the operation efficiency and accuracy of robot remote operation.

Description

Technical Field

[0001] This invention belongs to the field of robot motion control technology, and particularly relates to a remote operation control method, device and computer-readable medium based on grip force adjustment. Background Technology

[0002] Teleoperation systems enable humans to work in dangerous or inaccessible environments, but existing human-machine interfaces still have performance bottlenecks. Traditional joystick or button interfaces require operators to translate intuitive motion intentions into discrete input actions, which increases cognitive load and further affects operational accuracy under high-pressure scenarios.

[0003] In contrast, grip strength, as a natural control dimension that has not yet been developed in robotic teleoperation, has significant advantages. In daily life, humans unconsciously and precisely adjust their grip strength, for example: lightly gripping fragile items, gripping tools with moderate force, and naturally tightening it when accelerating. This speed mapping process of grip strength adjustment is consistent with human cognitive level, which can effectively reduce the operational complexity and cognitive load during teleoperation.

[0004] It is worth noting that although bilateral force feedback has been extensively studied, force feedback in robot teleoperation needs to be combined with the development of sensorimotor skills. Currently, force-based interactive control in teleoperation mainly achieves unidirectional force transmission from robot to human, enabling the operator to perceive environmental interactions; however, research on force mapping from human to robot remains insufficient, despite its potential to enhance control intuition. Most teleoperation interfaces rely on position or velocity mapping methods, without considering the modulating effect of operator-applied forces on robot behavior. Although recent research has begun to explore the application of force input in teleoperation, research using human intervention as control input remains insufficient. Current research on force control has investigated the use of finger pressure for robot control, but has not explored the regulation of full-hand grip force; therefore, a systematic study on grip force control and traditional control methods remains lacking. Summary of the Invention

[0005] This invention provides a teleoperation control method, apparatus, and computer-readable medium based on grip strength adjustment. The method offers a continuous and natural control channel with rich tactile feedback, improving the efficiency and accuracy of teleoperation.

[0006] According to a first aspect of the present invention, a teleoperation control method based on grip force adjustment is provided, comprising a master device and a slave device communicatively connected to the master device; the slave device includes a robotic arm; the method is applied to the master device and includes: at the current sampling time, acquiring a current grip force corresponding to a user's operation intention via a grip force sensing device, and acquiring a current direction corresponding to the user's operation intention via a joystick; normalizing the current direction to generate a current normalized direction vector; generating a current robotic arm motion vector based on the current grip force, the current normalized direction vector, and the maximum movement speed of the robotic arm according to a grip force adaptive model; converting the current robotic arm motion vector into a motion control command and sending it to the slave device, so that the slave device controls the robotic arm to perform an operation corresponding to the user's operation intention according to the motion control command; receiving the robotic arm operation result fed back by the slave device, and feeding back the robotic arm operation result to the user.

[0007] Optionally, the step of generating the current robotic arm motion vector based on the current grip force, the current normalized direction vector, and the maximum movement speed of the robotic arm according to the grip force adaptive model includes: determining the current velocity factor corresponding to the current grip force based on the teleoperation task; and generating the current robotic arm motion vector using the grip force adaptive model based on the current velocity factor, the current normalized direction vector, and the maximum movement speed of the robotic arm.

[0008] Optionally, determining the current velocity factor corresponding to the current grip force based on the teleoperation task includes: determining a target velocity factor based on the change amplitude of the current grip force and the grip force release time before the current sampling time; performing damping adaptation processing on the target velocity factor to generate an actual velocity factor; when the teleoperation task is executed under a first preset condition, correcting the actual velocity factor according to the path curvature to generate a corrected velocity factor; and using the corrected velocity factor as the current velocity factor corresponding to the current grip force; wherein, the first preset condition is a predefined trajectory following operation or an environment with clearly defined structural features; when the teleoperation task is executed under a second preset condition, using the actual velocity factor as the current velocity factor corresponding to the current grip force; wherein, the second preset condition is a teleoperation mode in which an open-loop control method without real-time feedback is used to perform unconstrained flexible motion control of the controlled object in a dynamic or non-fixed structure environment.

[0009] Optionally, determining the target speed factor based on the change range of the current grip force and the grip force release time prior to the current sampling time includes: determining the change range of the current grip force based on the absolute value of the grip force difference between the current grip force and the last significant grip force prior to the current sampling time; if the change range of the current grip force is greater than a preset threshold, constructing a second function inversely proportional to the speed factor, and determining the target speed factor corresponding to the current sampling time based on the second function; if the change range of the current grip force is not greater than the preset threshold, obtaining the previous moment corresponding to the last significant grip force prior to the current sampling time; and determining the target speed factor based on the time difference between the current sampling time and the previous moment. The time difference determines the grip release time; if the grip release time is not greater than the preset memory time, the ideal speed factor is used as the target speed factor corresponding to the current sampling time; if the grip release time is greater than the preset memory time, the previous target speed factor corresponding to the previous sampling time adjacent to the current sampling time is determined; and based on the change in speed factor caused by the memory decay of the previous target speed factor and the sampling interval time, the decayed speed factor corresponding to the current sampling time is determined; if the decayed speed factor is not less than 1, 1 is selected as the target speed factor corresponding to the current sampling time; if the decayed speed factor is less than 1, the decayed speed factor is used as the target speed factor corresponding to the current sampling time.

[0010] Optionally, the method further includes: determining an ideal speed factor corresponding to the current grip force; determining the ideal speed factor corresponding to the current grip force includes: normalizing the current grip force based on the dead zone threshold and the measurable maximum force of the grip force sensing device to generate a current normalized grip force; constructing a first function that is directly proportional to the grip force and the speed factor; and determining the ideal speed factor corresponding to the current grip force based on the current normalized grip force and the first function.

[0011] Optionally, the step of performing damping adaptation processing on the target velocity factor to generate an actual velocity factor includes: obtaining the previous actual velocity factor corresponding to the previous sampling time adjacent to the current sampling time; determining the damped velocity factor corresponding to the current sampling time based on the previous actual velocity factor and the velocity factor change caused by the damping effect of the sampling interval; if the previous actual velocity factor is less than the target velocity factor, then selecting the minimum value from the damped velocity factor and the target velocity factor as the actual velocity factor corresponding to the current grip force; if the previous actual velocity factor is not less than the target velocity factor, then selecting the maximum value from the damped velocity factor and the target velocity factor as the actual velocity factor corresponding to the current grip force.

[0012] Optionally, the step of correcting the actual speed factor based on the path curvature to generate a corrected speed factor includes: calculating the local path curvature based on the three-point method of the angle between continuous path segments; determining the current curvature deceleration factor based on the local path curvature; selecting the minimum value from the current curvature deceleration factor and the maximum curvature deceleration factor as the speed correction coefficient; and correcting the actual speed factor based on the speed correction coefficient to generate the corrected speed factor.

[0013] Optionally, the step of normalizing the current grip force based on the dead zone threshold of the grip force sensing device and the measurable maximum force to generate a current normalized grip force includes: if the current grip force is not greater than the dead zone threshold of the grip force sensing device, then the current grip force is determined to be noise or an invalid signal, and the normalization result of the current grip force is set to 0; if the current grip force is not less than the measurable maximum force, then the normalization result of the current grip force is determined to be 1; if the current grip force is not less than the measurable maximum force and not greater than the dead zone threshold, then the normalization result of the current grip force increases linearly with the increase of the current grip force.

[0014] According to a second aspect of the present invention, a remote operation control device based on grip force adjustment is also provided, including a master device and a slave device communicatively connected to the master device; the slave device includes a robotic arm; the device utilizes the master device; and includes at least: a data acquisition module, configured to acquire, at the current sampling time, a current grip force corresponding to a user's operation intention via a grip force sensing device, and a current direction corresponding to the user's operation intention via a joystick; a direction normalization module, configured to normalize the current direction to generate a current normalized direction vector; a generation module, configured to generate a current robotic arm motion vector based on the current grip force, the current normalized direction vector, and the maximum movement speed of the robotic arm, according to the grip force adaptive model; a sending module, configured to convert the current robotic arm motion vector into motion control commands and send them to the slave device, so that the slave device controls the robotic arm to perform an operation corresponding to the user's operation intention according to the motion control commands; and a feedback module, configured to receive the robotic arm operation results fed back by the slave device and feed them back to the user.

[0015] According to a third aspect of the present invention, an electronic device is also provided, comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the method as described in the first aspect.

[0016] According to a fourth aspect of the present invention, a computer-readable medium is also provided, on which a computer program is stored, wherein the program, when executed by a processor, implements the method described in the first aspect.

[0017] This invention provides a teleoperation control method, device, and computer-readable medium based on grip force adjustment, including a master device and a slave device communicatively connected to the master device; the slave device includes a robotic arm; the method is applied to the master device and includes: first, at the current sampling time, acquiring the current grip force corresponding to the user's operation intention through a grip force sensing device, and acquiring the current direction corresponding to the user's operation intention through a joystick; then, normalizing the current direction to generate a current normalized direction vector; second, based on the current grip force, the current normalized direction vector, and the maximum movement speed of the robotic arm, generating a current robotic arm motion vector according to a grip force adaptive model; then, converting the current robotic arm motion vector into motion control commands and sending them to the slave device, so that the slave device controls the robotic arm to perform the operation corresponding to the user's operation intention according to the motion control commands; finally, receiving the robotic arm operation result fed back by the slave device and feeding back the robotic arm operation result to the user. This embodiment establishes a mapping relationship between human natural perception and robot motion expectations during robot teleoperation based on the adaptive speed mapping method of grip force adjustment, thereby improving the efficiency and accuracy of robot teleoperation. Attached Figure Description

[0018] The following sections will describe some specific embodiments of the invention in a detailed manner by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0019] Figure 1 This is a flowchart illustrating a remote control method based on grip strength adjustment according to an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the process for determining the current speed factor corresponding to the current grip force based on a teleoperation task in one embodiment of the present invention;

[0021] Figure 3 A schematic diagram of a remote control device based on grip strength adjustment provided in an embodiment of the present invention;

[0022] Figure 4 This invention provides a grip force adaptive speed interaction system according to an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] like Figure 1 The diagram shown is a flowchart illustrating a remote control method based on grip strength adjustment according to an embodiment of the present invention.

[0025] A remote control method based on grip force adjustment includes a master device and a slave device communicatively connected to the master device; the slave device includes a robotic arm; the method applies the master device; and includes at least the following steps:

[0026] S101, at the current sampling moment, the current grip force corresponding to the user's operation intention is collected through the grip force sensing device, and the current direction corresponding to the user's operation intention is collected through the joystick;

[0027] S102, Normalize the current direction to generate the current normalized direction vector;

[0028] S103, Based on the current grip force, the current normalized direction vector, and the maximum movement speed of the robotic arm, generate the current robotic arm motion vector according to the grip force adaptive model;

[0029] S104, convert the current robotic arm motion vector into motion control commands and send them to the slave device so that the slave device can control the robotic arm to perform operations corresponding to the user's intentions according to the motion control commands;

[0030] S105 receives the robotic arm operation results from the slave device and sends the robotic arm operation results back to the user.

[0031] In S101, the grip force sensing device is a device capable of detecting and measuring the magnitude of a user's grip force. For example, a common grip force sensor can accurately sense the force applied by the hand and convert it into a processable form such as an electrical or digital signal for subsequent control of the robotic arm's movement rate. The joystick is a device used to determine the direction of the robotic arm's movement; it can acquire the directional information input by the user to guide the robotic arm to move in a specific direction. In this embodiment, the grip force sensing device collects the grip force signal applied by the user in real time (such as changes in force magnitude), while the joystick synchronously captures the user's directional intentions (such as spatial angles and displacement). These two channels work in parallel, independently recording the raw operational data in the force and direction dimensions respectively, achieving physical decoupling between speed control and direction control.

[0032] In S102, the current direction is normalized to generate a current normalized direction vector; this includes: obtaining the two-dimensional vector corresponding to the current direction; normalizing the two-dimensional vector to generate the current normalized direction vector.

[0033] In S103, for example, based on the teleoperation task, the current velocity factor corresponding to the current grip force is determined according to preset rules or a trained model; based on the current velocity factor, the current normalized direction vector, and the maximum motion speed of the robotic arm, the current robotic arm motion vector is generated using a grip force adaptive model. The formula for the grip force adaptive model is shown below:

[0034] v=(Φ(f)v max ) j Equation (1);

[0035] Where v is the motion vector of the robotic arm; Φ(f) is a mapping function that converts the gripping force f into a velocity factor. The mapping function can take various forms, such as linear, quadratic, logarithmic, exponential, etc. max is the maximum speed of the robotic arm; j is the normalized direction vector from the remote sensing input.

[0036] These different types of mapping functions will result in different relationships between grip force and speed, thus producing different control dynamics characteristics. For example, a linear mapping may make the speed change uniformly with grip force; a quadratic mapping may make the speed change slower when the grip force is small and faster when the grip force is large.

[0037] Here, the current motion vector of the robotic arm includes: the robotic arm's motion speed and direction;

[0038] In steps S104 and S105, the master device converts the current robotic arm motion vector into motion control commands and sends them to the slave device. The slave device then controls the robotic arm to perform operations corresponding to the user's intended actions based on these commands. Simultaneously, the slave device uses a mounted camera to capture real-time video streams of the robotic arm's movement and transmits these streams to the master device. The user receives real-time updates on the robotic arm's motion status through the master device's human-machine interface. This allows the user to adjust the robotic arm's motion control strategy promptly based on the observed data.

[0039] Compared to force-based remote robot control, this embodiment uses an adaptive speed mapping method based on grip force adjustment. This establishes a mapping relationship between human natural sensation and robot movement expectations during robot teleoperation, improving the efficiency and accuracy of robot teleoperation to a certain extent. Furthermore, unlike traditional speed control based on binary switches or linear joystick control, this embodiment's grip force-based interaction design directly utilizes a natural proprioceptive feedback mechanism, providing a continuous and natural control channel with rich tactile feedback.

[0040] like Figure 2 The diagram shown is a flowchart illustrating the process of determining the current speed factor corresponding to the current grip force based on a teleoperation task in one embodiment of the present invention.

[0041] Based on the teleoperation task, determine the current velocity factor corresponding to the current grip force; this includes at least the following steps:

[0042] S201, Determine the target velocity factor based on the current grip force change amplitude and the grip force release time before the current sampling time;

[0043] S202, Damping adaptation processing is applied to the target velocity factor to generate the actual velocity factor;

[0044] S203, when the teleoperation task is performed under the first preset condition, the actual speed factor is corrected according to the path curvature to generate the corrected speed factor; and the corrected speed factor is used as the current speed factor corresponding to the current grip force; wherein, the first preset condition is a predefined trajectory following operation or an environment with clear structural features;

[0045] S204, When the teleoperation task is executed under the second preset condition, the actual speed factor is used as the current speed factor corresponding to the current grip force; wherein, the second preset condition is a teleoperation mode in which the controlled object is manipulated without constraints by adopting an open-loop control method without real-time feedback in a dynamic or non-fixed environment.

[0046] In S201, the target velocity factor is determined based on the magnitude of the change in the current grip force and the grip force release time before the current sampling time, according to preset rules or algorithm models.

[0047] For example, determining a target speed factor based on the magnitude of the change in current grip force and the grip force release time prior to the current sampling time includes: determining the magnitude of the change in current grip force based on the absolute value of the grip force difference between the current grip force and the last significant grip force prior to the current sampling time; if the magnitude of the change in current grip force is greater than a preset threshold, constructing a second function inversely proportional to the speed factor, and determining the target speed factor corresponding to the current sampling time based on the second function; if the magnitude of the change in current grip force is not greater than the preset threshold, obtaining the previous moment corresponding to the last significant grip force prior to the current sampling time; and determining the target speed factor based on the current sampling time and the previous moment. The time difference between the two values ​​determines the grip release time; if the grip release time is not greater than the preset memory time, the ideal speed factor is used as the target speed factor; if the grip release time is greater than the preset memory time, the previous target speed factor corresponding to the previous sampling time adjacent to the current sampling time is determined; and based on the change in speed factor caused by the memory decay of the previous target speed factor and the sampling interval time, the decayed speed factor corresponding to the current sampling time is determined; if the decayed speed factor is not less than 1, 1 is selected as the target speed factor corresponding to the current sampling time; if the ideal speed factor is less than 1, the decayed speed factor is used as the target speed factor corresponding to the current sampling time.

[0048] For example: based on the dead zone threshold and the measurable maximum force of the grip force sensing device, the grip force is normalized to generate a normalized grip force; based on the normalized grip force, a second function in which the grip force and the speed factor are inversely proportional is constructed; as shown in formula (2).

[0049] Based on the magnitude of the current grip force change and the grip force release time prior to the current sampling moment, the target velocity factor v at the current moment is calculated using the following formula. t (t).

[0050]

[0051] Among them, f n (t) represents the current normalized grip force; f(t) represents the current grip force, f l v represents the last significant grip force prior to the current sampling time. t (t-Δt) represents the target velocity factor corresponding to the previous sampling time adjacent to the current sampling time, Δt is the sampling interval, and γ is the target velocity factor. d It is the attenuation rate, t l This represents the time preceding the last significant grip force, i.e., the moment corresponding to the significant grip force; δ g A preset threshold for the range of grip strength variation; τ m Preset the memory time.

[0052] This embodiment solves the problem of "fatigue caused by continuous force application" in basic grip force adjustment by "memorizing" the user's grip force preference and maintaining the target speed after the grip force is released without continuous force application.

[0053] The ideal speed factor can be preset or obtained based on grip strength.

[0054] For example, determining the ideal speed factor corresponding to the current grip force includes: normalizing the current grip force based on the dead zone threshold of the grip force sensing device and the measurable maximum force to generate a current normalized grip force; constructing a first function that is directly proportional to the grip force and the speed factor; and determining the ideal speed factor corresponding to the current grip force based on the current normalized grip force and the first function. Specifically, normalizing the current grip force based on the dead zone threshold of the grip force sensing device and the measurable maximum force to generate a current normalized grip force includes: if the current grip force is not greater than the dead zone threshold of the grip force sensing device, then the current grip force is determined to be noise or an invalid signal, and the normalization result of the current grip force is set to 0; if the current grip force is not less than the measurable maximum force, then the normalization result of the current grip force is set to 1; if the current grip force is not less than the measurable maximum force and not greater than the dead zone threshold, then the normalization result of the current grip force increases linearly with the increase of the current grip force.

[0055] For example, the grip force f is normalized using the following formula to generate a normalized grip force f. n ;

[0056]

[0057] Among them, f d f is the dead zone threshold of the grip force sensing device. m This is the maximum measurable force.

[0058] Substituting formula (3) into the mapping function of the velocity factor in formula (1), we can construct the first function, as shown in formula (4):

[0059]

[0060] Input the current normalized grip force into the first function to determine the ideal speed factor corresponding to the current grip force.

[0061] In S202, based on preset rules or algorithm models, the target velocity factor is subjected to damping adaptation processing to generate the actual velocity factor.

[0062] For example, the step of performing damping adaptation processing on the target velocity factor to generate an actual velocity factor includes: obtaining the previous actual velocity factor corresponding to the previous sampling time adjacent to the current sampling time; determining the damped velocity factor corresponding to the current sampling time based on the previous actual velocity factor and the velocity factor change caused by the damping effect of the sampling interval; if the previous actual velocity factor is less than the target velocity factor, then selecting the minimum value from the damped velocity factor and the target velocity factor as the actual velocity factor corresponding to the current grip force; if the previous actual velocity factor is not less than the target velocity factor, then selecting the maximum value from the damped velocity factor and the target velocity factor as the actual velocity factor corresponding to the current grip force.

[0063] For example, to prevent sudden transitions, the target velocity factor is damped and adapted based on the following formula to generate the actual velocity factor v. f (t).

[0064]

[0065] Among them, v f (t-Δt) is the previous actual velocity factor corresponding to the previous sampling time adjacent to the current sampling time; v t (t) is the target velocity factor at the current moment; γ a For the fitness rate.

[0066] This embodiment uses damping adaptation to gradually adjust the current speed, avoiding sudden acceleration or deceleration, so that the actual speed of the robotic arm smoothly follows the target speed, improving the stability of remote operation.

[0067] In S203, for example, the actual speed factor is corrected based on the path curvature according to a preset rule or algorithm model to generate a corrected speed factor.

[0068] For example, the local path curvature is calculated based on the three-point method of the included angle between continuous path segments; the current curvature deceleration factor is determined according to the local path curvature; the minimum value is selected from the current curvature deceleration factor and the maximum curvature deceleration factor as the speed correction coefficient; the actual speed factor is corrected based on the speed correction coefficient to generate the corrected speed factor.

[0069] For example, the formula for calculating the local path curvature κ is shown below;

[0070]

[0071] in, and It represents a normalized vector of a continuous path.

[0072] The formula for calculating the corrected velocity factor is as follows:

[0073] v context =v f ·min(γ curve ,1-10κ); Equation (7);

[0074] Where κ∈[0,1] is the local path curvature, γ curve It is the maximum curvature deceleration factor; min(γ) curve ,1-10κ) is the velocity correction factor.

[0075] This embodiment combines automatic speed adjustment based on path curvature to achieve scenario adaptation; thereby simulating natural human movement habits and reducing the need for manual speed adjustment by the user. For example, by automatically reducing movement speed in high curvature areas, the user does not need to actively adjust grip strength to change speed when curvature changes, thus reducing the operational load.

[0076] It should be noted that in predefined trajectory following operations or environments with well-defined structural features, the speed needs to be calculated and adjusted in real time based on the path curvature to reduce the user's grip force load. In dynamic or non-fixed structure environments, free-form teleoperation under open-loop conditions cannot achieve path curvature correction. The system mainly maintains its core functions by adjusting the speed based on grip force, but it does not have an automatic curvature compensation mechanism.

[0077] This embodiment of the method constructs a grip force adaptive model to convert and map grip force to speed; at the same time, by enhancing the grip force adaptive model, the robotic arm has grip force memory and situational adaptation capabilities, which reduces operator fatigue during long-term operation and effectively avoids sudden speed changes caused by sudden changes in grip force. The situational adaptation function mainly helps the operator update the operation rate by combining grip force with the curvature of the operation trajectory when performing detailed operations.

[0078] The following describes in detail a remote control method based on grip strength adjustment provided in this embodiment, taking into account specific application scenarios.

[0079] A remote control method based on grip force adjustment includes a master device and a slave device communicatively connected to the master device; the slave device includes a robotic arm; the method applies the master device; and includes at least the following steps:

[0080] S1, at the current sampling moment, the current grip force corresponding to the user's operation intention is collected through the grip force sensing device, and the current direction corresponding to the user's operation intention is collected through the joystick.

[0081] S2, normalize the current direction to generate the current normalized direction vector.

[0082] S3, based on the dead zone threshold and the measurable maximum force of the grip force sensing device, normalize the current grip force to generate the current normalized grip force; construct a first function that is directly proportional to the grip force and the speed factor; based on the current normalized grip force, determine the ideal speed factor corresponding to the current grip force according to the first function.

[0083] S4, determine the variation range of the current grip force based on the absolute value of the grip force difference between the current grip force and the last significant grip force before the current sampling time; if the variation range of the current grip force is greater than a preset threshold, construct a second function that is inversely proportional to the speed factor, and determine the target speed factor corresponding to the current sampling time based on the second function; if the variation range of the current grip force is not greater than the preset threshold, obtain the previous moment corresponding to the last significant grip force before the current sampling time; determine the grip force release time based on the time difference between the current sampling time and the previous moment; if the grip force release... If the release time is not greater than the preset memory time, the ideal speed factor is used as the target speed factor. If the grip release time is greater than the preset memory time, the previous target speed factor corresponding to the previous sampling time adjacent to the current sampling time is determined. Based on the change in speed factor caused by the memory decay of the previous target speed factor and the sampling interval time, the decayed speed factor corresponding to the current sampling time is determined. If the decayed speed factor is not less than 1, 1 is selected as the target speed factor corresponding to the current sampling time. If the ideal speed factor is less than 1, the decayed speed factor is used as the target speed factor corresponding to the current sampling time.

[0084] S5, obtain the previous actual velocity factor corresponding to the previous sampling time adjacent to the current sampling time; based on the previous actual velocity factor and the velocity factor change caused by the damping effect of the sampling interval, determine the damped velocity factor corresponding to the current sampling time; if the previous actual velocity factor is less than the target velocity factor, select the minimum value from the damped velocity factor and the target velocity factor as the actual velocity factor corresponding to the current grip force; if the previous actual velocity factor is not less than the target velocity factor, select the maximum value from the damped velocity factor and the target velocity factor as the actual velocity factor corresponding to the current grip force.

[0085] S6, when the teleoperation task is executed under the first preset condition, the local path curvature is calculated based on the three-point method of the angle between continuous path segments; the current curvature deceleration factor is determined according to the local path curvature; the minimum value is selected from the current curvature deceleration factor and the maximum curvature deceleration factor as the speed correction coefficient; the actual speed factor is corrected based on the speed correction coefficient to generate a corrected speed factor. The corrected speed factor is then used as the current speed factor corresponding to the current grip force; wherein, the first preset condition is a predefined trajectory following operation or an environment with clearly defined structural features.

[0086] S7, When the teleoperation task is executed under the second preset condition, the actual speed factor is used as the current speed factor corresponding to the current grip force; wherein, the second preset condition is a teleoperation mode in which the controlled object is manipulated without constraints by adopting an open-loop control method without real-time feedback in a dynamic or non-fixed environment.

[0087] S8. Based on the current velocity factor, the current normalized direction vector, and the maximum motion speed of the robotic arm, the current robotic arm motion vector is generated using the grip force adaptive model.

[0088] S9, the current robotic arm motion vector is converted into motion control commands and sent to the slave device, so that the slave device controls the robotic arm to perform operations corresponding to the user's operation intention according to the motion control commands.

[0089] S10, receive the robotic arm operation results fed back by the slave device, and feed the robotic arm operation results back to the user.

[0090] When the method of this embodiment is applied to path tracking, obstacle avoidance, and target point traversal tasks in robot teleoperation, it performs well in terms of operational efficiency and accuracy.

[0091] In this embodiment of teleoperation, grip force is directly mapped to the movement rate of the robotic arm, and robot control is achieved in conjunction with a direction input device. Furthermore, by constructing grip force memory and scenario-adaptive functions, teleoperation can maintain speed without continuously applying grip force. Simultaneously, the mapping relationship between grip force and speed can be adaptively adjusted to adapt to different situations when the curvature of the movement path changes. This results in a continuous and natural control channel with rich tactile feedback.

[0092] like Figure 4 As shown, this is a grip force adaptive speed interaction system provided in an embodiment of the present invention.

[0093] The grip force adaptive model comprises three main components: a force-velocity mapping component that converts grip force into a velocity scaling factor, a context adaptation component, and a grip force memory component. The master device acquires the current direction corresponding to the user's operational intent via a joystick and the current grip force corresponding to the user's operational intent via a grip force sensing device. The grip force adaptive model processes the input current grip force and current direction, outputting the current robotic arm motion vector. Specifically, the input current grip force is processed sequentially through force-velocity mapping, context adaptation, and grip force memory to output the current velocity factor. The current velocity factor and current direction are combined to control the robotic arm's motion. The slave device acquires a video stream of the robotic arm's motion and provides visual feedback to the user on the master device.

[0094] like Figure 3 The diagram shown is a schematic representation of a remote control device based on grip strength adjustment according to an embodiment of the present invention.

[0095] A remote control device based on grip force adjustment includes: a master device and a slave device communicatively connected to the master device; the slave device includes a robotic arm; the device utilizes the master device; the device 300 includes: a data acquisition module 301, used to acquire the current grip force corresponding to the user's operation intention via a grip force sensing device at the current sampling time, and to acquire the current direction corresponding to the user's operation intention via a joystick; a direction normalization module 302, used to normalize the current direction to generate a current normalized direction vector; a generation module 303, used to generate a current robotic arm motion vector based on the current grip force, the current normalized direction vector, and the maximum movement speed of the robotic arm, according to a grip force adaptive model; a sending module 304, used to convert the current robotic arm motion vector into motion control commands and send them to the slave device, so that the slave device can control the robotic arm to perform an operation corresponding to the user's operation intention according to the motion control commands; and a feedback module 305, used to receive the robotic arm operation results fed back by the slave device and feed the robotic arm operation results back to the user.

[0096] In a preferred embodiment of this example, the generation module includes: a determining unit, used to determine the current velocity factor corresponding to the current grip force based on the teleoperation task; and a generation unit, used to generate the current robotic arm motion vector based on the current velocity factor, the current normalized direction vector, and the maximum motion speed of the robotic arm using the grip force adaptive model.

[0097] In a preferred embodiment of this example, the determining unit includes: a first determining subunit, used to determine a target velocity factor based on the change in current grip force and the grip force release time prior to the current sampling time; a damping adaptation processing subunit, used to perform damping adaptation processing on the target velocity factor to generate an actual velocity factor; a second determining subunit, used to modify the actual velocity factor according to the path curvature and generate a modified velocity factor when the teleoperation task is executed under a first preset condition; and use the modified velocity factor as the current velocity factor corresponding to the current grip force; wherein, the first preset condition is a predefined trajectory following operation or an environment with clearly defined structural features; a third determining subunit, used to use the actual velocity factor as the current velocity factor corresponding to the current grip force when the teleoperation task is executed under a second preset condition; wherein, the second preset condition is a teleoperation mode in which an open-loop control method without real-time feedback is used to perform unconstrained flexible motion manipulation of the controlled object in a dynamic or non-fixed structure environment.

[0098] In a preferred embodiment of this example, the first determining subunit includes: a first determining unit, configured to determine the variation range of the current grip force based on the absolute value of the grip force difference between the current grip force and the last significant grip force prior to the current sampling time; a second determining unit, configured to construct a second function inversely proportional to the speed factor if the variation range of the current grip force is greater than a preset threshold, and determine a target speed factor corresponding to the current sampling time based on the second function; and a third determining unit, configured to obtain the previous moment corresponding to the last significant grip force prior to the current sampling time if the variation range of the current grip force is not greater than a preset threshold, and determine the grip force based on the time difference between the current sampling time and the previous moment. The system includes a release time; a fourth determining unit, configured to, if the grip force release time is not greater than a preset memory time, use the ideal speed factor as the target speed factor corresponding to the current sampling time; and a fifth determining unit, configured to, if the grip force release time is greater than the preset memory time, determine the previous target speed factor corresponding to the previous sampling time adjacent to the current sampling time; and, based on the change in speed factor caused by the memory decay of the previous target speed factor and the sampling interval time, determine the decayed speed factor corresponding to the current sampling time; if the decayed speed factor is not less than 1, select 1 as the target speed factor corresponding to the current sampling time; if the decayed speed factor is less than 1, use the decayed speed factor as the target speed factor corresponding to the current sampling time.

[0099] In a preferred embodiment of this example, the first determining subunit further includes a sixth determining unit; the sixth determining unit includes: a normalization processing subunit, used to normalize the current grip force based on the dead zone threshold and the measurable maximum force of the grip force sensing device, to generate a current normalized grip force; a construction subunit, used to construct a first function that is directly proportional to the grip force and the speed factor; and a determining subunit, used to determine the ideal speed factor corresponding to the current grip force based on the current normalized grip force and according to the first function.

[0100] In a preferred embodiment of this example, the damping adaptation processing subunit includes: an acquisition unit, configured to acquire the previous actual velocity factor corresponding to the previous sampling time adjacent to the current sampling time; a first determination unit, configured to determine the damped post-processing velocity factor corresponding to the current sampling time based on the previous actual velocity factor and the velocity factor change caused by the damping effect of the sampling interval; a selection unit, configured to select the minimum value from the damped post-processing velocity factor and the target velocity factor as the actual velocity factor corresponding to the current grip force if the previous actual velocity factor is less than the target velocity factor; and a second determination unit, configured to select the maximum value from the damped post-processing velocity factor and the target velocity factor as the actual velocity factor corresponding to the current grip force if the previous actual velocity factor is not less than the target velocity factor.

[0101] In a preferred embodiment of this example, the third determining subunit includes: a calculation unit for calculating the local path curvature based on the three-point method of the angle between continuous path segments; a determining unit for determining the current curvature deceleration factor based on the local path curvature; a selection unit for selecting the minimum value from the current curvature deceleration factor and the maximum curvature deceleration factor as a speed correction coefficient; and a correction unit for correcting the actual speed factor based on the speed correction coefficient to generate a corrected speed factor.

[0102] In a preferred embodiment of this example, the sixth determining unit includes: a first determining subunit, configured to determine that the current grip force is noise or an invalid signal and normalize the current grip force to 0 if the current grip force is not greater than the dead zone threshold of the grip force sensing device; a second determining subunit, configured to determine that the normalized result of the current grip force is 1 if the current grip force is not less than the maximum measurable force; and a third determining subunit, configured to linearly increase the normalized result of the current grip force as the current grip force increases if the current grip force is not less than the maximum measurable force and not greater than the dead zone threshold.

[0103] The above-described device can execute a remote operation control method based on grip strength adjustment provided in an embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing a remote operation control method based on grip strength adjustment. Technical details not described in detail in this embodiment can be found in the remote operation control method based on grip strength adjustment provided in an embodiment of the present invention.

[0104] The present invention also provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement a remote operation control method based on grip force adjustment as described in the present invention.

[0105] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0106] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0107] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to the following embodiments of this application described in the "Exemplary Methods" section above.

[0108] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0109] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0110] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0111] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0112] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0113] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

[0114] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0115] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0116] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A teleoperation control method based on grip strength adjustment, characterized in that, The method includes a master device and a slave device communicatively connected to the master device; the slave device includes a robotic arm; the method applies the master device; and includes: At the current sampling moment, the grip force corresponding to the user's operation intention is collected through the grip force sensing device, and the current direction corresponding to the user's operation intention is collected through the joystick; The current direction is normalized to generate a current normalized direction vector; Based on the change in current grip force and the grip force release time prior to the current sampling moment, a target velocity factor is determined; the target velocity factor is subjected to damping adaptation processing to generate an actual velocity factor; when the teleoperation task is executed under a first preset condition, the actual velocity factor is corrected according to the path curvature to generate a corrected velocity factor; and the corrected velocity factor is used as the current velocity factor corresponding to the current grip force; wherein, the first preset condition is a predefined trajectory following operation or an environment with clearly defined structural features; when the teleoperation task is executed under a second preset condition, the actual velocity factor is used as the current velocity factor corresponding to the current grip force; wherein, the second preset condition is a teleoperation mode in a dynamic or non-fixed structure environment, using an open-loop control method without real-time feedback to perform unconstrained flexible motion manipulation of the controlled object; based on the current velocity factor, the current normalized direction vector, and the maximum motion speed of the robotic arm, the current robotic arm motion vector is generated using a grip force adaptive model; The current robotic arm motion vector is converted into motion control commands and sent to the slave device, so that the slave device can control the robotic arm to perform operations corresponding to the user's operation intentions according to the motion control commands; The system receives the robotic arm operation results from the slave device and then sends those results back to the user.

2. The method according to claim 1, characterized in that, The determination of the target velocity factor based on the current grip force change amplitude and the grip force release time prior to the current sampling time includes: The magnitude of the change in the current grip force is determined based on the absolute value of the difference between the current grip force and the last significant grip force prior to the current sampling time. If the change in the current grip force is greater than a preset threshold, a second function that is inversely proportional to the grip force and the speed factor is constructed, and the target speed factor corresponding to the current sampling time is determined based on the second function. If the change in current grip force is not greater than a preset threshold, then obtain the previous moment corresponding to the last significant grip force before the current sampling moment; determine the grip force release time based on the time difference between the current sampling moment and the previous moment; If the grip force release time is not greater than the preset memory time, then the ideal speed factor will be used as the target speed factor corresponding to the current sampling time. If the grip force release time is greater than the preset memory time, then the previous target velocity factor corresponding to the previous sampling time adjacent to the current sampling time is determined; and based on the change in velocity factor caused by the memory decay of the previous target velocity factor and the sampling interval time, the decayed velocity factor corresponding to the current sampling time is determined; if the decayed velocity factor is not less than 1, then 1 is selected as the target velocity factor corresponding to the current sampling time; if the decayed velocity factor is less than 1, then the decayed velocity factor is used as the target velocity factor corresponding to the current sampling time.

3. The method according to claim 2, characterized in that, Also includes: Determine the ideal velocity factor corresponding to the current grip force; The determination of the ideal speed factor corresponding to the current grip force includes: Based on the dead zone threshold and the measurable maximum force of the grip force sensing device, the current grip force is normalized to generate the current normalized grip force. Construct the first function that shows a direct proportional relationship between grip strength and speed factor; Based on the current normalized grip force, the ideal speed factor corresponding to the current grip force is determined according to the first function.

4. The method according to claim 1, characterized in that, The step of performing damping adaptation processing on the target velocity factor to generate the actual velocity factor includes: Obtain the previous actual velocity factor corresponding to the previous sampling time adjacent to the current sampling time; Based on the change in velocity factor caused by the damping effect of the previous actual velocity factor and the sampling interval time, determine the damped velocity factor corresponding to the current sampling moment. If the previous actual speed factor is less than the target speed factor, then the minimum value between the damped speed factor and the target speed factor is selected as the actual speed factor corresponding to the current grip force. If the previous actual speed factor is not less than the target speed factor, then the maximum value between the damped speed factor and the target speed factor is selected as the actual speed factor corresponding to the current grip force.

5. The method according to claim 1, characterized in that, The step of correcting the actual velocity factor based on the path curvature to generate the corrected velocity factor includes: Calculate local path curvature using the three-point method based on the angle between continuous path segments; Based on the local path curvature, determine the current curvature deceleration factor; The minimum value between the current curvature deceleration factor and the maximum curvature deceleration factor is selected as the velocity correction coefficient; The actual speed factor is corrected based on the speed correction coefficient to generate a corrected speed factor.

6. The method according to claim 3, characterized in that, The step of normalizing the current grip force based on the dead zone threshold and the measurable maximum force of the grip force sensing device to generate a current normalized grip force includes: If the current grip force is not greater than the dead zone threshold of the grip force sensing device, then the current grip force is determined to be noise or invalid signal, and the normalization result of the current grip force is set to 0; If the current grip force is not less than the maximum measurable force, then the normalized result of the current grip force is determined to be 1; If the current grip force is not less than the maximum measurable force and not greater than the dead zone threshold, then the normalized result of the current grip force increases linearly with the increase of the current grip force.

7. A remote control device based on grip strength adjustment, characterized in that, include: A master device and a slave device that is communicatively connected to the master device; The slave device includes a robotic arm; the device utilizes a master device; including: The acquisition module is used to acquire the current grip force corresponding to the user's operation intention through the grip force sensing device at the current sampling time, and to acquire the current direction corresponding to the user's operation intention through the joystick; The direction normalization module is used to normalize the current direction and generate the current normalized direction vector; A generation module is used to determine a target velocity factor based on the change in current grip force and the grip force release time prior to the current sampling time; perform damping adaptation processing on the target velocity factor to generate an actual velocity factor; when the teleoperation task is executed under a first preset condition, the actual velocity factor is corrected according to the path curvature to generate a corrected velocity factor; and the corrected velocity factor is used as the current velocity factor corresponding to the current grip force; wherein, the first preset condition is a predefined trajectory following operation or an environment with clearly defined structural features; when the teleoperation task is executed under a second preset condition, the actual velocity factor is used as the current velocity factor corresponding to the current grip force; wherein, the second preset condition is a teleoperation mode in a dynamic or non-fixed structure environment, using an open-loop control method without real-time feedback to perform unconstrained flexible motion manipulation of the controlled object; based on the current velocity factor, the current normalized direction vector, and the maximum motion speed of the robotic arm, a grip force adaptive model is used to generate the current robotic arm motion vector; The sending module is used to convert the current robotic arm motion vector into motion control commands and send them to the slave device, so that the slave device can control the robotic arm to perform an operation corresponding to the user's operation intention according to the motion control commands; The feedback module is used to receive the robotic arm operation results fed back by the slave device and to feed back the robotic arm operation results to the user.

8. A computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-6.

Images