Teleoperation system, teleoperation method, and robot arm

Abstract

This application belongs to the field of robotic arm technology, and relates to a teleoperation system, a teleoperation method, and a robotic arm. The teleoperation system includes a master robotic arm and a slave robotic arm controlled by the master robotic arm; wherein, at least one part of the master robotic arm is equipped with a force sensor; the slave robotic arm is configured with a constraint function, the constraint function including: constraining the movement of the slave robotic arm so that the output force of the slave robotic arm is close to and less than the constraint force, the constraint force originating from the master robotic arm and generated based on the sensing signal of the force sensor. The technical solution provided by this application can improve the stability and safety of the teleoperation system.

Description

Technical Field

[0001] This application belongs to the field of robotic arm technology, and particularly relates to a teleoperation system, teleoperation method and robotic arm. Background Technology

[0002] Teleoperation is an important branch of robot control applications, and its system mainly consists of master and slave robotic arms. In a teleoperation system, the operator can remotely control the slave robotic arm by operating the master robotic arm.

[0003] However, even with today's rapid advancements in communication technology, communication delays remain unavoidable in daily life, and teleoperation systems face the same challenge. In existing technologies, communication delays can prevent operators from timely obtaining information about the actual operating status of the robotic arm, leading to over-control and potentially causing operational accidents due to excessive movement of the robotic arm. Summary of the Invention

[0004] This application provides a teleoperation system, a teleoperation method, and a robotic arm, which can improve the stability and reliability of teleoperation.

[0005] The first aspect of this application provides a teleoperation system, including: a master robotic arm and a slave robotic arm controlled by the master robotic arm, wherein at least one part of the master robotic arm is provided with a force sensor.

[0006] The aforementioned slave robot arm is equipped with a constraint function, which includes: constraining the movement of the slave robot arm so that the output force of the slave robot arm is close to and less than the constraint force, wherein the constraint force comes from the master robot arm and is generated based on the sensing signal of the force sensor.

[0007] Based on the first aspect of this application, in a first possible implementation, the aforementioned robotic arm includes an execution end, and the output force of the aforementioned robotic arm is the output force of the aforementioned execution end.

[0008] Based on the first possible implementation of the first aspect of this application, in a second possible implementation, the constraint on the movement of the robotic arm, such that the output force of the robotic arm is close to and less than the constraint force, includes:

[0009] Control the movement of the robotic arm;

[0010] When the aforementioned actuator comes into contact with or is about to come into contact with the target object, the movement of the aforementioned actuator is controlled so that the output force of the aforementioned actuator gradually increases. The aforementioned output force is the interaction force between the aforementioned actuator and the aforementioned target object.

[0011] When the difference between the output force and the constraint force is not greater than a preset threshold, the motion of the execution end is constrained.

[0012] Based on the second possible implementation of the first aspect of this application, in the third possible implementation, the execution end is provided with a pressure sensor, which is used to collect the force between the execution end and the target object.

[0013] Based on the second possible implementation of the first aspect of this application, in the fourth possible implementation, the main robotic arm includes a gripping part, and the gripping part is provided with the force sensor.

[0014] Based on the fourth possible implementation of the first aspect of this application, in the fifth possible implementation, the force sensor is a tactile sensor.

[0015] Based on the fourth possible implementation of the first aspect of this application, in the sixth possible implementation, the aforementioned sensing signal is used to characterize the force information in each coordinate axis direction in the three-dimensional coordinate system, the aforementioned constraint force includes the constraint force in each coordinate axis direction, and the constraint force in each coordinate axis direction is mapped one-to-one with each of the aforementioned force information.

[0016] Correspondingly, when the difference between the output force and the constraint force is not greater than a preset threshold, the motion of the execution end is constrained, including:

[0017] When the difference between the output force in any coordinate axis direction and the constraint force in the corresponding coordinate axis direction is not greater than the preset threshold, the motion of the execution end is constrained.

[0018] Based on the second, third, fourth, fifth, or sixth possible implementations of the first aspect of this application, in the seventh possible implementation, the constraint on the movement of the execution end includes:

[0019] Control the aforementioned execution end to stop moving.

[0020] Based on the second, third, fourth, fifth, or sixth possible implementations of the first aspect of this application, in the eighth possible implementation, the aforementioned constraint function further includes:

[0021] When the difference between the output force and the constraint force is not greater than a preset threshold, a positioning indication signal is sent to the main robotic arm. The positioning indication signal is used to indicate that the actuator has moved into position.

[0022] A second aspect of this application provides a teleoperation method applied to a robotic arm, comprising:

[0023] The constraint force is received from the main robotic arm, wherein at least one part of the main robotic arm is provided with a force sensor, and the constraint force is generated based on the sensing signal of the force sensor.

[0024] It moves in response to the control commands of the main robotic arm.

[0025] During the movement, the movement of the aforementioned robotic arm is constrained so that the output force of the aforementioned robotic arm is close to and less than the aforementioned constraint force.

[0026] Based on the second aspect of this application, in a first possible implementation, the aforementioned robotic arm includes an execution end, and the output force of the aforementioned robotic arm is the output force of the aforementioned execution end.

[0027] Based on the first possible implementation of the second aspect of this application, in the second possible implementation, constraining the movement of the robotic arm so that the output force of the robotic arm is close to and less than the constraint force includes:

[0028] When the aforementioned actuator comes into contact with or is about to come into contact with the target object, the movement of the aforementioned actuator is controlled so that the output force of the aforementioned actuator gradually increases. The aforementioned output force is the interaction force between the aforementioned actuator and the aforementioned target object.

[0029] When the difference between the output force and the constraint force is not greater than a preset threshold, the motion of the execution end is constrained.

[0030] Based on the first possible implementation of the second aspect of this application, in the third possible implementation, the execution end is provided with a pressure sensor, which is used to collect the force between the execution end and the target object.

[0031] Based on the first possible implementation of the second aspect of this application, in the fourth possible implementation, the constraint force includes constraint forces in the directions of each coordinate axis in a three-dimensional coordinate system, and constraining the motion of the execution end when the difference between the output force and the constraint force is not greater than a preset threshold includes:

[0032] When the difference between the output force in any coordinate axis direction and the constraint force in the corresponding coordinate axis direction is not greater than the preset threshold, the motion of the execution end is constrained.

[0033] Based on the first, second, third, or fourth possible implementations of the second aspect of this application, in the fifth possible implementation, the constraint on the movement of the execution end includes:

[0034] Control the aforementioned execution end to stop moving.

[0035] Based on the first possible implementation, the second possible implementation, the third possible implementation, or the fourth possible implementation of the second aspect of this application, in the sixth possible implementation,

[0036] The above-mentioned teleoperation methods also include:

[0037] If the difference between the output force and the constraint force is determined to be no greater than a preset threshold, a position indication signal is sent to the main robotic arm. The position indication signal is used to indicate that the movement from the actuator has been completed.

[0038] A third aspect of this application provides a teleoperation method applied to a main robotic arm, wherein at least one part of the main robotic arm is provided with a force sensor, and the teleoperation method includes:

[0039] Constraint forces are generated based on the sensing signals from the aforementioned force sensor;

[0040] The constraint force is sent to the slave robot arm. The constraint force is used to instruct the slave robot arm to constrain its movement when a preset condition is met, so that the output force of the slave robot arm is close to and less than the constraint force. The preset condition includes that the difference between the output force and the constraint force is not greater than a preset threshold.

[0041] Based on the third aspect of this application, in a first possible implementation, the main robotic arm includes a gripping part, and the gripping part is provided with a force sensor.

[0042] Based on the first possible implementation of the third aspect of this application, in the second possible implementation, the force sensor is a tactile sensor.

[0043] Based on the first possible implementation of the third aspect of this application, in the third possible implementation, the aforementioned sensing signal is used to characterize the force information of each coordinate axis in the three-dimensional coordinate system, and the aforementioned constraint force includes the constraint force in each coordinate axis direction, and the constraint force in each coordinate axis direction is mapped one-to-one with each of the aforementioned force information.

[0044] The above-mentioned preset conditions include:

[0045] The difference between the output force in any coordinate axis direction and the constraint force in the corresponding coordinate axis direction shall not be greater than the preset threshold.

[0046] Based on the third aspect of this application, the first possible implementation of the third aspect of this application, the second possible implementation of the third aspect of this application, or the third possible implementation of the third aspect of this application, in the fourth possible implementation, after sending the aforementioned constraint force to the robotic arm, it further includes:

[0047] The system receives the positioning indication signal sent from the robotic arm, which indicates that the robotic arm has moved into position.

[0048] The fourth aspect of this application provides a robotic arm, including a memory and a processor, wherein the processor is configured to read and execute a computer program stored in the memory to implement steps as described in any possible implementation of the second or third aspect.

[0049] The fifth aspect of this application provides a computer storage medium storing a computer program that, when executed by a processor, implements the steps of the methods described in the first, second, and / or third aspects above.

[0050] The sixth aspect of this application provides a computer program product comprising a computer program that, when executed by one or more processors, implements the steps of the methods described in the first, second, and / or third aspects above.

[0051] In this application, at least one part of the main robotic arm is equipped with a force sensor, which generates a sensing signal in response to the operator's operation. After receiving the sensing signal, the main robotic arm can generate a constraint force based on the sensing signal, allowing the slave robotic arm to constrain its own movement based on the constraint force, thus preventing over-operation and operational accidents. The slave robotic arm may be configured with a constraint function, which constrains the movement of the slave robotic arm, controlling its output force to be close to but less than the constraint force. The constraint force can be considered as the maximum output force required for the slave robotic arm to complete a task. When the output force of the slave robotic arm is much less than the constraint force, it may prevent the slave robotic arm from completing the task, reducing the stability of the teleoperation system. When the output force of the slave robotic arm is greater than the constraint force, it may lead to over-operation of the slave robotic arm, causing operational accidents to the task target, reducing the safety of the teleoperation system. Therefore, when the output force of the slave robotic arm is close to but less than the constraint force, the slave robotic arm can complete the task without damaging the task target, improving the stability and safety of the teleoperation system. Attached Figure Description

[0052] To more clearly illustrate the method schemes of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a schematic diagram of the teleoperation system structure provided in the embodiments of this application;

[0054] Figure 2 This is a schematic flowchart of a teleoperation method provided in an embodiment of this application;

[0055] Figure 3 This is a schematic diagram illustrating the mapping relationship between applied forces and constraint forces provided in the embodiments of this application;

[0056] Figure 4 This is a schematic diagram showing the coordinate system between the master robotic arm and the slave robotic arm according to an embodiment of this application;

[0057] Figure 5 This is a flowchart illustrating another teleoperation method provided in an embodiment of this application;

[0058] Figure 6 This is a schematic diagram of the structure of a robotic arm provided in an embodiment of this application;

[0059] Figure 7 This is a schematic diagram of the structure of a main robotic arm provided in an embodiment of this application. Detailed Implementation

[0060] To make the objectives, methods, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] In this document, "and / or" is merely a way of describing the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Furthermore, in the description of the embodiments in this application, unless otherwise stated, "multiple" refers to two or more (i.e., more than two), and "at least one" or "one or more" refers to one, two, or more than two.

[0062] References to “one embodiment” or “some embodiments” as described in this specification mean that one or more embodiments of this application include a particular feature, structure, or characteristic described in connection with that embodiment. The terms “comprising,” “including,” “having,” and variations thereof mean “including, but not limited to,” unless otherwise specifically emphasized.

[0063] In practical applications, although operators can receive force feedback signals from the slave robotic arm during the operation of the master robotic arm to understand its current motion state, communication delays may prevent operators from obtaining real-time information about the slave robotic arm's actual motion state. In such cases, operators are prone to over-manipulating the slave robotic arm, potentially leading to accidents. To address this issue, this application proposes a teleoperation system. Once the slave robotic arm reaches its designated position, its movement is constrained, preventing over-manipulation and thus improving the stability and safety of the teleoperation system.

[0064] Figure 1 A remote operating system 100 is shown in one embodiment, including: a master robotic arm 110 and a slave robotic arm 120 controlled by the master robotic arm 110. A force sensor 111 is disposed on at least one portion of the master robotic arm 110, as shown in the figure. Figure 7 , Figure 7 A force sensor is installed at the end of the master robotic arm. The slave robotic arm 120 is configured with a constraint function, which includes: constraining the movement of the slave robotic arm 120 such that the output force of the slave robotic arm 120 is close to and less than the constraint force, wherein the constraint force comes from the master robotic arm 110 and is generated based on the sensing signal of the force sensor 111.

[0065] In a teleoperation system, the main robotic arm can be manually controlled by the operator and can be positioned in an environment that does not impede its movement. The secondary robotic arm, separate from the main robotic arm, is an independently operating robotic arm device controlled by the main robotic arm and can complete corresponding movements in response to the main robotic arm's control commands. Force sensors installed on the main robotic arm generate induced signals based on the operator's control of the main robotic arm, and the main robotic arm can then generate corresponding control commands and constraint forces based on these signals.

[0066] It should be noted that this embodiment does not limit the way the above control commands are generated. That is, the generation of the above control commands may also be independent of the above tactile sensor. The operator can control the main robotic arm in other ways (such as dragging the main robotic arm) to generate the above control commands.

[0067] To facilitate understanding, the generation of constraint forces is explained as follows: Assume an operator performs a control intention (a) on the main robotic arm to enable the secondary robotic arm to complete operation (b). During this process, a force sensor collects the corresponding force signal based on the operator's actions on the main robotic arm and determines the magnitude and direction of the applied force to generate a sensing signal. The main robotic arm then analyzes and processes this sensing signal to determine the operator's intention (a). To achieve the operation corresponding to control intention (a), the main robotic arm determines its own direction of movement, speed, and the force applied to the manipulated object based on this sensing signal. This applied force is the constraint force.

[0068] The aforementioned constraint force can be generated by the master robotic arm and then sent to the slave robotic arm. Since the operator's actual purpose in controlling the master robotic arm is to remotely operate the slave robotic arm, the slave robotic arm can constrain its own movement based on the constraint force. Therefore, after calculating the applied force of the master robotic arm based on the sensing signal, the master robotic arm can convert its own applied force into the constraint force for the slave robotic arm according to the mapping relationship between its applied force and the constraint force of the slave robotic arm, and then send the constraint force and the corresponding control command to the slave robotic arm. Assuming the above mapping relationship is 1:2, and the applied force of the master robotic arm is 5N, then the constraint force of the slave robotic arm, 10N, can be further calculated. Of course, if the above mapping relationship is 1:1, then the applied force calculated by the master robotic arm based on the sensing information does not need to be converted; that is, the applied force of the master robotic arm is the constraint force of the slave robotic arm.

[0069] In other specific embodiments, the mapping relationship between the applied force on the master robotic arm and the constraint force on the slave robotic arm can also be a non-linear mapping relationship. Various mapping relationships between applied forces and constraint forces can be referenced as follows: Figure 3 As shown, one of the mapping relationships can be selected to calculate the constraint force according to different application scenarios or needs.

[0070] After generating control commands and constraint forces, the master robotic arm can send both commands and constraint forces to the slave robotic arm simultaneously; alternatively, it can send the control commands to the slave robotic arm first, and then send the constraint forces to the slave robotic arm in response to a request from the slave robotic arm; or it can send the constraint forces to the slave robotic arm first, and then send the control commands to the slave robotic arm. That is, the timing of sending constraint forces is not limited in this embodiment.

[0071] After receiving control commands, the slave robotic arm can move in response to those commands. During movement, the slave robotic arm can constrain its own movement based on the constraint forces from the master robotic arm and its own configured constraint functions, so that the output force is close to but less than the constraint forces.

[0072] In this embodiment, the constraint force can be considered as the maximum output force required for the robotic arm to complete a task. When the output force of the robotic arm is much less than the constraint force, the robotic arm may be unable to complete the task, resulting in compromised task quality and reduced stability of the teleoperation system. When the output force of the robotic arm is greater than the constraint force, the robotic arm may be over-operated, causing operational accidents to the task target and reducing the safety of the teleoperation system. Therefore, the robotic arm can constrain its own movement to make the output force close to but less than the constraint force, thus completing the task without damaging the task target and improving the stability and safety of the teleoperation system.

[0073] In some embodiments, the slave robot arm may include an actuator, such as a gripper or a suction cup, and the constraint force of the slave robot arm is the output force of the actuator.

[0074] In some embodiments, in order to enable the robotic arm to grasp the timing of constraining its own output force and ensure that the robotic arm does not damage the task target while completing the task, the above-mentioned constraint function includes: controlling the movement of the robotic arm; when the actuator is in contact with or about to contact the target object, controlling the movement of the actuator to gradually increase the output force of the actuator; and constraining the movement of the actuator when the difference between the output force and the constraint force is not greater than a preset threshold.

[0075] During the process of controlling a robotic arm to perform a task, when the actuator contacts or is about to contact the target object, it indicates that the actuator has reached a position where it can perform the corresponding operation. At this time, the robotic arm can control the movement of the actuator, causing the force exerted by the actuator on the target object to gradually increase. This force is the output force of the actuator. To accurately determine when to constrain the movement of the actuator, a threshold can be preset. This threshold can be set based on experience. When setting it, it is necessary to ensure that the difference between the output force and the constraint force is not greater than this threshold, so that the actuator can complete the corresponding operation without being over-operated. In other words, the timing for constraining the movement of the robotic arm is when the difference between the output force and the constraint force is not greater than the preset threshold.

[0076] To facilitate understanding, let's take the example of a gripper picking up an egg to illustrate the above constraint function: Assume the robotic arm receives a constraint force F1 to pick up the egg, with a preset threshold of F. During the egg-picking process, the robotic arm first moves the gripper to a position where it can pick up the egg, i.e., to the position where the gripper contacts or is about to contact the egg. After this movement, to ensure the gripper can pick up the egg, the robotic arm controls the gripper to slowly retract, gradually increasing the output force F2, thus increasing the force acting on the egg. When F1-F2≤F, it means the output force F2 is close to but less than the constraint force, and the robotic arm can constrain the movement of the gripper, for example, by stopping the gripper to prevent excessive manipulation of the robotic arm from crushing the egg. When F1-F2>F, the robotic arm can control the gripper to continue retracting and continuing the picking operation until F1-F2≤F, at which point the movement of the gripper is constrained.

[0077] In some embodiments, to accurately determine the output force of the actuator, a pressure sensor can be provided at the actuator to collect the force between the actuator and the target object. Alternatively, a current loop sensor can be installed at the joint of the robotic arm to calculate the output force of the actuator from the current.

[0078] In some embodiments, to facilitate operator control of the main robotic arm and subsequently the slave robotic arm, the main robotic arm may be equipped with a gripping portion, on which a force sensor is mounted. This allows the operator to grip the gripping portion and control the main robotic arm. During operator control of the main robotic arm, the force sensor on the gripping portion can collect the force signal applied by the operator to the main robotic arm, generating a corresponding sensing signal. The main robotic arm, through this sensing signal, can then generate a corresponding constraint force.

[0079] In some embodiments, the force sensor described above is a sensor capable of acquiring force signals in at least one dimension. Preferably, it is a sensor capable of acquiring force signals in three or more dimensions, such as a tactile sensor.

[0080] In some embodiments, the sensing signal is used to characterize the force information in each coordinate axis direction in a three-dimensional coordinate system. The constraint force includes the constraint force in each coordinate axis direction, and the constraint force in each coordinate axis direction is mapped one-to-one with each force information. Based on this, in order to accurately determine the timing of the output force of the constraint execution end, the above-mentioned steps for constraining the output force include:

[0081] When the difference between the output force in any coordinate axis direction and the constraint force in the corresponding coordinate axis direction is not greater than a preset threshold, the motion of the execution end is constrained.

[0082] The tactile sensor installed on the grip is a sensor that can detect tactile pressure values ​​in three different dimensions. Accordingly, the sensing signal generated by the tactile sensor can characterize the force information in each coordinate axis direction in the three-dimensional coordinate system, and the force information of each coordinate axis can be mapped to a corresponding constraint force.

[0083] The mapping between constraint forces and force information can be a coaxial mapping within the same three-dimensional coordinate system. As an example, assuming force information and constraint forces are represented in a three-dimensional coordinate system A, where A contains three coordinate axes: x-axis, y-axis, and z-axis, the mapping relationship between constraint forces and force information can be expressed as: x-axis force information - x-axis constraint force, y-axis force information - y-axis constraint force, and z-axis force information - z-axis constraint force.

[0084] Of course, besides coaxial mapping within the same 3D coordinate system, the mapping method can also be coaxial mapping across different coordinate systems. (See reference...) Figure 4 As an example only, assume that the three-dimensional coordinate system A1 (x1 axis, y1 axis, and z1 axis) where the force information collected by the main robotic arm is established based on the tactile sensor. Then, the force information on the three coordinate axes can be represented as the force information on the x1 axis, the force information on the y1 axis, and the force information on the z1 axis. The three-dimensional coordinate system A2 (x2 axis, y2 axis, and z2 axis) where the constraint forces are located is established based on the robotic arm. The constraint forces on the three coordinate axes are the constraint forces on the x2 axis, the constraint forces on the y2 axis, and the constraint forces on the z2 axis. If the distance between the corresponding coordinate axes of the three-dimensional coordinate systems A1 and A2 is determined... The conversion relationships are as follows: x1 axis * x' = x2 axis, y1 axis * y' = y2 axis, and z1 axis * z' = z2 axis. After generating the x1 axis constraint force, y1 axis constraint force, and z1 axis constraint force based on the x1 axis force information, y1 axis force information, and z1 axis information, the constraint forces in the three-dimensional coordinate system A1 can be transformed to the three-dimensional coordinate system A2. Based on this, the mapping relationship between force information and constraint force can be expressed as: x1 axis force information - x1 axis constraint force * x', y1 axis force information - y1 axis constraint force * y', and z1 axis force information - z1 axis constraint force * z'.

[0085] It is understandable that the above mapping method can also perform non-coaxial mapping in different coordinate systems. The difference between this mapping method and coaxial mapping in different coordinate systems is that when determining the conversion relationship between two coordinate axes, this mapping method is based on a preset coordinate axis correspondence. For ease of understanding, let's take the two three-dimensional coordinate systems mentioned above as an example: Assuming that the preset coordinate axis correspondence of the two three-dimensional coordinate systems is x1 axis - y2 axis, y1 axis - z2 axis, and z1 axis - x2 axis, the conversion relationships between the corresponding coordinate axes of the two three-dimensional coordinate systems can be determined as x1 axis * a = y2 axis, y1 axis * b = z2 axis, and z1 axis * c = x2 axis, respectively. Therefore, the mapping relationship between force information and constraint force can be expressed as: x1 axis force information - x1 axis constraint force * a, y1 axis force information - y1 axis constraint force b, z1 axis force information - z1 axis constraint force * c.

[0086] As shown above, constraint forces include those along each coordinate axis. Therefore, when determining when to constrain the motion of the actuator, a condition can be set that the difference between the output force and the constraint force along any coordinate axis is no greater than a preset threshold. For example, this condition could be that the motion of the actuator is constrained when the difference between the output force and the constraint force along the x-axis is no greater than the preset threshold, or the difference between the output force and the constraint force along the y-axis is no greater than the preset threshold, or the difference between the output force and the constraint force along the z-axis is no greater than the preset threshold. In other words, when the difference between the output force and the constraint force along any one of the three coordinate axes is no greater than the preset threshold, it can be determined that the robotic arm has reached its designated position. Thus, by constraining the motion of the actuator, the output force along the corresponding coordinate axis can be controlled to be close to and less than the constraint force along that axis, thereby preventing over-operation of the robotic arm.

[0087] Of course, the above conditions can also be set according to specific application scenarios. For example, the condition can be that the difference between the output force in any two of the three coordinate axes and the constraint force in the corresponding coordinate axis is not greater than a preset threshold; or the condition can be that the difference between the output force in all three coordinate axes and the constraint force in the corresponding coordinate axis is not greater than a preset threshold. In this embodiment, no limitation is made.

[0088] It should be noted that the aforementioned "corresponding coordinate axis direction" not only refers to the scenario of the same coordinate axis direction, but can also include the scenario of "customizing the correspondence between the coordinate axis direction of the output force and the coordinate axis direction of the constraint force." In this scenario, "corresponding coordinate axis direction" means the coordinate axis direction determined based on this correspondence. For example, the correspondence between the constraint force in the x-axis direction and the output force in the y-axis direction can be customized. Accordingly, the constraint force in the corresponding coordinate axis direction of the "output force in the y-axis direction" is the constraint force in the x-axis direction.

[0089] In some embodiments, constraining the motion of the actuator can include controlling the actuator to stop moving. It is understood that if the actuator of the robotic arm consists of multiple moving parts, when controlling the actuator to stop moving, it can be determined which coordinate axis(s) has an output force (or several axes) whose difference between the output force and the constraint force in the corresponding coordinate axis(s) is not greater than a preset threshold, and then the moving parts are controlled to stop moving in the corresponding coordinate axis(s).

[0090] Taking the egg gripping example again, assuming that the gripper consists of three gripping parts, if the output force of the gripper in the y-axis direction is not less than the constraint force in the y-axis direction (let the constraint force in the corresponding coordinate axis direction of the "output force in the y-axis direction" be the constraint force in the y-axis direction), then the three gripping parts can be controlled to stop moving in the y-axis direction. Correspondingly, the three gripping parts can continue to move in the x-axis direction and the z-axis direction.

[0091] Optionally, in specific application scenarios, when the retraction operation of the actuator will not cause damage to the target object, the aforementioned movement of the robotic arm can also be a retraction operation. For example, if an egg is simply held in a fixed position (the egg has a support and is not suspended), and the force detection unit detects that the force in the x-axis direction is not less than the constraint force in the x-axis direction (let's assume that the constraint force in the x-axis direction is the constraint force in the x-axis direction corresponding to the "output force in the x-axis direction"), the moving parts in the x-axis direction can be controlled to stop moving, and a retraction operation can be performed.

[0092] The purpose of the retraction operation is to control the actuator to complete the clamping operation while avoiding damage to the manipulated object. That is, the retraction operation does not return the force in the x-axis direction to its initial state. Therefore, the retraction operation can be performed according to a preset ratio. For example, if a force of 5N is applied, the retraction operation can reduce the force by 5%, that is, reduce the currently applied force from the robotic arm and maintain the force at 4.75N.

[0093] In some embodiments, the above-mentioned constraint function further includes:

[0094] When the difference between the output force and the constraint force is not greater than the preset threshold, a position indication signal is sent to the main robotic arm. The position indication signal is used to indicate that the actuator has moved into position.

[0095] During the process of an operator controlling a slave robotic arm based on the master robotic arm, the slave robotic arm can generate corresponding force feedback signals based on its current movement and provide real-time feedback to the operator. This allows the operator to experience the process firsthand and control the slave robotic arm more precisely through the master robotic arm. However, during the feedback process, communication delays may occur, causing the operator to perceive a different operational state than the actual state, thus hindering precise control. To indicate to the operator that the slave robotic arm has completed its movement, the slave robotic arm's constraint function can send a position indication signal to the master robotic arm when the difference between the output force and the constraint force is no greater than a preset threshold. This signal indicates that the slave actuator has reached its designated position. The master robotic arm's presentation of the position indication signal may include, but is not limited to, the illumination of a corresponding indicator light or vibration at a preset frequency.

[0096] Another embodiment of this application also provides a teleoperation method, such as Figure 2 As shown, this teleoperation method is applied to a robotic arm, including:

[0097] Step 210: Receive the constraint force sent by the main robotic arm.

[0098] The constraint force received from the robotic arm originates from the main robotic arm. At least one part of the main robotic arm is equipped with a force sensor; for example, the main robotic arm includes a gripping part, on which a force sensor is mounted. Preferably, this force sensor can be a tactile sensor. The constraint force is generated by the main robotic arm based on the sensing signal from the force sensor. The process of generating the constraint force can be referred to the relevant description in the foregoing embodiments, and will not be repeated here.

[0099] Step 220: Move in response to the control commands of the main robotic arm.

[0100] In addition to receiving constraint forces, the robotic arm can also receive control commands from the main robotic arm and move in response to these commands. The method for generating these control commands can be found in the descriptions in the preceding embodiments, and will not be repeated here.

[0101] Step 230: During the movement, constrain the movement of the robotic arm so that the output force of the robotic arm is close to and less than the constraint force.

[0102] During the movement of the robotic arm, the constraint force can be considered as the maximum force required by the robotic arm to complete a certain operation. When the output force of the robotic arm is much less than this constraint force, it may prevent the robotic arm from completing the task, compromising task quality and reducing the stability of the teleoperation system. Conversely, when the output force of the robotic arm exceeds this constraint force, it may lead to over-manipulation of the robotic arm, causing operational accidents to the task target and reducing the reliability of the teleoperation system. Therefore, to improve the stability and reliability of the teleoperation system, the movement of the robotic arm can be constrained so that its output force is close to but less than the constraint force, ensuring that the robotic arm completes the task without damaging the task target.

[0103] In some embodiments, the slave robot arm may include an actuator, such as a gripper or a suction cup, and the constraint force of the slave robot arm is the output force of the actuator.

[0104] In some embodiments, in order to enable the robotic arm to grasp the timing of constraining its own output force and ensure that the robotic arm is not over-operated while completing the corresponding operation, the above-mentioned constraint function includes: controlling the movement of the robotic arm; when the actuator is in contact with or about to contact the target object, controlling the movement of the actuator to gradually increase the output force of the actuator; and constraining the movement of the actuator when the difference between the output force and the constraint force is not greater than a preset threshold.

[0105] The movement of the constraint actuator is performed after the actuator completes the operation corresponding to the control command. Accordingly, before this, the robotic arm can first move the actuator to a position where it can perform the corresponding operation, facilitating control of the actuator to complete the operation. For example, if it is determined that the actuator is in contact with or about to contact the target object, the actuator can be further controlled to move, and the movement of the actuator is constrained when the difference between the output force of the actuator and the constraint force exceeds a preset threshold. This teleoperation method ensures that the actuator completes the corresponding operation without damaging the target object due to excessive manipulation.

[0106] As an example, assuming the target object is a sheet-like object and the actuator is a suction cup, when the robotic arm moves to the point where the suction cup is about to contact the sheet-like object on the worktable, the suction cup can be controlled to gradually increase its suction force, causing the sheet-like object to be attracted to the suction cup. When the difference between the suction force and the constraint force exceeds a preset threshold, the movement of the suction cup can be constrained. For example, the suction force of the suction cup can be controlled to no longer increase, so that the sheet-like object can be picked up by the suction cup and removed from the worktable without being damaged due to excessive suction force.

[0107] In some embodiments, to accurately determine the output force of the actuator, a pressure sensor can be provided at the actuator to collect the force between the actuator and the target object. Alternatively, a current loop sensor can be installed at the joint of the robotic arm to calculate the output force of the actuator from the current.

[0108] In some embodiments, the constraint force includes constraint forces in each coordinate axis direction in a three-dimensional coordinate system. When the difference between the output force and the constraint force is not greater than a preset threshold, the motion of the execution end is constrained, including: when the difference between the output force in any coordinate axis direction and the constraint force in the corresponding coordinate axis direction is not greater than a preset threshold, the motion of the execution end is constrained.

[0109] In one application scenario, the constraint force can be a three-dimensional force, which includes constraint forces along the three coordinate axes in a three-dimensional coordinate system. To determine when to constrain the motion of the actuator, the output force along each coordinate axis can be subtracted from the corresponding constraint force. Based on the relationship between the difference and a preset threshold, it can be determined whether to constrain the motion of the actuator, ensuring that the output force along each coordinate axis is close to and less than the corresponding constraint force.

[0110] As an example only, when the difference between the output force in the x-axis direction and the constraint force in the x-axis direction is not greater than a preset threshold, the actuator can be controlled to stop moving in the x-axis direction.

[0111] It should be noted that the "corresponding coordinate axis direction" mentioned above refers not only to the scenario of the same coordinate axis direction, but also to the scenario of "the correspondence between the coordinate axis direction of the force detected by the custom force detection unit and the coordinate axis direction of the constraint force". For specific scenarios, please refer to the relevant descriptions in the foregoing embodiments, which will not be repeated here.

[0112] In some embodiments, constraining the movement of the robotic arm includes stopping the movement of the robotic arm.

[0113] Specifically, the motion constraint method for the robotic arm can be referred to the relevant description in the foregoing embodiments, and will not be repeated here.

[0114] In some embodiments, the above-mentioned constraint function further includes:

[0115] When the difference between the output force and the constraint force is not greater than the preset threshold, a position indication signal is sent to the main robotic arm. The position indication signal is used to indicate that the actuator has moved into position.

[0116] After the slave robot arm determines that the movement of the actuator is in place, in addition to constraining the movement of the actuator, it can also send a positioning indication message to the master robot arm to notify the operator that the movement of the slave robot arm has reached the desired position. This allows the operator to control the slave robot arm to complete a new operation based on the positioning indication message. The interpretation of the positioning indication message can be found in the descriptions in the preceding embodiments, and will not be repeated here.

[0117] One embodiment of this application also provides another teleoperation method, such as Figure 5 As shown, this teleoperation method is applied to a main robotic arm, where at least one part of the main robotic arm is equipped with a force sensor. The teleoperation method includes:

[0118] Step 310: Generate constraint force based on the sensing signal of the force sensor.

[0119] The force sensor mounted on the main robotic arm can generate corresponding sensing signals based on the operator's actions on the main robotic arm. To ensure that the slave robotic arm is not over-operated, the main robotic arm can generate corresponding constraint forces based on these sensing signals to constrain the movement of the slave robotic arm. The method for generating the constraint forces can be referred to the relevant descriptions in the foregoing embodiments, and will not be repeated here.

[0120] In some embodiments, the main robotic arm may include a gripper on which a force sensor is disposed. The force sensor is a sensor capable of acquiring force signals in at least one dimension, and preferably, it may be a sensor capable of acquiring force signals in three or more dimensions, such as a tactile sensor.

[0121] Step 320: Send constraint force to the robotic arm.

[0122] After generating the constraint force, the master robotic arm can send the constraint force to the slave robotic arm to control the slave robotic arm to determine whether its own movement meets the preset conditions based on the constraint force, that is, whether the difference between the output force and the constraint force is not greater than a preset threshold. When the difference between the output force and the constraint force is not greater than the preset threshold, the slave robotic arm can constrain its own movement so that the output force of the slave robotic arm is close to and less than the constraint force.

[0123] To better understand how a robotic arm can constrain its own movement, let's take an example: When the robotic arm determines that its movement meets the aforementioned preset conditions based on the constraint force, it can choose to stop moving or perform a preset operation (such as a rollback operation) to constrain its own movement, depending on the specific application scenario.

[0124] The timing and method of sending the constraint force can be referred to the relevant descriptions in the foregoing embodiments, and will not be repeated here.

[0125] In some embodiments, the sensing signal is used to characterize the force information of each coordinate axis in the three-dimensional coordinate system. The constraint force includes the constraint force in each coordinate axis direction, and the constraint force in each coordinate axis direction is mapped one-to-one with each force information.

[0126] The preset conditions include:

[0127] The difference between the output force in any coordinate axis direction and the constraint force in the corresponding coordinate axis direction shall not exceed a preset threshold.

[0128] To achieve precise control over the slave robotic arm, the master robotic arm is equipped with force sensors capable of measuring pressure values ​​in three dimensions, such as tactile sensors. Correspondingly, the sensing signals generated by these tactile sensors can characterize the force information along each coordinate axis in a three-dimensional coordinate system. Similarly, constraint forces are also three-dimensional forces, meaning they can include constraint forces along the three coordinate axes in a three-dimensional coordinate system. The mapping relationship between force information and constraint forces can be referred to the relevant descriptions in the preceding embodiments, and will not be repeated here.

[0129] When determining whether the current movement of the robotic arm meets the preset conditions, the difference between the output force in any coordinate axis direction and the constraint force in the corresponding coordinate axis direction is not greater than a preset threshold as a preset condition. When the movement of the robotic arm meets the preset condition, the movement of the robotic arm is constrained to ensure that the robotic arm can complete the corresponding operation and that the operated object will not be damaged due to excessive movement of the robotic arm.

[0130] In certain applications, the robot arm can only be considered in place if the output forces in two or more coordinate axes are close to and less than the constraint forces in the corresponding coordinate axes. In this case, to accurately constrain the movement of the robot arm, a preset condition can be set as the condition that the output forces in any two or three coordinate axes and the constraint forces in the corresponding coordinate axes are all no greater than a preset threshold.

[0131] It should be noted that the "corresponding coordinate axis direction" mentioned above refers not only to the scenario of the same coordinate axis direction, but also to the scenario of "the correspondence between the coordinate axis direction of the custom output force and the coordinate axis direction of the constraint force". For specific scenarios, please refer to the relevant descriptions in the aforementioned embodiments, which will not be repeated here.

[0132] In some embodiments, after step 320 above, the method further includes receiving a positioning indication signal sent from the robotic arm, the positioning indication signal being used to indicate that the robotic arm has moved into position.

[0133] After the robotic arm determines that its movement has met the preset conditions, it can send a positioning indication signal to the main robotic arm based on the judgment result. After receiving the positioning indication signal, the main robotic arm can display the positioning indication signal, such as lighting up the corresponding indicator light or vibrating at a preset frequency, to remind the operator that the robotic arm has moved into position.

[0134] For example, such as Figure 6 As shown, this application embodiment also provides a robotic arm 4, including: a memory 41 and a processor 43, the processor 43 being used to read and execute a computer program 42 stored in the memory 41 to implement the steps of the above-described teleoperation method.

[0135] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0136] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the method. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0137] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. Furthermore, integrated modules / units, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in computer-readable media may be appropriately added to or subtracted from the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, computer-readable media may not include electrical carrier signals and telecommunication signals, in accordance with legislation and patent practice.

[0138] The above embodiments are only used to illustrate the system and method of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the method solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the method features. Such modifications or substitutions do not cause the essence of the corresponding method solutions to deviate from the spirit and scope of the method solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A teleoperation system, comprising: A master robotic arm and a slave robotic arm controlled by the master robotic arm, characterized in that at least one part of the master robotic arm is provided with a force sensor; The slave robotic arm is equipped with a constraint function, which includes: constraining the movement of the slave robotic arm so that the output force of the slave robotic arm is close to and less than the constraint force, wherein the constraint force comes from the master robotic arm and is generated based on the sensing signal of the force sensor; The slave robotic arm includes an actuator, and the output force of the slave robotic arm is the output force of the actuator. The constraint on the movement of the robotic arm, such that the output force of the robotic arm is close to and less than the constraint force, includes: Control the movement of the robotic arm; When the actuator comes into contact with or is about to come into contact with the target object, the actuator is controlled to move, so that the output force of the actuator gradually increases. The output force is the force between the actuator and the target object. When the difference between the output force and the constraint force is not greater than a preset threshold, the motion of the execution end is constrained.

2. The teleoperation system according to claim 1, characterized in that, The actuator is equipped with a pressure sensor, which is used to collect the force between the actuator and the target object.

3. The teleoperation system according to claim 1, characterized in that, The main robotic arm includes a gripping part, and the gripping part is equipped with the force sensor.

4. The teleoperation system according to claim 3, characterized in that, The force sensor is a tactile sensor.

5. The teleoperation system according to claim 3, characterized in that, The sensing signal is used to characterize the force information in each coordinate axis direction in the three-dimensional coordinate system. The constraint force includes the constraint force in each coordinate axis direction, and the constraint force in each coordinate axis direction is mapped one-to-one with each of the force information. Correspondingly, when the difference between the output force and the constraint force is not greater than a preset threshold, the motion of the actuator is constrained, including: When the difference between the output force in any coordinate axis direction and the constraint force in the corresponding coordinate axis direction is not greater than the preset threshold, the motion of the execution end is constrained.

6. The teleoperation system according to any one of claims 1-5, characterized in that, The constraint on the motion of the execution end includes: Control the actuator to stop moving.

7. The teleoperation system according to any one of claims 1-5, characterized in that, The constraint function also includes: When the difference between the output force and the constraint force is not greater than a preset threshold, a position indication signal is sent to the main robotic arm. The position indication signal is used to indicate that the actuator has moved into position.

8. A teleoperation method applied to a robotic arm, characterized in that, include: The system receives constraint forces sent by the main robotic arm, wherein at least one part of the main robotic arm is equipped with a force sensor, and the constraint force is generated based on the sensing signal of the force sensor. It moves in response to the control commands of the main robotic arm; During the movement, the movement of the slave robot arm is constrained so that the output force of the slave robot arm is close to and less than the constraint force. The slave robotic arm includes an actuator, and the output force of the slave robotic arm is the output force of the actuator. The constraint on the movement of the robotic arm, such that the output force of the robotic arm is close to and less than the constraint force, includes: When the actuator comes into contact with or is about to come into contact with the target object, the actuator is controlled to move, so that the output force of the actuator gradually increases. The output force is the force between the actuator and the target object. When the difference between the output force and the constraint force is not greater than a preset threshold, the motion of the execution end is constrained.

9. The teleoperation method according to claim 8, characterized in that, The actuator is equipped with a pressure sensor, which is used to collect the force between the actuator and the target object.

10. The teleoperation method according to claim 8, characterized in that, The constraint force includes constraint forces along each coordinate axis in a three-dimensional coordinate system. When the difference between the output force and the constraint force is not greater than a preset threshold, the motion of the execution end is constrained, including: When the difference between the output force in any coordinate axis direction and the constraint force in the corresponding coordinate axis direction is not greater than the preset threshold, the motion of the execution end is constrained.

11. The teleoperation method according to any one of claims 8-10, characterized in that, The constraint on the motion of the execution end includes: Control the actuator to stop moving.

12. The teleoperation method according to any one of claims 8-10, characterized in that, The teleoperation method further includes: If the difference between the output force and the constraint force is determined to be no greater than a preset threshold, a position indication signal is sent to the main robotic arm. The position indication signal is used to indicate that the movement of the actuator has reached the position.

13. A teleoperation method applied to a main robotic arm, characterized in that, At least one part of the main robotic arm is equipped with a force sensor, and the teleoperation method includes: A constraint force is generated based on the sensing signal from the force sensor; The constraint force is sent to the slave robot arm, which is used to instruct the slave robot arm to constrain its movement when a preset condition is met, so that the output force of the slave robot arm is close to and less than the constraint force. The preset condition includes that the difference between the output force and the constraint force is not greater than a preset threshold. The slave robotic arm includes an actuator, and the output force of the slave robotic arm is the output force of the actuator. The constraint on the movement of the robotic arm, such that the output force of the robotic arm is close to and less than the constraint force, includes: Control the movement of the robotic arm; When the actuator comes into contact with or is about to come into contact with the target object, the actuator is controlled to move, so that the output force of the actuator gradually increases. The output force is the force between the actuator and the target object. When the difference between the output force and the constraint force is not greater than a preset threshold, the motion of the execution end is constrained.

14. The teleoperation method according to claim 13, characterized in that, The main robotic arm includes a gripping part, and the gripping part is equipped with a force sensor.

15. The teleoperation method according to claim 14, characterized in that, The force sensor is a tactile sensor.

16. The teleoperation method according to claim 14, characterized in that, The sensing signal is used to characterize the force information of each coordinate axis in the three-dimensional coordinate system. The constraint force includes the constraint force in each coordinate axis direction, and the constraint force in each coordinate axis direction is mapped one-to-one with each of the force information. The preset conditions include: The difference between the output force in any coordinate axis direction and the constraint force in the corresponding coordinate axis direction is not greater than the preset threshold.

17. The teleoperation method according to any one of claims 13-16, characterized in that, After sending the constraint force to the robotic arm, the method further includes: The system receives a positioning indication signal sent from the robotic arm, which indicates that the robotic arm has moved into position.

18. A robotic arm, characterized in that, include: A memory and a processor, the processor being configured to read and execute a computer program stored in the memory to implement the steps of the method as claimed in any one of claims 8-12 or to implement the steps of the method as claimed in any one of claims 13-17.

19. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 8-17.

Images