Manipulator opening and closing control method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a manipulator opening and closing control method and device, electronic equipment and a storage medium. The method comprises the steps that relative force detected by a sensor installed on the mechanical arm is obtained, and the relative force is generated under the condition that the sensor makes contact with a target object capable of being grabbed by the mechanical arm; according to the change information of the relative force, the mechanical arm is controlled to execute operation, and the operation comprises opening operation, maintaining operation or closing operation. According to the technical scheme, automatic control over opening and closing of the mechanical arm is achieved through the change of the relative force between the sensor and the target object, so that a wearing object of the mechanical arm only needs to give a touch signal for triggering control, and follow-up opening and closing operation can be automatically executed under control; therefore, the problem that the opening and closing control performance of the manipulator cannot be guaranteed due to the influence of factors such as the arm posture, the muscle state, the arm muscle amount and the brain attention is solved, and the opening and closing control stability of the manipulator is guaranteed.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of manipulators, and in particular, to a method, device, electronic device, and storage medium for controlling the opening and closing of a manipulator. Background Art

[0002] In recent years, by controlling the opening and closing of manipulators such as prostheses to assist people in need with grasping operations required for daily activities, it has been increasingly widely used.

[0003] In terms of prosthesis control, the user's motion intention can be decoded from motor nerve signals, and the prosthesis can be manipulated based on this motion intention to complete corresponding operations, thereby realizing human-machine interaction. Among them, electromyographic signals are the most commonly used signal sources because they have the advantages of simple acquisition, non-invasiveness, and rich motion information. However, the prosthesis control scheme based on the decoding of motion intention from electromyographic signals is easily affected by factors such as arm posture and muscle fatigue, resulting in inaccurate decoding of motion intention and thus affecting the control performance; moreover, when the user's arm muscles are insufficient, it is also impossible to provide sufficient signal sources, which will also affect the control performance.

[0004] To solve the above problems, some research scholars have proposed a solution to install tactile sensors on prostheses and feedback the tactile signals detected by the tactile sensors to the human body through vibration and / or electrical stimulation, etc., so as to realize the closed-loop control of the prosthesis. However, as a feedback information, the tactile signal still has a difference from the real human feeling. In prosthesis control, the brain needs to provide additional cognitive attention to process the feedback information in order to improve the control performance. Obviously, this implementation scheme requires the user to maintain a high degree of concentration. That is, when the user's attention drops, it will also affect the control performance. Summary of the Invention

[0005] Embodiments of the present invention provide a method, device, electronic device, and storage medium for controlling the opening and closing of a manipulator, which solves the problem that the opening and closing control performance of the manipulator cannot be guaranteed.

[0006] According to one aspect of the present invention, there is provided a method for controlling the opening and closing of a manipulator, which may include:

[0007] Obtain the relative force detected by a sensor installed on the manipulator, where the relative force is generated when the sensor contacts a target object that can be grasped by the manipulator;

[0008] Control the manipulator to perform an operation according to the change information of the relative force, where the operation includes an opening operation, a holding operation, or a closing operation.

[0009] According to another aspect of the present invention, there is provided a device for controlling the opening and closing of a manipulator, which may include:

[0010] A relative force acquisition module for acquiring the relative force detected by a sensor installed on a robotic arm, where the relative force is generated when the sensor comes into contact with a target object that can be grasped by the robotic arm;

[0011] A robotic arm operation control module for controlling the robotic arm to perform operations according to the change information of the relative force, where the operations include an opening operation, a holding operation, or a closing operation.

[0012] According to another aspect of the present invention, there is provided an electronic device, which may include:

[0013] At least one processor; and

[0014] A memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, it enables the at least one processor to implement the robotic arm opening and closing control method provided in any embodiment of the present invention.

[0016] According to another aspect of the present invention, there is provided a computer-readable storage medium, on which computer instructions are stored, and when the computer instructions are used to be executed by a processor, they implement the robotic arm opening and closing control method provided in any embodiment of the present invention.

[0017] The technical solution of the embodiment of the present invention is directed to a sensor installed on a robotic arm and a target object that can be grasped by the robotic arm. When the sensor comes into contact with the target object, the relative force between the sensor and the target object detected by the sensor can be acquired; further, according to the change information of the relative force, the robotic arm is controlled to perform an opening operation, a holding operation, or a closing operation to assist the wearer of the robotic arm to grasp or put down the target object. The above technical solution realizes the automatic control of the opening and closing of the robotic arm through the change of the relative force between the sensor and the target object, so that the wearer only needs to control the contact between the robotic arm and the target object, that is, only needs to give a tactile signal for triggering control, and the subsequent opening and closing operations can be automatically executed under control, that is, the wearer does not need to control through the arm or the brain, etc. Thus, the problem that the opening and closing control performance of the robotic arm cannot be guaranteed due to factors such as arm posture, muscle state, arm muscle mass, and brain attention is solved, and the stability of the robotic arm opening and closing control is ensured.

[0018] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. Description of the Drawings

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings required in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0020] Figure 1 is a flowchart of a method for controlling the opening and closing of a manipulator according to an embodiment of the present invention;

[0021] Figure 2 is a schematic diagram of the relative force in a method for controlling the opening and closing of a manipulator according to an embodiment of the present invention;

[0022] Figure 3 is a flowchart of another method for controlling the opening and closing of a manipulator according to an embodiment of the present invention;

[0023] Figure 4 is a flowchart of yet another method for controlling the opening and closing of a manipulator according to an embodiment of the present invention;

[0024] Figure 5 is a schematic diagram of the range in a method for controlling the opening and closing of a manipulator according to an embodiment of the present invention;

[0025] Figure 6 is a flowchart of still another method for controlling the opening and closing of a manipulator according to an embodiment of the present invention;

[0026] Figure 7 is a schematic diagram of the state transition of a finite state machine of an optional example in still another method for controlling the opening and closing of a manipulator according to an embodiment of the present invention;

[0027] Figure 8 is a structural block diagram of a device for controlling the opening and closing of a manipulator according to an embodiment of the present invention;

[0028] Figure 9 is a schematic structural diagram of an electronic device for implementing the method for controlling the opening and closing of a manipulator in the embodiments of the present invention. Detailed implementation manners

[0029] In order to enable those skilled in the art to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The same is true for "target", "original", etc., which will not be elaborated here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0031] Figure 1 FIG. is a flowchart of a method for controlling the opening and closing of a robotic arm provided by an embodiment of the present invention. This embodiment is applicable to the situation of controlling the opening and closing of a robotic arm. This method can be executed by a robotic arm opening and closing control device provided by an embodiment of the present invention. The device can be implemented in software and / or hardware, and can be integrated on an electronic device, which can be various user terminals or servers.

[0032] See Figure 1 , the method of the embodiment of the present invention specifically includes the following steps:

[0033] S110. Obtain the relative force detected by a sensor installed on the robotic arm, where the relative force is generated when the sensor contacts a target object that can be grasped by the robotic arm.

[0034] Among them, the target object can be understood as an object that can be grasped by the robotic arm. When the robotic arm grasps the target object, that is, when the robotic arm contacts the target object, a relative force (i.e., a relative force) is generated between the two, and this relative force can change with the transfer of the grasping process and the putting-down process of the robotic arm.

[0035] The sensor can be understood as a sensor installed on the robotic arm that can be used to directly or indirectly detect force. Considering the application scenarios that the embodiments of the present invention may involve, optionally, the sensor can be, for example, a tactile sensor or a pressure sensor (especially a three-axis pressure sensor), etc., which is not specifically limited here; additionally, the sensor can be installed at parts of the robotic arm's fingertips, knuckles or palm that can contact the target object. When the robotic arm contacts the target object, further when the sensor contacts the target object, a relative force is generated between the sensor and the target object, and at this time the relative force can be detected by the sensor.

[0036] In practical applications, optionally, the relative force may be a normal force perpendicular to the sensor. Taking the z-axis in the xyz axes as perpendicular to the sensor as an example, the normal force may be Figure 2 Fz shown in Figure 2 ; the tangential force may also be a tangential force parallel to the sensor. Taking the x-axis / z-axis in the xyz axes as parallel to the sensor as an example, the tangential force may be

[0037] Fx on the x-axis or Fy on the y-axis shown in

[0038] It should be noted that, for the sake of vivid description, Fx, Fy, and Fz in the above examples will also be continued to be used hereinafter, but this is only an example and not a specific limitation.

[0038] Obtain the relative force detected by the sensor.

[0039] Combined with the application scenarios that the embodiments of the present invention may involve, optionally, when the sensor is a tactile sensor, obtaining the relative force detected by the sensor installed on the robotic arm may include: obtaining the tactile signal detected by the tactile sensor installed on the robotic arm; then, based on the tactile signal, obtaining the relative force between the tactile sensor and the target object.

[0040] The application of the tactile sensor realizes the accurate detection of the relative force. On this basis, combined with the subsequent steps, the change of the relative force can be characterized based on the change of the tactile signal, thereby realizing the accurate control of the opening and closing of the robotic arm.

[0041] S120. Control the robotic arm to perform operations according to the change information of the relative force, where the operations include an opening operation, a holding operation, or a closing operation.

[0042] Among them, the change information can characterize the change situation of the relative force, for example, it may be at least one of the magnitude change information, the type change information, and the direction change information, etc., which is not specifically limited herein.

[0043] It can be known through experimental verification that during the process of the robotic arm grasping and releasing the target object, the relative force between the robotic arm and the target object also changes, that is, the change of the relative force can reflect the transfer of the grasping process and the releasing process. Therefore, by detecting the relative force in real time through the sensor, the change information of the relative force can be obtained, and then the robotic arm can be controlled to perform operations according to the change information, such as performing a closing operation to grasp the target object, performing a holding operation to hold the target object, or performing an opening operation to release the target object.

[0044] The technical solution of the embodiment of the present invention is directed to a sensor installed on a robotic arm and a target object that can be grasped by the robotic arm. When the sensor is in contact with the target object, the relative force between the sensor and the target object detected by the sensor can be obtained. Further, according to the change information of the relative force, the robotic arm is controlled to perform an opening operation, a holding operation, or a closing operation to assist the wearer of the robotic arm to grasp or release the target object. The above technical solution realizes the automatic control of the opening and closing of the robotic arm through the change of the relative force between the sensor and the target object, so that the wearer only needs to control the contact between the robotic arm and the target object, that is, only needs to give a tactile signal for triggering control, and the subsequent opening and closing operations can be automatically executed under control, that is, there is no need for the wearer to control through the arm or the brain, etc. Thus, the problem that the opening and closing control performance of the robotic arm cannot be guaranteed due to factors such as arm posture, muscle state, arm muscle mass, and brain attention is solved, and the stability of the opening and closing control of the robotic arm is ensured.

[0045] Figure 3 It is a flowchart of another method for controlling the opening and closing of a robotic arm provided in the embodiment of the present invention. This embodiment is optimized based on the above technical solutions. In this embodiment, optionally, controlling the robotic arm to perform an operation according to the change information of the relative force includes: controlling the robotic arm to perform an operation according to the change information of the relative force and static information, where the static information includes at least one of magnitude, type, and direction. Among them, the explanations of the same or corresponding terms in the above embodiments will not be repeated here.

[0046] See Figure 3 , the method of this embodiment may specifically include the following steps:

[0047] S210. Obtain the relative force detected by a sensor installed on the robotic arm, where the relative force is generated when the sensor is in contact with a target object that can be grasped by the robotic arm.

[0048] S220. Control the robotic arm to perform an operation according to the change information of the relative force and static information, where the static information includes at least one of magnitude, type, and direction, and the operation includes an opening operation, a holding operation, or a closing operation.

[0049] Among them, the static information can characterize the instantaneous situation of the relative force. Combining with the application scenarios that may be involved in the embodiment of the present invention, for example, it can be at least one of magnitude, type, and direction, where the type can be, for example, a normal force or a tangential force, and the direction can be, for example, a positive direction or a negative direction. Exemplarily, continuing with the above Figure 2 as an example, Fz = -5 Newtons (N) can represent a relative force with a magnitude of 5 N, a type of normal force, and a direction of the negative z-axis.

[0050] Static information and changing information can characterize the relative force from different aspects. Therefore, to further improve the stability of the opening and closing control of the manipulator, the manipulator can be jointly controlled to perform operations according to the changing information and the static information.

[0051] The technical solution of the embodiment of the present invention jointly controls the manipulator to perform operations by combining the changing information and the static information, thereby further improving the stability of the opening and closing control of the manipulator.

[0052] Figure 4 It is a flowchart of another method for controlling the opening and closing of the manipulator provided in the embodiment of the present invention. This embodiment is optimized based on the above technical solutions. In this embodiment, optionally, according to the changing information and the static information of the relative force, controlling the manipulator to perform operations includes: using a finite state machine to determine whether the manipulator satisfies any one of the preset at least one state transition conditions according to the changing information and the static information of the relative force; in the case of yes, for the target transition condition satisfied in the at least one state transition condition, determining the state after transition corresponding to the target transition condition; and controlling the manipulator to perform operations according to the state after transition. Among them, the explanations of the same or corresponding terms as those in the above embodiments are not repeated here.

[0053] See Figure 4 , the method of this embodiment may specifically include the following steps:

[0054] S310. Obtain the relative force detected by the sensor installed on the manipulator, where the relative force is generated when the sensor is in contact with the target object that can be grasped by the manipulator.

[0055] S320. Use a finite state machine to determine whether the manipulator satisfies any one of the preset at least one state transition conditions according to the changing information and the static information of the relative force, where the static information includes at least one of magnitude, type, and direction.

[0056] Among them, a finite state machine (FSM) can be understood as a mathematical model representing a finite number of states and behaviors such as transitions and actions between these states.

[0057] The state transition condition can be understood as a condition preset to characterize that a state transition can occur. The number of state transition conditions can be one or more, which is related to the actual situation and is not specifically limited here. It should be noted that in the embodiment of the present invention, the state transition condition of the FSM can be related to the relative force.

[0058] Using the FSM, based on two aspects of information, i.e., the change information and the static information, to determine whether the manipulator meets any of the state transition conditions. This helps to determine whether the manipulator needs to perform a state transition currently, and if so, to determine the post-transition state of the manipulator after the state transition.

[0059] S330. If so, for the target transition condition satisfied among at least one state transition condition, determine the post-transition state corresponding to the target transition condition.

[0060] Among them, after judgment, if the manipulator meets any of the state transition conditions, this indicates that the manipulator needs to perform a state transition currently. Then, the state transition condition satisfied among the state transition conditions is called the target transition condition, and the post-transition state corresponding to the target transition condition is determined.

[0061] S340. Control the manipulator to perform an operation according to the post-transition state, where the operation includes an opening operation, a holding operation, or a closing operation.

[0062] Among them, control the manipulator to perform an operation according to the post-transition state, so that the manipulator after the operation execution is in the post-transition state. Exemplarily, when the post-transition state is the grasping state, the manipulator can be controlled to perform a closing operation; when the post-transition state is the holding state, the manipulator can be controlled to perform a holding operation; when the post-transition state is the releasing state, the manipulator can be controlled to perform an opening operation; and so on, which is not specifically limited here.

[0063] In the technical solution of the embodiment of the present invention, by using the FSM, when it is determined that the manipulator meets any of the state transition conditions by combining the change information and the static information, this indicates that the manipulator needs to perform a state transition currently. Then, based on the post-transition state that the manipulator needs to be in after the transition, the manipulator can be controlled to perform an operation, thereby achieving the effect of accurate control of the opening and closing of the manipulator.

[0064] An optional technical solution is that the above-mentioned manipulator opening and closing control method further includes:

[0065] Using the finite state machine, when the current state of the manipulator is the releasing state and the current range of the manipulator is the preset initial range, the waiting state is taken as the current state, and the control of the manipulator is stopped.

[0066] Among them, the current state can be understood as the state where the manipulator is currently located. The releasing state can be understood as the state of releasing the target object. The waiting state can be understood as the state of waiting to grasp the target object.

[0067] For a certain two fingers on the robotic arm, the current range can be understood as the distance between the two fingers at the current moment; the initial range can be understood as the preset distance between the two fingers when waiting to grasp the target object. Figure 5 Exemplarily shows the meaning of the range, that is, the distance between two fingers.

[0068] Using the FSM, when the robotic arm is currently in the release state and the current range is the initial range, this indicates that the robotic arm has completely released the target object and returned to the initial form where the two fingers are at the initial range. Therefore, the waiting state can be used as the current state, and the control of the robotic arm is stopped to wait for the next grasping process, thus ensuring that the robotic arm starts to grasp the target object from the initial form each time.

[0069] Figure 6 It is a flowchart of another robotic arm opening and closing control method provided in an embodiment of the present invention. This embodiment is optimized based on the above technical solutions. In this embodiment, optionally, the above robotic arm opening and closing control method further includes: obtaining the current state of the robotic arm; determining whether the robotic arm satisfies any one of the preset at least one state transition condition according to the change information of the relative force and the static information, including: determining whether the robotic arm satisfies any one of the preset at least one state transition condition according to the current state and the change information of the relative force and the static information. Among them, the explanations of the same or corresponding terms as those in the above embodiments are not repeated here.

[0070] See Figure 6 , the method of this embodiment may specifically include the following steps:

[0071] S410. Obtain the relative force detected by the sensor installed on the robotic arm, where the relative force is generated when the sensor contacts the target object that can be grasped by the robotic arm.

[0072] S420. Obtain the current state of the robotic arm.

[0073] Among them, the current state can be understood as the state where the robotic arm is currently located. Combining with the application scenarios that may be involved in the embodiments of the present invention, for example, it may be a waiting state, a grasping state, a holding state, a release state, or a mis-grasping state, etc., which is related to the actual situation and is not specifically limited here. Obtain the current state.

[0074] S430. Using the finite state machine, determine whether the robotic arm satisfies any one of the preset at least one state transition condition according to the current state and the change information of the relative force and the static information, where the static information includes at least one of size, type, and direction.

[0075] Among them, it is assumed that there are N states that the manipulator can be in. In practical applications, the manipulator may not necessarily be able to transfer from any one of these N states to any other state, that is, the transfer between states is restricted. Exemplarily, for state A among the N states, the manipulator can only transfer from state A to state B or state C, and cannot transfer to state D or state E.

[0076] In view of this, in order to accurately determine whether the manipulator needs to perform a state transfer currently, and to determine the post-transfer state of the manipulator after the state transfer, it is possible to jointly determine whether each manipulator meets any one of the state transfer conditions according to the current state, the change information, and the static information.

[0077] S440. In the case of yes, for the target transfer condition satisfied among at least one state transfer condition, determine the post-transfer state corresponding to the target transfer condition.

[0078] S450. According to the post-transfer state, control the manipulator to perform operations, where the operations include opening operations, holding operations, or closing operations.

[0079] The technical solution of the embodiment of the present invention realizes the effect of accurately determining whether the manipulator needs to perform a state transfer currently and determining the post-transfer state of the manipulator after the state transfer by combining the current state to determine whether each manipulator meets any one of the state transfer conditions.

[0080] On this basis, in order to better understand the following various optional technical solutions, the application scenarios they may involve are exemplarily described here. Exemplarily, in this example, a 3-axis tactile sensor is installed at the fingertip of the manipulator, and a system for tactile signal acquisition and manipulator control is built. On this basis, using this system, the change rules of tactile signals (or relative forces) at different stages when the tactile sensor contacts and releases the target object are collected and analyzed, and a control strategy is designed based on this change rule, and the FSM is used to control the manipulator to realize the state transfer of the manipulator's grasping and releasing.

[0081] The variation rules described above are as follows: Assume that the robotic arm touches the target object before grasping it. At this time, the target object generates a normal force on the contact surface. After the tactile sensor detects this normal force, the robotic arm starts to grasp the target object. During the grasping process, the normal force between the contact surface and the target object gradually increases until the target object is firmly grasped. After the grasping action is completed, the normal force can be stabilized within a preset range according to the grip force of the robotic arm and the softness / hardness of the target object, and the normal force remains stable when the robotic arm is moved smoothly. If the target object is about to fall, a downward tangential force is generated between the grasping surface of the robotic arm and the target object at this time. If it is necessary to control the robotic arm to release the target object, the target object can be supported upward and placed on a platform. At this time, an upward tangential force is generated between the grasping surface of the robotic arm and the target object. After the tactile sensor detects this tangential force, the robotic arm releases (i.e., drops) the target object, and at this time all relative forces quickly reduce to zero.

[0082] In response to this, a grasping-lowering task can be divided into four sequentially occurring stages: the Waiting stage, the Grasp stage, the Holding stage, and the Release stage. Then, the robotic arm in different stages has corresponding states. Exemplarily, the waiting state (S1) corresponding to the waiting stage, the grasping state (S2) corresponding to the grasping stage, the holding state (S3) corresponding to the holding stage, and the release state corresponding to the release stage. On this basis, further, an error grasping state (Error, S5) can be added to ensure that the robotic arm can recover after an error grasp; and / or, a tight locking state can also be added, which can be applied to application scenarios where the target object is held tightly without being released. The transition between each state is implemented through an FSM. The following gives the robotic arm control schemes under several state transition conditions.

[0083] Optionally, when the target transfer condition is that the current state is the waiting state, the type is the normal force, and the magnitude is greater than the first normal force threshold, the state after transition is the grasping state. Then, according to the state after transition, the robotic arm is controlled to perform operations, including: controlling the robotic arm to perform a closing operation according to the grasping state.

[0084] Among them, the first normal force threshold can be understood as a threshold related to the magnitude of the normal force set in advance. In practical applications, optionally, the first normal force threshold can be set according to the response sensitivity required for grasping control. Generally, the lower the first normal force threshold, the more sensitive the robotic arm can respond to grasping control.

[0085] The above technical solution realizes the effective grasping of the target object.

[0086] Optionally, when the target transfer condition is that the current state is the grasping state, the type is the normal force, the magnitude is greater than the second normal force threshold, and the change value of the relative force is less than the normal force change threshold, the transferred state is the holding state. According to the transferred state, control the manipulator to perform operations, including: controlling the manipulator to perform a holding operation according to the holding state.

[0087] Among them, the second normal force threshold can be understood as a threshold related to the magnitude of the normal force set in advance. In practical applications, optionally, the second normal force threshold can be set according to the response sensitivity required for holding control. Generally, the lower the second normal force threshold, the more sensitive the manipulator can respond to holding control.

[0088] The change value of the relative force can characterize the change of the relative force within the target time period. The normal force change threshold can be understood as a threshold related to the change of the normal force set in advance. In practical applications, optionally, the normal force change threshold can be set according to the response sensitivity required for holding control. Generally, the higher the normal force change threshold, the more sensitive the manipulator can respond to holding control. It should be noted that the change value of the relative force being less than the normal force change threshold indicates that the magnitude of the relative force has stabilized within the preset range.

[0089] The above technical solution realizes the effective grasping of the target object.

[0090] Optionally, when the target transfer condition is that the current state is the holding state, the type is the tangential force, and the change value of the relative force is greater than the tangential force change threshold, the transferred state is the release state. According to the transferred state, control the manipulator to perform operations, including: controlling the manipulator to perform an opening operation according to the release state.

[0091] Among them, the tangential force change threshold can be understood as a threshold related to the change of the tangential force set in advance. In practical applications, optionally, the tangential force change threshold can be set according to the response sensitivity required for release control. Generally, the lower the tangential force change threshold, the more sensitive the manipulator can respond to release control.

[0092] The above technical solution realizes the effective release of the target object.

[0093] In practical applications, optionally, the target transfer condition from the holding state to the release state can also be a sudden change in the magnitude of the relative force, that is, the change value of the relative force per unit time is greater than the tangential force change threshold, and this tangential force change threshold can be the same as or different from the tangential force change threshold in the above text. This target transfer condition is particularly applicable to the application scenario of opening the manipulator to release the target object when swinging the target object.

[0094] Alternatively, when the target transfer condition is that the current state is a grasping state or a holding state, and the size is less than the relative force threshold, the state after the transfer is a misgrasp state, and according to the state after the transfer, the manipulator is controlled to perform an operation, including: according to the misgrasp state, controlling the manipulator to perform an opening operation.

[0095] The relative force threshold can be understood as a pre-set threshold related to the relative force magnitude. In practical applications, the relative force threshold can be optionally set according to the response sensitivity required for misgrasp state judgment. Generally, the higher the relative force threshold, the more sensitive the manipulator can respond to misgrasp judgment, but it should be noted that an excessively high relative force threshold may also affect the accuracy of misgrasp judgment, so it is necessary to set the relative force threshold by balancing the response sensitivity and accuracy of misgrasp judgment.

[0096] The above technical solution realizes the state recovery of the robot when an erroneous operation occurs.

[0097] On this basis, in order to better understand the above optional technical solutions as a whole, the following is an exemplary description of them in combination with specific examples. Figure 2 Based on Figure 7 :

[0098] T1: Transition from the waiting state (S1) to the grasping state (S2). The initial state of the manipulator starts from S1, and the current range of the manipulator is the initial range. When it is determined that the tactile sensor is in contact with the target object, it is transferred to S2. The target transfer condition is: Fz>Fz th 1, where Fz can represent the relative force magnitude of the normal force, Fz th 1 may represent the first normal force threshold.

[0099] T2: Transition from the grasping state (S2) to the holding state (S3). In S2, the robot is controlled to close (i.e., close) at a preset speed to grasp the target object. When it is determined that the target object has been grasped firmly, it is transferred to S3. The target transfer condition is: Fz>Fz th 2, and Fz is kept within a preset range (i.e. ΔFz < ΔFz th ), where Fz th 2 can represent the second normal force threshold, ΔFz can represent the change value of Fz, ΔFz th It can represent the normal force change threshold.

[0100] T3: Transfer from the hold state (S3) to the release state (S4). In S3, the manipulator motor is controlled to stop rotating and maintain the current range to maintain the target object. When it is determined that the person wearing the manipulator intends to put down the target object, it transfers to S4. The target transfer condition is: ΔFx>ΔFx thOr ΔFy > ΔFy th where Fx can represent the magnitude of the relative force on the x-axis with the type being the tangential force, ΔFx can represent the change value of Fx, and ΔFx th can represent the tangential force change threshold corresponding to the tangential force on the x-axis; Fy can represent the magnitude of the relative force on the y-axis with the type being the tangential force, ΔFy can represent the change value of Fy, and ΔFy th can represent the tangential force change threshold corresponding to the tangential force on the y-axis.

[0101] T4: Transition from the release state (S4) to the waiting state (S1). At S4, the manipulator opens to the initial range, i.e., the current range θ MH = the initial range θ O , and then switches to S1. At this time, a process from grasping to releasing the target object is completed.

[0102] T5 or T6: Transition from S2 to the misgrasp state (S5) or from S3 to S5. At S2 or S3, if the manipulator does not actually grasp the target object, it belongs to a misgrasp and transfers to S5. The target transfer condition is: Fr < Fr th , where after transferring to S5, it automatically transfers to S4, thus ensuring that the manipulator can automatically reset after a misgrasp.

[0103] The above example provides a manipulator opening and closing control scheme triggered by tactile signals. This control scheme belongs to a semi-automatic control scheme. The wearer only needs to give a tactile signal for triggering control, and subsequent action switching can be automatically completed under the control of the controller. Exactly because this example uses a semi-automatic control scheme to control the actions of the manipulator, it enables the wearer not to control the manipulator through the arm or brain, etc., thereby solving the problem that the opening and closing control performance of the manipulator cannot be guaranteed due to factors such as arm posture, muscle state, arm muscle mass, and brain attention, thus ensuring the stability of the manipulator opening and closing control and performing excellently in preventing the target object from falling.

[0104] Figure 8 is the structural block diagram of the manipulator opening and closing control device provided in the embodiment of the present invention. This device is used to execute the manipulator opening and closing control method provided in any of the above embodiments. This device and the manipulator opening and closing control methods of the above embodiments belong to the same inventive concept. For the details not described in detail in the embodiment of the manipulator opening and closing control device, reference can be made to the embodiments of the above manipulator opening and closing control methods. See Figure 8 , and this device may specifically include: a relative force acquisition module 510 and a manipulator operation control module 520.

[0105] Among them, the relative force acquisition module 510 is configured to acquire the relative force detected by a sensor installed on the mechanical arm, where the relative force is generated when the sensor contacts a target object that can be grasped by the mechanical arm;

[0106] The manipulator operation control module 520 is configured to control the manipulator to perform operations according to the change information of the relative force, where the operations include an opening operation, a holding operation, or a closing operation.

[0107] Optionally, the manipulator operation control module 520 may include:

[0108] The manipulator operation control sub-module is configured to control the manipulator to perform operations according to the change information of the relative force and the static information, where the static information includes at least one of size, type, and direction.

[0109] On this basis, optionally, the manipulator operation control sub-module may include:

[0110] The state transition condition satisfaction determination unit is configured to use a finite state machine to determine whether the manipulator satisfies any one of the preset at least one state transition condition according to the change information of the relative force and the static information;

[0111] The post-transition state determination unit is configured to, in the case of yes, determine the post-transition state corresponding to the target transition condition for the target transition condition satisfied among the at least one state transition condition;

[0112] The manipulator operation control unit is configured to control the manipulator to perform operations according to the post-transition state.

[0113] On this basis, optionally, the above-mentioned manipulator opening and closing control device further includes:

[0114] The current state acquisition module is configured to acquire the current state in which the manipulator is currently located;

[0115] Correspondingly, the state transition condition satisfaction determination unit is specifically configured to:

[0116] Use a finite state machine to determine whether the manipulator satisfies any one of the preset at least one state transition condition according to the current state, the change information of the relative force, and the static information.

[0117] On this basis, optionally, when the target transition condition is that the current state is the waiting state, the type is the normal force, and the magnitude is greater than the first normal force threshold, the post-transition state is the grasping state, and the manipulator operation control unit is specifically configured to: control the manipulator to perform a closing operation according to the grasping state; and / or,

[0118] When the target transfer condition includes that the current state is the grasping state, the type is the normal force, the magnitude is greater than the second normal force threshold, and the change value of the relative force is less than the normal force change threshold, the transferred state is the holding state. The manipulator operation control unit can be specifically used for: controlling the manipulator to perform a holding operation according to the holding state; and / or,

[0119] When the target transfer condition is that the current state is the holding state, the type is the tangential force, and the change value of the relative force is greater than the tangential force change threshold, the transferred state is the releasing state. The manipulator operation control unit is specifically used for: controlling the manipulator to perform an opening operation according to the releasing state.

[0120] Another option is that when the target transfer condition is that the current state is the grasping state or the holding state, and the magnitude is less than the relative force threshold, the transferred state is the mis-grasping state. The manipulator operation control unit can be specifically used for: controlling the manipulator to perform an opening operation according to the mis-grasping state.

[0121] Another option is that the above manipulator opening and closing control device further includes:

[0122] A manipulator stop control module, which can be used to use a finite state machine to set the waiting state as the current state and stop controlling the manipulator when the current state of the manipulator is the releasing state and the current range of the manipulator is the preset initial range.

[0123] Optionally, the sensor is a tactile sensor. The relative force acquisition module 510 includes:

[0124] A tactile signal acquisition unit, which is used to acquire tactile signals detected by a tactile sensor installed on the manipulator;

[0125] A relative force obtaining unit, which is used to obtain the relative force between the tactile sensor and the target object based on the tactile signal.

[0126] The manipulator opening and closing control device provided by the embodiment of the present invention, through the relative force acquisition module, for the sensor installed on the manipulator and the target object that can be grasped by the manipulator, when the sensor contacts the target object, can acquire the relative force between the sensor and the target object detected by the sensor; further, through the manipulator operation control module, according to the change information of the relative force, control the manipulator to perform an opening operation, a holding operation or a closing operation, so as to assist the wearing object of the manipulator to grasp or put down the target object. The above device realizes the automatic control of the manipulator opening and closing through the change of the relative force between the sensor and the target object, so that the wearing object only needs to control the contact between the manipulator and the target object, that is, only needs to give a tactile signal for triggering control, and the subsequent opening and closing operations can be automatically executed under control, that is, there is no need for the wearing object to control through the arm or the brain, etc., thus solving the problem that the opening and closing control performance of the manipulator cannot be guaranteed due to factors such as arm posture, muscle state, arm muscle mass and brain attention, and ensuring the stability of the manipulator opening and closing control.

[0127] The manipulator opening and closing control device provided by the embodiment of the present invention can execute the manipulator opening and closing control method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0128] It should be noted that in the embodiments of the above manipulator opening and closing control device, the included units and modules are only divided according to the functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the present invention.

[0129] Figure 9 FIG. shows a schematic structural diagram of an electronic device 10 that can be used to implement the embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device (such as a helmet, glasses, a watch, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0130] As Figure 9As shown, the electronic device 10 includes at least one processor 11 and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. Among them, the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0131] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0132] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the manipulator opening and closing control method.

[0133] In some embodiments, the manipulator opening and closing control method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the manipulator opening and closing control method described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute the manipulator opening and closing control method in any other appropriate way (for example, by means of firmware).

[0134] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0135] The computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer programs can be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0136] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0137] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0138] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0139] A computing system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0140] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is imposed herein.

[0141] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for controlling the opening and closing of a manipulator, characterized in that, it includes: Obtain the relative force detected by a sensor installed on the manipulator, where the relative force is generated when the sensor comes into contact with a target object that can be grasped by the manipulator; According to the change information of the relative force, control the manipulator to perform an operation, where the operation includes an opening operation, a holding operation, or a closing operation.

2. The method according to claim 1, characterized in that, the controlling the manipulator to perform an operation according to the change information of the relative force includes: Controlling the manipulator to perform an operation according to the change information of the relative force and static information, where the static information includes at least one of magnitude, type, and direction.

3. The method according to claim 2, characterized in that, the controlling the manipulator to perform an operation according to the change information of the relative force and static information includes: Using a finite state machine, determine whether the manipulator satisfies any one of at least one preset state transition condition according to the change information of the relative force and static information; In the case of yes, for the target transition condition satisfied in the at least one state transition condition, determine the state after transition corresponding to the target transition condition; Control the manipulator to perform an operation according to the state after transition.

4. The method according to claim 3, characterized in that, it further includes: Obtain the current state of the manipulator; Correspondingly, the determining whether the manipulator satisfies any one of at least one preset state transition condition according to the change information of the relative force and static information includes: Determine whether the manipulator satisfies any one of at least one preset state transition condition according to the current state, the change information of the relative force, and static information.

5. The method according to claim 4, characterized in that, When the target transition condition is that the current state is the waiting state, the type is the normal force, and the magnitude is greater than the first normal force threshold, the state after transition is the grasping state, and the controlling the manipulator to perform an operation according to the state after transition includes: controlling the manipulator to perform a closing operation according to the grasping state; and / or, When the target transition condition includes that the current state is the grasping state, the type is the normal force, the magnitude is greater than the second normal force threshold, and the change value of the relative force is less than the normal force change threshold, the state after transition is the holding state, and the controlling the manipulator to perform an operation according to the state after transition includes: controlling the manipulator to perform a holding operation according to the holding state; and / or, When the target transition condition is that the current state is the holding state, the type is the tangential force, and the change value of the relative force is greater than the tangential force change threshold, the state after transition is the releasing state, and the controlling the manipulator to perform an operation according to the state after transition includes: controlling the manipulator to perform an opening operation according to the releasing state.

6. The method according to claim 4, wherein, when the target transfer condition is that the current state is the grasping state or the holding state, and the size is less than the relative force threshold, the state after transfer is the mis-grasp state, and controlling the manipulator to perform an operation according to the state after transfer includes: controlling the manipulator to perform an opening operation according to the mis-grasp state.

7. The method according to claim 3, wherein, further comprising: using the finite state machine, when the current state of the manipulator is the release state and the current range of the manipulator is the preset initial range, taking the waiting state as the current state and stopping controlling the manipulator.

8. The method according to claim 1, wherein, the sensor is a tactile sensor, and obtaining the relative force detected by the sensor installed on the manipulator includes: obtaining the tactile signal detected by the tactile sensor installed on the manipulator; based on the tactile signal, obtaining the relative force between the tactile sensor and the target object.

9. A manipulator opening and closing control device, wherein, comprising: a relative force obtaining module, configured to obtain the relative force detected by the sensor installed on the manipulator, wherein the relative force is generated when the sensor contacts a target object that can be grasped by the manipulator; a manipulator operation control module, configured to control the manipulator to perform an operation according to the change information of the relative force, wherein the operation includes an opening operation, a holding operation or a closing operation.

10. An electronic device, wherein, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the manipulator opening and closing control method according to any one of claims 1-8.

11. A computer-readable storage medium, wherein, the computer-readable storage medium stores computer instructions for causing a processor to implement the manipulator opening and closing control method according to any one of claims 1-8 when executed.

Images