Artificial hand control method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses an artificial hand control method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring an electromyographic signal of a muscle on a wearing end of the artificial hand and a pressure signal generated by the muscle due to upheaval, and identifying a first motion intention based on the electromyographic signal and the pressure signal; generating a first control signal based on the first motion intention to control the artificial hand to move based on the first control signal; in the motion process of the artificial hand, acquiring an angle signal of each finger joint on the artificial hand, an inertia signal of the artificial hand and a grasping signal for reflecting grasping force applied by the artificial hand, and identifying a second motion intention based on the angle signal, the inertia signal and the grasping signal; generating a second control signal according to the first motion intention and the second motion intention, and controlling the artificial hand to move based on the second control signal. According to the technical scheme, the stability of artificial hand control can be improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of automatic control technology, and in particular to a prosthetic hand control method, device, electronic device and storage medium. Background Art

[0002] In recent years, thanks to the rapid development of artificial intelligence technology and pattern recognition technology, solutions for prosthetic hand control based on electromyographic signals have become increasingly mature.

[0003] However, investigations have found that the current prosthetic hand control scheme has a problem of reduced stability in prosthetic hand control when the electromyographic signal is affected by sweat, electrode displacement, and muscle fatigue of the prosthetic hand wearer, which needs to be solved urgently. Summary of the invention

[0004] Embodiments of the present invention provide a prosthetic hand control method, device, electronic device and storage medium to improve the stability of prosthetic hand control.

[0005] According to one aspect of the present invention, a method for controlling a prosthetic hand is provided, which may include:

[0006] Acquire the myoelectric signals of the muscles on the wearing end of the artificial hand and the pressure signals generated by the muscles bulging, and recognize the first movement intention based on the myoelectric signals and the pressure signals;

[0007] generating a first control signal based on the first movement intention to control the movement of the prosthetic hand based on the first control signal;

[0008] During the movement of the prosthetic hand, an angle signal of each finger joint on the prosthetic hand, an inertia signal of the prosthetic hand, and a gripping signal reflecting the gripping force applied by the prosthetic hand are obtained, and a second movement intention is identified based on the angle signal, the inertia signal, and the gripping signal;

[0009] A second control signal is generated according to the first movement intention and the second movement intention to control the movement of the prosthetic hand based on the second control signal.

[0010] According to another aspect of the present invention, there is provided a prosthetic hand control device, which may include:

[0011] A first movement intention recognition module is used to obtain the myoelectric signals of the muscles on the wearing end of the artificial hand and the pressure signals generated by the muscles due to the bulge, and recognize the first movement intention based on the myoelectric signals and the pressure signals;

[0012] A first artificial hand control module, used to generate a first control signal based on the first movement intention, so as to control the movement of the artificial hand based on the first control signal;

[0013] A second movement intention recognition module is used to obtain an angle signal of each finger joint on the prosthetic hand, an inertia signal of the prosthetic hand, and a grip signal reflecting the gripping force applied by the prosthetic hand during the movement of the prosthetic hand, and recognize a second movement intention based on the angle signal, the inertia signal, and the grip signal;

[0014] The second artificial hand control module is used to generate a second control signal according to the first movement intention and the second movement intention, so as to control the movement of the artificial hand based on the second control signal.

[0015] According to another aspect of the present invention, there is provided an electronic device, which may include:

[0016] at least one processor; and

[0017] a memory communicatively connected to at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that the at least one processor implements the prosthetic hand control method provided by any embodiment of the present invention when executing the computer program.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, on which computer instructions are stored. The computer instructions are used to enable a processor to implement the prosthetic hand control method provided by any embodiment of the present invention when executed.

[0020] The technical solution of the embodiment of the present invention is to obtain the electromyographic signals of the muscles on the wearing end of the prosthetic hand and the pressure signals generated by the muscles due to the bulge, and identify the first movement intention based on the electromyographic signals and the pressure signals, that is, the first movement intention is identified based on the feedforward signal (that is, the electromyographic signals and the pressure signals); generate a first control signal based on the first movement intention to control the movement of the prosthetic hand based on the first control signal, that is, this movement process is controlled based on the feedforward signal; during the movement of the prosthetic hand, obtain the angle signal of each finger joint on the prosthetic hand, the inertia signal of the prosthetic hand and the gripping signal used to reflect the gripping force applied by the prosthetic hand, and identify the second movement intention based on the angle signal, the inertia signal and the gripping signal, that is, the second movement intention is identified based on the feedforward signal (that is, the angle signal, the inertia signal and the gripping signal); further, generate a second control signal based on the first movement intention and the second movement intention to control the movement of the prosthetic hand based on the second control signal, that is, this movement process is controlled based on the feedforward signal and the feedback signal. The above technical scheme, during the movement of the prosthetic hand, especially during the interaction between the prosthetic hand and objects in the environment, corrects the first control signal of the prosthetic hand obtained based on the feedforward signal by means of the feedback signal collected from the prosthetic hand, thereby solving the problem of decreased stability of the prosthetic hand control caused by factors such as the influence of sweat on the electromyographic signal, electrode displacement and muscle fatigue, and improving the stability of the prosthetic hand control.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flow chart of a prosthetic hand control method provided according to an embodiment of the present invention;

[0024] Figure 2 is a flow chart of another prosthetic hand control method provided according to an embodiment of the present invention;

[0025] Figure 3 is a flow chart of another prosthetic hand control method provided according to an embodiment of the present invention;

[0026] Figure 4 is a first schematic diagram of an optional example of another prosthetic hand control method provided according to an embodiment of the present invention;

[0027] Figure 5 is a second schematic diagram of an optional example of another prosthetic hand control method provided in an embodiment of the present invention;

[0028] Figure 6 is a structural block diagram of a prosthetic hand control device provided according to an embodiment of the present invention;

[0029] Figure 7 It is a structural schematic diagram of an electronic device for implementing the prosthetic hand control method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The situations of "target", "original", etc. are similar and will not be repeated here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Figure 1 This is a flow chart of a prosthetic hand control method provided by an embodiment of the present invention. This embodiment is applicable to the case of prosthetic hand control, and is particularly applicable to the case of prosthetic hand control by fusing feedforward signals and feedback signals. The method can be executed by a prosthetic hand control device provided by an embodiment of the present invention, and the device can be implemented by software and / or hardware, and the device can be integrated on an electronic device (i.e., a host computer), and the electronic device can be various user terminals or servers.

[0033] See also Figure 1 The method of the embodiment of the present invention specifically includes the following steps:

[0034] S110. Acquire the myoelectric signals of the muscles on the wearing end of the artificial hand and the pressure signals generated by the muscles bulging, and identify the first movement intention based on the myoelectric signals and the pressure signals.

[0035] The artificial hand may be understood as the artificial hand to be controlled. The wearer may be understood as the object wearing the artificial hand. In the embodiment of the present invention, the object may be an amputee, especially a forearm amputee. The wearing end may be understood as the part of the wearer where the artificial hand is worn. In the embodiment of the present invention, the part may be a stump, i.e., the stump of a forearm amputee.

[0036] The muscle can be understood as the muscle on the wearing end. The electromyographic signal can be understood as the electrical signal of the muscle. In the embodiment of the present invention, the prosthetic hand can optionally be integrated with an electromyographic sensor, and the electromyographic signal can be collected by the electromyographic sensor. The pressure signal can be understood as the electrical signal generated by the muscle due to the bulge. In the embodiment of the present invention, the prosthetic hand can optionally be integrated with a pressure sensor, and the pressure sensor can be further integrated in the electromyographic sensor, and the pressure signal can be collected by the pressure sensor.

[0037] The collected electromyographic signal and pressure signal are obtained, and then the first movement intention of the wearer, or more specifically, the movement intention of the artificial hand worn by the wearer, can be identified based on the electromyographic signal and pressure signal. It can be understood that in the embodiment of the present invention, the electromyographic signal and pressure signal are equivalent to feedforward signals.

[0038] S120. Generate a first control signal based on the first movement intention, so as to control the movement of the prosthetic hand based on the first control signal.

[0039] A first control signal is generated based on the first movement intention, so that the movement of the prosthetic hand can be controlled based on the first control signal, in particular, the prosthetic hand can be controlled to perform a movement process that conforms to the first movement intention.

[0040] S130. During the movement of the prosthetic hand, obtain the angle signal of each finger joint on the prosthetic hand, the inertia signal of the prosthetic hand, and the gripping signal reflecting the gripping force applied by the prosthetic hand, and identify the second movement intention based on the angle signal, the inertia signal, and the gripping signal.

[0041] Among them, the angle signal represents the bending angle of the finger joints on the prosthetic hand, and in particular, the change of the bending angle, which can feedback the movement state of the prosthetic hand. In an embodiment of the present invention, optionally, the angle signal can be measured by an angle sensor or a bending sensor set on the finger joints. Optionally, in combination with the application scenarios that may be involved in the embodiments of the present invention, the prosthetic hand may interact with objects in the environment during movement, then the angle signal can feedback the posture of the prosthetic hand when actually grasping the object, and the posture corresponds to the shape of the object grasped by the prosthetic hand; on this basis, optionally, it can also feedback whether the grasping action of the prosthetic hand matches the actual movement intention of the wearer.

[0042] The inertial signal can be measured by an inertial measurement unit (IMU) provided on the prosthetic hand, and the IMU can also be called an inertial sensor. In an embodiment of the present invention, optionally, the IMU can be provided on the back of the prosthetic hand. Optionally, the IMU can be used to measure inertial signals such as the acceleration, angular velocity and tilt angle of the prosthetic hand, so that at least one of the movement direction, movement speed and movement posture of the prosthetic hand can be obtained through the inertial signal. On this basis, optionally, the movement posture can be, for example, a tilt angle and / or a rotation angle, so that the prosthetic hand can sense its own position in space.

[0043] The grasping signal can be understood as a signal used to characterize the grasping force applied by the prosthetic hand. In combination with the application scenarios that may be involved in the embodiments of the present invention, the grasping force can be the force applied to the object when the prosthetic hand contacts the object. In the embodiments of the present invention, optionally, the grasping signal can be collected by a current sensor integrated in the prosthetic hand, a multi-point pressure sensor at the fingertips and palms, an ultrasonic sensor, or an optical sensor, etc., which is not specifically limited here. Taking the current sensor as an example here, the grasping signal is a current signal, which reflects the change in current when the prosthetic hand contacts the object. The current change can be a change in the working current of the servo motor set in the prosthetic hand, and the grasping force applied by the prosthetic hand is fed back through the current change.

[0044] During the movement of the prosthetic hand, angle signals, inertia signals and grasping signals are obtained, and the wearer's second movement intention can be identified based on these three (i.e., three feedback signals).

[0045] S140. Generate a second control signal according to the first movement intention and the second movement intention, so as to control the movement of the prosthetic hand based on the second control signal.

[0046] Among them, the second control signal is generated according to the first movement intention and the second movement intention. For example, the actual movement intention X of the wearer can be obtained by analyzing the two movement intentions; for another example, the first movement intention can be corrected by the second movement intention to obtain the movement intention X; and so on, and then the second control signal can be generated based on the movement intention X. It can be seen that compared with using the first control signal generated only based on the first movement intention to control the movement of the prosthetic hand, by using the second control signal generated based on the first movement intention and the second movement intention to control the movement of the prosthetic hand, a movement process can be made to better match the movement intention X.

[0047] There are three points that need to be explained here. First, since the embodiment of the present invention does not only use the feedforward signal to control the prosthetic hand, but combines the feedforward signal with the feedback signal to control the prosthetic hand, this makes it possible to appropriately reduce the number of electrodes of the electromyographic sensor attached to the wearing end, thereby improving the convenience of the wearer when wearing the prosthetic hand; second, the embodiment of the present invention integrates a sensor for measuring the feedback signal at the prosthetic hand end, which eliminates the need to add additional sensors for measuring the feedforward signal at the wearing end, reduces the hardware complexity, and makes the acquisition of control signals easier and more accurate; third, the embodiment of the present invention can be used not only in the field of medical rehabilitation, but also in the industrial field, for performing dangerous or delicate operating tasks, such as handling toxic chemicals or assembling precision instruments, etc., which are not specifically limited here.

[0048] The technical solution of the embodiment of the present invention is to obtain the electromyographic signals of the muscles on the wearing end of the prosthetic hand and the pressure signals generated by the muscles due to the bulge, and identify the first movement intention based on the electromyographic signals and the pressure signals, that is, the first movement intention is identified based on the feedforward signal (that is, the electromyographic signals and the pressure signals); generate a first control signal based on the first movement intention to control the movement of the prosthetic hand based on the first control signal, that is, this movement process is controlled based on the feedforward signal; during the movement of the prosthetic hand, obtain the angle signal of each finger joint on the prosthetic hand, the inertia signal of the prosthetic hand and the gripping signal used to reflect the gripping force applied by the prosthetic hand, and identify the second movement intention based on the angle signal, the inertia signal and the gripping signal, that is, the second movement intention is identified based on the feedforward signal (that is, the angle signal, the inertia signal and the gripping signal); further, generate a second control signal based on the first movement intention and the second movement intention to control the movement of the prosthetic hand based on the second control signal, that is, this movement process is controlled based on the feedforward signal and the feedback signal. The above technical scheme, during the movement of the prosthetic hand, especially during the interaction between the prosthetic hand and objects in the environment, corrects the first control signal of the prosthetic hand obtained based on the feedforward signal by means of the feedback signal collected from the prosthetic hand, thereby solving the problem of decreased stability of the prosthetic hand control caused by factors such as the influence of sweat on the electromyographic signal, electrode displacement and muscle fatigue, and improving the stability of the prosthetic hand control.

[0049] Figure 2 It is a flow chart of another prosthetic hand control method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, a first movement intention is identified based on an electromyographic signal and a pressure signal, including: obtaining a first movement intention recognition model, inputting the electromyographic signal and the pressure signal into the first movement intention recognition model, and / or, a second movement intention is identified based on an angle signal, an inertial signal, and a grasping signal, which may include: obtaining a second movement intention recognition model, and inputting the angle signal, the inertial signal, and the grasping signal into the second movement intention recognition model to obtain a second movement intention. Among them, the explanations of the terms that are the same or corresponding to the above-mentioned embodiments are not repeated here.

[0050] See also Figure 2 The method of this embodiment may specifically include the following steps:

[0051] S210. Acquire the myoelectric signals of the muscles on the wearing end of the artificial hand and the pressure signals generated by the muscles bulging.

[0052] S220. Obtain a first movement intention recognition model, input the electromyographic signal and the pressure signal into the first movement intention recognition model, and obtain a first movement intention.

[0053] S230. Generate a first control signal based on the first movement intention, so as to control the movement of the prosthetic hand based on the first control signal.

[0054] S240. During the movement of the prosthetic hand, an angle signal of each finger joint on the prosthetic hand, an inertia signal of the prosthetic hand, and a grasping signal reflecting the grasping force applied by the prosthetic hand are obtained.

[0055] S250. Obtain a second motion intention recognition model, and input the angle signal, the inertia signal, and the grasping signal into the second motion intention recognition model to obtain a second motion intention.

[0056] In the embodiment of the present invention, optionally, the second motion intention recognition model and the first motion intention recognition model can be the same motion intention recognition model, and in particular can be different motion intention recognition models, which helps to improve the accuracy of motion intention recognition.

[0057] S260. Generate a second control signal according to the first movement intention and the second movement intention, so as to control the movement of the prosthetic hand based on the second control signal.

[0058] The technical solution of the embodiment of the present invention realizes accurate recognition of movement intention by applying a movement intention recognition model.

[0059] In an optional technical solution, the first motion intention recognition model is pre-trained through the following steps:

[0060] Obtaining sample electromyographic signals of the muscles, sample pressure signals generated by the muscle bulging, and movement intention labels corresponding to the sample electromyographic signals and the sample pressure signals, and using the sample electromyographic signals, the sample pressure signals, and the movement intention labels as a set of training samples;

[0061] The original motion intention recognition model is trained based on the obtained multiple groups of training samples to obtain a first motion intention recognition model.

[0062] In the present technical solution, optionally, corresponding first motion intention recognition models can be trained for different wearers respectively, that is, personalized training of the first motion intention recognition model can be performed, thereby further improving the accuracy of motion intention recognition.

[0063] Figure 3 is a flow chart of another artificial hand control method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, controlling the movement of the artificial hand based on the second control signal includes: sending the second control signal to the single-chip microcomputer so that the single-chip microcomputer generates a pulse width modulation signal based on the received second control signal, and controlling the movement of the artificial hand through the pulse width modulation signal. Among them, the explanations of the terms that are the same or corresponding to the above-mentioned embodiments are not repeated here.

[0064] See also Figure 3 The method of this embodiment may specifically include the following steps:

[0065] S310. Acquire the myoelectric signals of the muscles on the wearing end of the artificial hand and the pressure signals generated by the muscles bulging, and identify the first movement intention based on the myoelectric signals and the pressure signals.

[0066] S320. Generate a first control signal based on the first movement intention, so as to control the movement of the prosthetic hand based on the first control signal.

[0067] Among them, in an embodiment of the present invention, optionally, the first control signal can be sent to the single-chip microcomputer, so that the single-chip microcomputer can generate a pulse width modulation (PWM) signal based on the received first control signal, and control the movement of the prosthetic hand through the PWM signal.

[0068] S330. During the movement of the prosthetic hand, obtain the angle signal of each finger joint on the prosthetic hand, the inertia signal of the prosthetic hand, and the gripping signal reflecting the gripping force applied by the prosthetic hand, and identify the second movement intention based on the angle signal, the inertia signal, and the gripping signal.

[0069] S340. Generate a second control signal according to the first movement intention and the second movement intention.

[0070] S350. Send the second control signal to the single chip microcomputer, so that the single chip microcomputer generates a pulse width modulation signal based on the received second control signal, and controls the movement of the artificial hand through the pulse width modulation signal.

[0071] Among them, the second control signal is sent to the single-chip microcomputer, so that the single-chip microcomputer can generate a PWM signal based on the received second control signal. It should be noted that the PWM signal generated in this step is a PWM signal corresponding to the second control signal, rather than the PWM signal corresponding to the first control signal generated in S320; then, the single-chip microcomputer can control the movement of the prosthetic hand through the PWM signal.

[0072] According to the technical solution of the embodiment of the present invention, the single chip microcomputer realizes effective control of the prosthetic hand through PWM signals.

[0073] In an optional technical solution, the grasping signal is represented by a current signal for describing the working current of a servo motor disposed in the artificial hand, and the single chip microcomputer further performs the following steps:

[0074] Obtain angle signal and current signal;

[0075] According to the angle signal, the change rate of the bending angle of the finger joint is obtained;

[0076] When the change rate is less than the preset change rate threshold and the current signal is within the preset current interval, the pulse width modulation signal is maintained.

[0077] Among them, the current signal can be understood as a signal used to describe the working current of the servo motor set in the artificial hand, and the grasping signal is represented by the current signal. The change rate of the bending angle ΔAngle is obtained according to the angle signal. If ΔAngle is less than the preset change rate threshold, it means that the bending angle is in a stable state; and if the current signal is within the preset current range, it means that the working current is in a stable state. When both the bending angle and the working current are in a stable state, it means that the artificial hand has grasped the object, and the single-chip microcomputer can maintain the current PWM signal to avoid unnecessary adjustment of the grasping force and ensure stable grasping of the object.

[0078] In another optional technical solution, the grasping signal can be represented by a current signal for describing the working current of a servo motor disposed in the artificial hand, and the single chip microcomputer further performs the following steps:

[0079] When the current signal becomes smaller, the duty cycle of the pulse width modulation signal is increased.

[0080] If the current signal becomes smaller, that is, the working current becomes smaller, which usually occurs when the object slides and the gripping force weakens, the gripping force can be increased by increasing the duty cycle of the PWM signal to compensate for the sliding tendency, thereby preventing the object from sliding and damaging it. This is an anti-slip mechanism.

[0081] On this basis, optionally, the microcontroller also performs the following steps:

[0082] When the current signal is less than the preset upper limit of the safe current, and the difference between the current signal and the upper limit of the safe current is less than or equal to the preset difference threshold, the duty cycle is stopped from being increased.

[0083] The upper limit of the safe current can be understood as the upper limit of the working current that can avoid damage to the object and the artificial hand. After increasing the duty cycle to increase the current signal, if the current signal is less than the preset upper limit of the safe current, and the difference between the current signal and the upper limit of the safe current is less than or equal to the preset difference threshold, that is, if the current signal is close to the upper limit of the safe current, the duty cycle can be stopped from being increased, that is, the current signal can be stopped from being increased, and then the gripping force can be stopped from being increased, thereby ensuring the safety of the artificial hand and the object.

[0084] On this basis, in order to better understand the above-mentioned various technical solutions as a whole, the following can be illustrated with reference to specific examples. Figure 4 and Figure 5 The specific implementation process is as follows:

[0085] Step 1: The technician finds multiple muscles with strong muscle activity on the stump of the forearm amputee, wipes them with alcohol, and wears a prosthetic hand with a multi-channel electromyographic sensor on the stump to prepare for electromyographic signal collection. In this example, the multi-channel electromyographic sensor can optionally be integrated with a pressure sensor, so that the electromyographic signal and the pressure signal can be collected synchronously through the multi-channel electromyographic sensor.

[0086] Step 2: Open the serial port connection, ensure that the EMG sensor communicates normally with the host computer software, calibrate the signal reception status, and start real-time monitoring of the EMG and pressure signals. After confirming that the signal transmission is stable, the technician can start collecting the EMG and pressure signals and further process them to send them to the host computer.

[0087] Here, the electromyographic signal is taken as an example, and the pressure signal is the same. In this example, the electromyographic signal can be processed by a signal processing unit, which can be composed of a signal acquisition module (AD7606) and a single-chip microcomputer (STM32F767IGT6). The signal acquisition module can be used to collect electromyographic signals, and then send the electromyographic signals to the single-product machine through the serial peripheral interface (Serial Peripheral Interface, SPI) for median mean filtering. The processed electromyographic signals are sent to the host computer through the serial communication interface (Serial Communications Interface, SCI) for processing. In the host computer, the electromyographic signal can be displayed and pre-processed, for example, the frequency domain features can be extracted from the electromyographic signal in the time domain using fast Fourier transform.

[0088] Next, electromyographic signals are used as an example to explain, but in actual applications, the same steps can be used to process pressure signals, because electromyographic signals and pressure signals need to be used together for model training and motion intention recognition.

[0089] Step 3: The stump performs preset muscle activity, and the myoelectric sensor collects the myoelectric signal. The technician maps the muscle activity to a certain movement intention through the host computer software. In this example, the movement intention can be understood as the movement of the prosthetic hand, that is, the combination of the position data of each servo in the prosthetic hand.

[0090] Step 4: Operate on the host computer software to pre-process the collected electromyographic signals corresponding to each movement intention to obtain frequency domain features, and then train the model on this basis, and monitor the loss reduction of the model in real time through the visual interface on the software. If the model learns normally, the model can be saved for subsequent control of the prosthetic hand based on real-time electromyographic signals; if the model training effect is not good, you can start from step 3 to let the stump make electromyographic signals with greater muscle activity differences, so that the model can obtain higher quality electromyographic signals for model training.

[0091] Step 5: Use the movement intention recognition model saved in step 4 to recognize the current movement intention of the forearm amputee based on the real-time electromyographic signal, and generate a control signal based on the recognition result and send it to the microcontroller. The microcontroller sends a PWM signal to control the rotation angle of the servo of the traction tendon rope, thereby driving the fingers of the prosthetic hand to flex and extend, realizing various movements of the prosthetic hand.

[0092] Step 6: In the process of real-time control of the prosthetic hand to interact with objects in the environment, the feedback signals collected by the angle sensor, IMU and current sensor are uploaded to the host computer through the single-chip microcomputer, and the motion intention recognition model is used to identify the motion intention, and the servo is fine-tuned to make the control of the prosthetic hand more stable. In this example, optionally, if an abnormality occurs in the grasping process of the prosthetic hand (such as applying too much or too little grasping force), the single-chip microcomputer can immediately adjust the PWM signal according to the feedback signal to avoid damage to the prosthetic hand or the object, thereby ensuring the safety and accuracy of the prosthetic hand operation.

[0093] The above example has at least the following advantages:

[0094] 1) By integrating the angle sensor, IMU and current sensor on the prosthetic hand, there is no need to add additional sensors for measuring feedforward signals on the stump, thereby reducing the hardware complexity and making the acquisition of control signals easier and more accurate.

[0095] 2) The angle sensor can provide real-time feedback on the bending angle of the finger joints of the prosthetic hand, allowing the host computer to sense the posture of the prosthetic hand when it is actually grasping an object, and accurately judge the shape of the object and whether the grasping action is successful based on the posture, thereby significantly improving the accuracy of the prosthetic hand operation.

[0096] 3) The current sensor can monitor the changes in the working current of the servo when the prosthetic hand contacts an object, and use this change to provide real-time feedback on the gripping force applied by the prosthetic hand, ensuring that the host computer can dynamically adjust the movement of the prosthetic hand based on the gripping force feedback, thereby achieving more precise gripping force control and improving grip stability and safety.

[0097] 4) Through the multimodal fusion of inertial signals, angle signals and current signals (i.e. various feedback signals) with electromyographic information and pressure signals (i.e. various feedforward signals), this example can provide more comprehensive feedback information to the host computer, thereby improving the stability and responsiveness of the prosthetic hand in complex operation scenarios.

[0098] Figure 6 This is a structural block diagram of a prosthetic hand control device provided in an embodiment of the present invention. The device is used to execute the prosthetic hand control method provided in any of the above embodiments. The device and the prosthetic hand control method in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the prosthetic hand control device, please refer to the embodiment of the above prosthetic hand control method. Figure 6 The device may specifically include: a first motion intention recognition module 410, a first artificial hand control module 420, a second motion intention recognition module 430 and a second artificial hand control module 440.

[0099] A first movement intention recognition module 410 is used to obtain the myoelectric signals of the muscles on the wearing end of the artificial hand and the pressure signals generated by the muscles due to the bulge, and recognize the first movement intention based on the myoelectric signals and the pressure signals;

[0100] A first artificial hand control module 420, which can be used to generate a first control signal based on the first movement intention, so as to control the movement of the artificial hand based on the first control signal;

[0101] The second movement intention recognition module 430 may be used to obtain an angle signal of each finger joint on the prosthetic hand, an inertia signal of the prosthetic hand, and a gripping signal reflecting the gripping force applied by the prosthetic hand during the movement of the prosthetic hand, and recognize a second movement intention based on the angle signal, the inertia signal, and the gripping signal;

[0102] The second prosthetic hand control module 440 may be configured to generate a second control signal according to the first movement intention and the second movement intention, so as to control the movement of the prosthetic hand based on the second control signal.

[0103] Optionally, the first motion intention recognition module 410 may include:

[0104] A first movement intention obtaining unit is used to obtain a first movement intention recognition model, input the electromyographic signal and the pressure signal into the first movement intention recognition model, and obtain the first movement intention; and / or,

[0105] The second motion intention recognition module 430 may include:

[0106] The second motion intention obtaining unit is used to obtain a second motion intention recognition model, and input the angle signal, the inertia signal and the grasping signal into the second motion intention recognition model to obtain the second motion intention.

[0107] On this basis, optionally, the first motion intention recognition model is pre-trained by the following modules:

[0108] A training sample acquisition module is used to obtain sample electromyographic signals of muscles, sample pressure signals generated by muscle bulge, and movement intention labels corresponding to the sample electromyographic signals and sample pressure signals, and use the sample electromyographic signals, sample pressure signals and movement intention labels as a group of training samples;

[0109] The first motion intention recognition model obtaining module is used to train the original motion intention recognition model based on the obtained multiple groups of training samples to obtain the first motion intention recognition model.

[0110] Optionally, the second artificial hand control module 440 may include:

[0111] The artificial hand control unit can be used to send the second control signal to the single chip microcomputer, so that the single chip microcomputer generates a pulse width modulation signal based on the received second control signal, and controls the movement of the artificial hand through the pulse width modulation signal.

[0112] On this basis, optionally, the grasping signal is represented by a current signal for describing the working current of the servo motor arranged in the artificial hand, and the single chip microcomputer also works through the following modules:

[0113] The change rate obtaining module is used to obtain the angle signal and the current signal, and obtain the change rate of the bending angle of the finger joint according to the angle signal;

[0114] a pulse width modulation signal maintaining module, used for maintaining the pulse width modulation signal when the change rate is less than a preset change rate threshold and the current signal is within a preset current range; and / or,

[0115] The duty cycle increasing module is used to increase the duty cycle of the pulse width modulation signal when the current signal becomes smaller.

[0116] On this basis, optionally, when the single-chip microcomputer increases the duty cycle, the single-chip microcomputer also works through the following modules:

[0117] The duty cycle increase stopping module is used to stop increasing the duty cycle when the current signal is less than a preset safety current upper limit and the difference between the current signal and the safety current upper limit is less than or equal to a preset difference threshold.

[0118] Based on any of the above-mentioned artificial hand control devices, optionally, the angle signal is used to feedback the motion state of the artificial hand and / or the shape of the object grasped by the artificial hand; and / or,

[0119] The inertial signal is used to feed back at least one of the movement direction, movement speed and movement posture of the prosthetic hand.

[0120] The artificial hand control device provided in the embodiment of the present invention obtains the electromyographic signal of the muscle on the wearing end of the artificial hand and the pressure signal generated by the muscle due to the bulge through the first movement intention recognition module, and recognizes the first movement intention based on the electromyographic signal and the pressure signal, that is, the first movement intention is recognized based on the feedforward signal (that is, the electromyographic signal and the pressure signal); through the first artificial hand control module, a first control signal is generated based on the first movement intention to control the movement of the artificial hand based on the first control signal, that is, this movement process is controlled based on the feedforward signal; through the second movement intention recognition module, during the movement of the artificial hand, the angle signal of each finger joint on the artificial hand, the inertia signal of the artificial hand and the gripping signal used to reflect the gripping force applied by the artificial hand are obtained, and the second movement intention is recognized based on the angle signal, the inertia signal and the gripping signal, that is, the second movement intention is recognized based on the feedforward signal (that is, the angle signal, the inertia signal and the gripping signal); further, through the second artificial hand control module, a second control signal is generated according to the first movement intention and the second movement intention to control the movement of the artificial hand based on the second control signal, that is, this movement process is controlled based on the feedforward signal and the feedback signal. The above-mentioned device, during the movement of the prosthetic hand, especially during the interaction between the prosthetic hand and objects in the environment, corrects the first control signal of the prosthetic hand obtained based on the feedforward signal through the feedback signal collected from the prosthetic hand, thereby solving the problem of decreased stability of the prosthetic hand control caused by factors such as the influence of sweat on the electromyographic signal, electrode displacement and muscle fatigue, and improving the stability of the prosthetic hand control.

[0121] The artificial hand control device provided in the embodiment of the present invention can execute the artificial hand control method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0122] It is worth noting that in the embodiment of the above-mentioned prosthetic hand control device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0123] Figure 7A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0124] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0125] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0126] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a prosthetic hand control method.

[0127] In some embodiments, the prosthetic hand control method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the prosthetic hand control method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the prosthetic hand control method in any other appropriate manner (e.g., by means of firmware).

[0128] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0130] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0131] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0132] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0133] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0134] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0135] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for controlling a prosthetic hand, characterized in that: include: Acquire myoelectric signals of muscles on the wearing end of the artificial hand and pressure signals generated by the muscles due to bulging, and identify a first movement intention based on the myoelectric signals and the pressure signals; generating a first control signal based on the first movement intention, so as to control the movement of the prosthetic hand based on the first control signal; During the movement of the prosthetic hand, an angle signal of each finger joint on the prosthetic hand, an inertia signal of the prosthetic hand, and a gripping signal reflecting the gripping force applied by the prosthetic hand are acquired, and a second movement intention is identified based on the angle signal, the inertia signal, and the gripping signal; A second control signal is generated according to the first movement intention and the second movement intention, so as to control the movement of the prosthetic hand based on the second control signal.

2. The method according to claim 1, characterized in that The identifying a first movement intention based on the electromyographic signal and the pressure signal comprises: Acquire a first movement intention recognition model, input the electromyographic signal and the pressure signal into the first movement intention recognition model, and obtain a first movement intention; and / or, The identifying the second motion intention based on the angle signal, the inertia signal and the grip signal comprises: A second motion intention recognition model is acquired, and the angle signal, the inertia signal, and the grasping signal are input into the second motion intention recognition model to obtain a second motion intention.

3. The method according to claim 2, characterized in that The first motion intention recognition model is pre-trained by the following steps: Acquire sample electromyographic signals of the muscle, sample pressure signals generated by the muscle bulging, and movement intention labels corresponding to the sample electromyographic signals and the sample pressure signals, and use the sample electromyographic signals, the sample pressure signals, and the movement intention labels as a group of training samples; The original motion intention recognition model is trained based on the obtained multiple groups of training samples to obtain the first motion intention recognition model.

4. The method according to claim 1, characterized in that: The controlling the movement of the artificial hand based on the second control signal comprises: The second control signal is sent to the single chip microcomputer, so that the single chip microcomputer generates a pulse width modulation signal based on the received second control signal, and controls the movement of the artificial hand through the pulse width modulation signal.

5. The method according to claim 4, characterized in that The grasping signal is represented by a current signal for describing the working current of the servo motor arranged in the artificial hand, and the single chip microcomputer further performs the following steps: Acquiring the angle signal and the current signal, and obtaining the rate of change of the bending angle of the finger joint according to the angle signal; When the change rate is less than a preset change rate threshold and the current signal is within a preset current interval, maintaining the pulse width modulation signal; and / or, When the current signal decreases, the duty cycle of the pulse width modulation signal is increased.

6. The method according to claim 5, characterized in that When the single chip microcomputer increases the duty cycle, the single chip microcomputer further performs the following steps: When the current signal is less than a preset safety current upper limit, and the difference between the current signal and the safety current upper limit is less than or equal to a preset difference threshold, the duty cycle is stopped from being increased.

7. The method according to any one of claims 1 to 6, characterized in that The angle signal is used to feedback the motion state of the prosthetic hand and / or the shape of the object grasped by the prosthetic hand; and / or, The inertial signal is used to feed back at least one of the movement direction, movement speed and movement posture of the prosthetic hand.

8. A prosthetic hand control device, characterized in that: include: A first movement intention recognition module, used to obtain an electromyographic signal of a muscle on a wearing end of the artificial hand and a pressure signal generated by the muscle due to bulging, and recognize a first movement intention based on the electromyographic signal and the pressure signal; a first artificial hand control module, configured to generate a first control signal based on the first movement intention, so as to control the movement of the artificial hand based on the first control signal; a second movement intention recognition module, configured to obtain, during the movement of the prosthetic hand, an angle signal of each finger joint on the prosthetic hand, an inertia signal of the prosthetic hand, and a gripping signal reflecting the gripping force applied by the prosthetic hand, and recognize a second movement intention based on the angle signal, the inertia signal, and the gripping signal; The second artificial hand control module is used to generate a second control signal according to the first movement intention and the second movement intention, so as to control the movement of the artificial hand based on the second control signal.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the prosthetic hand control method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the prosthetic hand control method according to any one of claims 1 to 7 when executed.

Images