A method, device, electronic device and storage medium for controlling a prosthetic hand

Abstract

This invention discloses a method, device, electronic device, and storage medium for controlling a prosthetic hand. The method may include: acquiring electromyographic signals of muscles on the wearing end of the prosthetic hand and pressure signals generated by muscle bulge; identifying a first movement intention based on the electromyographic and pressure signals; generating a first control signal based on the first movement intention to control the movement of the prosthetic hand; during the movement of the prosthetic hand, acquiring angle signals of each finger joint on the prosthetic hand, inertial signals of the prosthetic hand, and gripping signals reflecting the gripping force applied by the prosthetic hand; identifying a second movement intention based on the angle signals, inertial signals, and gripping signals; and generating a second control signal based on the first and second movement intentions to control the movement of the prosthetic hand. The technical solution of this invention can improve the stability of prosthetic hand control.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of automatic control technology, and in particular to a prosthetic hand control method, device, electronic device and storage medium. Background Technology

[0002] In recent years, thanks to the rapid development of artificial intelligence and pattern recognition technologies, solutions for prosthetic hand control based on electromyography signals have become increasingly mature.

[0003] However, research has revealed that the current prosthetic hand control scheme suffers from decreased stability due to factors such as the influence of sweat on electromyographic signals, electrode displacement, and muscle fatigue in the prosthetic hand wearer. This issue urgently needs to be addressed. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for controlling a prosthetic hand, thereby improving the stability of prosthetic hand control.

[0005] According to one aspect of the present invention, a method for controlling a prosthetic hand is provided, which may include:

[0006] Acquire electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscles due to bulging, and identify the first movement intention based on the electromyographic signals and pressure signals;

[0007] A first control signal is generated based on a first motion intention, and the movement of the prosthetic hand is controlled based on the first control signal;

[0008] During the movement of the prosthetic hand, the angle signal of each finger joint on the prosthetic hand, the inertial signal of the prosthetic hand, and the grip signal that reflects the gripping force applied by the prosthetic hand are acquired, and the second movement intention is identified based on the angle signal, the inertial signal, and the grip signal.

[0009] A second control signal is generated based on the first and second movement intentions, so as to control the movement of the prosthetic hand based on the second control signal.

[0010] According to another aspect of the present invention, a prosthetic hand control device is provided, which may include:

[0011] The first motion intention recognition module is used to acquire electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscles due to bulging, and to identify the first motion intention based on the electromyographic signals and pressure signals.

[0012] The first prosthetic hand control module is used to generate a first control signal based on a first movement intention, so as to control the movement of the prosthetic hand based on the first control signal;

[0013] The second motion intention recognition module is used to acquire the angle signal of each finger joint on the prosthetic hand, the inertial signal of the prosthetic hand, and the grip signal that reflects the gripping force applied by the prosthetic hand during the movement of the prosthetic hand, and to identify the second motion intention based on the angle signal, the inertial signal and the grip signal.

[0014] The second prosthetic hand control module is used to generate a second control signal based on the first movement intention and the second movement intention, so as to control the movement of the prosthetic hand based on the second control signal.

[0015] According to another aspect of the present invention, an electronic device is provided, which may include:

[0016] At least one processor; and

[0017] A memory that is communicatively connected to at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by at least one processor, such that when the at least one processor executes the program, it implements the prosthetic hand control method provided in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided having computer instructions stored thereon for causing a processor to execute and implement the prosthetic hand control method provided in any embodiment of the present invention.

[0020] The technical solution of this invention involves acquiring electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscle bulge, and identifying a first movement intention based on the electromyographic signals and pressure signals. This first movement intention is obtained based on feedforward signals (i.e., electromyographic signals and pressure signals). A first control signal is generated based on the first movement intention to control the movement of the prosthetic hand; this movement process is controlled based on feedforward signals. During the movement of the prosthetic hand, angle signals of each finger joint, inertial signals of the prosthetic hand, and gripping signals reflecting the gripping force applied by the prosthetic hand are acquired. A second movement intention is identified based on the angle signals, inertial signals, and gripping signals; this second movement intention is obtained based on feedforward signals (i.e., angle signals, inertial signals, and gripping signals). Furthermore, a second control signal is generated based on the first and second movement intentions to control the movement of the prosthetic hand; this movement process is controlled based on feedforward signals and feedback signals. The above-mentioned technical solution, during the movement of the prosthetic hand, especially during the interaction between the prosthetic hand and objects in the environment, corrects the first control signal of the prosthetic hand obtained based on the feedforward signal by using feedback signals collected from the prosthetic hand. This solves the problem of decreased stability of prosthetic hand control caused by factors such as the influence of sweat, electrode displacement and muscle fatigue on electromyographic signals, and improves the stability of prosthetic hand control.

[0021] It should be understood that the description in this section is not intended to identify key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of a prosthetic hand control method provided according to an embodiment of the present invention;

[0024] Figure 2 This is a flowchart of another prosthetic hand control method provided according to an embodiment of the present invention;

[0025] Figure 3 This is a flowchart of another prosthetic hand control method provided according to an embodiment of the present invention;

[0026] Figure 4 This is a first schematic diagram of an optional example of another prosthetic hand control method provided according to an embodiment of the present invention;

[0027] Figure 5 This is a second schematic diagram of an optional example of another prosthetic hand control method provided according to an embodiment of the present invention;

[0028] Figure 6 This is a structural block diagram of a prosthetic hand control device provided according to an embodiment of the present invention;

[0029] Figure 7 This is a schematic diagram of the structure of an electronic device that implements the prosthetic hand control method of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. The same applies to "target," "original," etc., and will not be repeated here. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] Figure 1 This is a flowchart of a prosthetic hand control method provided in an embodiment of the present invention. This embodiment is applicable to prosthetic hand control, and is particularly suitable for prosthetic hand control by fusing feedforward and feedback signals. The method can be executed by the prosthetic hand control device provided in this embodiment of the present invention. This device can be implemented by software and / or hardware, and can be integrated into an electronic device (i.e., a host computer), which can be various user terminals or servers.

[0033] See Figure 1 The method of this invention specifically includes the following steps:

[0034] S110. Acquire electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscles due to bulging, and identify the first movement intention based on the electromyographic signals and pressure signals.

[0035] In this invention, the prosthetic hand can be understood as a prosthetic hand to be controlled. The wearer can be understood as the object wearing the prosthetic hand. In this embodiment of the invention, optionally, the object can be an amputee, especially a forearm amputee. The wearing end can be understood as the part of the wearer's body on which the prosthetic hand is worn. In this embodiment of the invention, optionally, this part can be a stump, that is, the stump of a forearm amputee.

[0036] Muscles can be understood as the muscles on the wearable end. Electromyography (EMG) signals can be understood as the electrical signals of the muscles. In this embodiment of the invention, optionally, an EMG sensor is integrated within the prosthetic hand, which can be used to collect the EMG signals. Pressure signals can be understood as the electrical signals generated by the muscles due to bulging. In this embodiment of the invention, optionally, a pressure sensor is integrated within the prosthetic hand; further, the pressure sensor can be integrated within the EMG sensor, which can then be used to collect the pressure signals.

[0037] The acquired electromyographic (EMG) and pressure signals are used to identify the wearer's primary movement intention, or more specifically, the movement intention of the prosthetic hand worn by the wearer. It is understood that, in this embodiment of the invention, the EMG and pressure signals are equivalent to feedforward signals.

[0038] S120. Generate a first control signal based on the first motion intention, so as to control the movement of the prosthetic hand based on the first control signal.

[0039] Specifically, a first control signal is generated based on a first motion intention, thereby controlling the movement of the prosthetic hand based on the first control signal, especially controlling the prosthetic hand to perform a movement process that conforms to the first motion intention.

[0040] S130. During the movement of the prosthetic hand, acquire the angle signal of each finger joint on the prosthetic hand, the inertial signal of the prosthetic hand, and the gripping signal to reflect the gripping force applied by the prosthetic hand, and identify the second movement intention based on the angle signal, the inertial signal, and the gripping signal.

[0041] The angle signal represents the bending angle of the finger joints on the prosthetic hand, particularly the change in this bending angle, thus providing feedback on the movement state of the prosthetic hand. Optionally, in this embodiment, the angle signal can be measured using an angle sensor or bending sensor mounted on the finger joints. Further, considering potential application scenarios, the prosthetic hand may interact with objects in the environment during movement; in this case, the angle signal can reflect the posture of the prosthetic hand when actually grasping the object, corresponding to the shape of the object being grasped. Furthermore, optionally, it can also indicate whether the grasping action of the prosthetic hand matches the wearer's actual movement intention.

[0042] Inertial signals can be measured by an inertial measurement unit (IMU) mounted on the prosthetic hand; the IMU can also be called an inertial sensor. Optionally, in this embodiment of the invention, the IMU can be mounted on the back of the prosthetic hand. Further, the IMU can measure inertial signals such as acceleration, angular velocity, and tilt angle of the prosthetic hand. Therefore, at least one of the following can be obtained from these inertial signals: the direction of motion, the speed of motion, and the posture of motion. Optionally, the posture can be, for example, a tilt angle and / or a rotation angle, allowing the prosthetic hand to perceive its own position in space.

[0043] A gripping signal can be understood as a signal used to characterize the gripping force applied by the prosthetic hand. In the context of possible application scenarios in this invention, this gripping force can be the force applied to an object when the prosthetic hand comes into contact with it. In this invention, optionally, the gripping signal can be acquired by a current sensor integrated within the prosthetic hand, a multi-point pressure sensor at the fingertips and palm, an ultrasonic sensor, or an optical sensor, etc., without specific limitations. Taking a current sensor as an example, the gripping signal is a current signal that reflects the change in current when the prosthetic hand comes into contact with an object. This current change can be a change in the operating current of a servo motor installed within the prosthetic hand, and the gripping force applied by the prosthetic hand is fed back through this current change.

[0044] During the movement of the prosthetic hand, angle signals, inertial signals, and grasping signals are acquired, and the wearer's second movement intention can be identified based on these three (i.e., three types of feedback signals).

[0045] S140. Generate a second control signal based on the first motion intention and the second motion intention, so as to control the movement of the prosthetic hand based on the second control signal.

[0046] Specifically, a second control signal is generated based on a first movement intention and a second movement intention. For example, the wearer's actual movement intention X can be obtained by analyzing these two movement intentions; or the first movement intention can be corrected by the second movement intention to obtain the movement intention X; and so on. Then, the second control signal can be generated based on the movement intention X. Therefore, compared to controlling the movement of the prosthetic hand using a first control signal generated solely based on the first movement intention, controlling the movement of the prosthetic hand using a second control signal generated jointly by the first and second movement intentions allows the movement process to be more closely matched with the movement intention X.

[0047] Three points need to be clarified here. First, since the embodiments of the present invention do not use only feedforward signals for prosthetic hand control, but combine feedforward signals and feedback signals for prosthetic hand control, this allows for a suitable reduction in the number of electrodes of the electromyography sensor attached to the wearing end, thereby improving the convenience for the wearer when wearing the prosthetic hand. Second, the embodiments of the present invention integrate a sensor for measuring feedback signals at the prosthetic hand end, which eliminates the need to add an additional sensor for measuring feedforward signals at the wearing end, reducing hardware complexity and making the acquisition of control signals simpler and more accurate. Third, the embodiments of the present invention can be applied not only in the field of medical rehabilitation, but also in the industrial field for performing dangerous or delicate operational tasks, such as handling toxic chemicals or assembling precision instruments, etc., without specific limitations.

[0048] The technical solution of this invention involves acquiring electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscle bulge, and identifying a first movement intention based on the electromyographic signals and pressure signals. This first movement intention is obtained based on feedforward signals (i.e., electromyographic signals and pressure signals). A first control signal is generated based on the first movement intention to control the movement of the prosthetic hand; this movement process is controlled based on feedforward signals. During the movement of the prosthetic hand, angle signals of each finger joint, inertial signals of the prosthetic hand, and gripping signals reflecting the gripping force applied by the prosthetic hand are acquired. A second movement intention is identified based on the angle signals, inertial signals, and gripping signals; this second movement intention is obtained based on feedforward signals (i.e., angle signals, inertial signals, and gripping signals). Furthermore, a second control signal is generated based on the first and second movement intentions to control the movement of the prosthetic hand; this movement process is controlled based on feedforward signals and feedback signals. The above-mentioned technical solution, during the movement of the prosthetic hand, especially during the interaction between the prosthetic hand and objects in the environment, corrects the first control signal of the prosthetic hand obtained based on the feedforward signal by using feedback signals collected from the prosthetic hand. This solves the problem of decreased stability of prosthetic hand control caused by factors such as the influence of sweat, electrode displacement and muscle fatigue on electromyographic signals, and improves the stability of prosthetic hand control.

[0049] Figure 2 This is a flowchart of another prosthetic hand control method provided in this embodiment of the invention. This embodiment is an optimization based on the above-described technical solutions. In this embodiment, optionally, identifying a first movement intention based on electromyographic signals and pressure signals includes: acquiring a first movement intention recognition model, inputting electromyographic signals and pressure signals into the first movement intention recognition model, and obtaining the first movement intention; and / or, identifying a second movement intention based on angle signals, inertial signals, and grasping signals may include: acquiring a second movement intention recognition model, and inputting angle signals, inertial signals, and grasping signals into the second movement intention recognition model to obtain the second movement intention. The explanations of terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0050] See Figure 2 The method in this embodiment may specifically include the following steps:

[0051] S210. Acquire electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscles due to bulging.

[0052] S220. Obtain the first motor intention recognition model by inputting electromyographic signals and pressure signals into the first motor intention recognition model to obtain the first motor intention.

[0053] S230. Generate a first control signal based on the first motion intention, so as to control the movement of the prosthetic hand based on the first control signal.

[0054] S240. During the movement of the prosthetic hand, acquire the angle signal of each finger joint on the prosthetic hand, the inertia signal of the prosthetic hand, and the grip signal to reflect the gripping force applied by the prosthetic hand.

[0055] S250. Obtain the second motion intention recognition model, and input the angle signal, inertial signal and grip signal into the second motion intention recognition model to obtain the second motion intention.

[0056] In this embodiment of the invention, optionally, the second motion intention recognition model and the first motion intention recognition model can be the same motion intention recognition model, or in particular, they can be different motion intention recognition models, which helps to improve the accuracy of motion intention recognition.

[0057] S260. Generate a second control signal based on the first motion intention and the second motion intention, so as to control the movement of the prosthetic hand based on the second control signal.

[0058] The technical solution of this invention achieves accurate recognition of motion intent by applying a motion intent recognition model.

[0059] An alternative technical solution involves pre-training the first motion intent recognition model through the following steps:

[0060] Acquire sample electromyographic signals of muscles and sample pressure signals generated by muscle bulging, as well as motion intention labels corresponding to sample electromyographic signals and sample pressure signals, and use sample electromyographic signals, sample pressure signals and motion intention labels as a set of training samples.

[0061] The original motion intent recognition model is trained based on the obtained multiple sets of training samples to obtain the first motion intent recognition model.

[0062] In this technical solution, optionally, a corresponding first motion intention recognition model can be trained for different wearers, that is, personalized training of the first motion intention recognition model can be carried out, thereby further improving the accuracy of motion intention recognition.

[0063] Figure 3 This is a flowchart of another prosthetic hand control method provided in this embodiment of the invention. This embodiment is an optimization based on the above-mentioned technical solutions. In this embodiment, optionally, controlling the movement of the prosthetic hand based on a second control signal includes: sending the second control signal to a microcontroller, so that the microcontroller generates a pulse width modulation signal based on the received second control signal, and controls the movement of the prosthetic hand through the pulse width modulation signal. The explanations of terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0064] See Figure 3 The method in this embodiment may specifically include the following steps:

[0065] S310. Acquire electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscles due to bulging, and identify the first movement intention based on the electromyographic signals and pressure signals.

[0066] S320. Generate a first control signal based on the first motion intention, so as to control the movement of the prosthetic hand based on the first control signal.

[0067] In one embodiment of the invention, optionally, the first control signal can be sent to the microcontroller. In this way, the microcontroller can generate a pulse width modulation (PWM) signal based on the received first control signal and control the movement of the prosthetic hand through the PWM signal.

[0068] S330. During the movement of the prosthetic hand, acquire the angle signal of each finger joint on the prosthetic hand, the inertial signal of the prosthetic hand, and the gripping signal to reflect the gripping force applied by the prosthetic hand, and identify the second movement intention based on the angle signal, the inertial signal, and the gripping signal.

[0069] S340. Generate a second control signal based on the first motion intention and the second motion intention.

[0070] S350. Send the second control signal to the microcontroller so that the microcontroller generates a pulse width modulation signal based on the received second control signal and controls the movement of the prosthetic hand through the pulse width modulation signal.

[0071] In this process, the second control signal is sent to the microcontroller. The microcontroller can then generate a PWM signal based on the received second control signal. It should be noted that the PWM signal generated in this step is the PWM signal corresponding to the second control signal, not the PWM signal corresponding to the first control signal generated in S320. Then, the microcontroller can control the movement of the prosthetic hand through this PWM signal.

[0072] In the technical solution of this invention embodiment, the microcontroller achieves effective control of the prosthetic hand through PWM signals.

[0073] In one alternative technical solution, the gripping signal is represented by a current signal describing the operating current of a servo motor located within the prosthetic hand, and the microcontroller further performs the following steps:

[0074] Acquire angle and current signals;

[0075] Based on the angle signal, the rate of change of the bending angle of the finger joint is obtained;

[0076] The pulse width modulation signal is maintained when the rate of change is less than a preset rate of change threshold and the current signal is within a preset current range.

[0077] The current signal can be understood as a signal describing the operating current of the servo motor installed in the prosthetic hand, and the gripping signal is represented by this current signal. The rate of change of the bending angle, ΔAngle, is obtained from the angle signal. If ΔAngle is less than a preset rate of change threshold, it indicates that the bending angle is in a stable state; and if the current signal is within a preset current range, it indicates that the operating current is in a stable state. When both the bending angle and the operating current are in a stable state, it means that the prosthetic hand has grasped the object. The microcontroller can then maintain the current PWM signal to avoid unnecessary adjustments to the gripping force and ensure stable gripping of the object.

[0078] Another alternative technical solution is that the grip signal can be represented by a current signal that describes the operating current of the servo motor set inside the prosthetic hand. The microcontroller also performs the following steps:

[0079] When the current signal decreases, increase the duty cycle of the pulse width modulation signal.

[0080] If the current signal decreases, meaning the operating current decreases, this typically occurs when the grip weakens due to the object slipping. In this case, the grip strength can be increased by increasing the duty cycle of the PWM signal to compensate for the slipping tendency, thereby preventing the object from slipping and being damaged. This is an anti-slip mechanism.

[0081] Based on this, optionally, the microcontroller also performs the following steps:

[0082] If the current signal is less than the preset safe current limit and the difference between the current signal and the safe current limit is less than or equal to the preset difference threshold, the duty cycle will stop increasing.

[0083] The safe current limit can be understood as the upper limit of the working current that can avoid damage to the object and the prosthetic hand. After increasing the duty cycle to increase the current signal, if the current signal is less than the preset safe current limit, and the difference between the current signal and the safe current limit is less than or equal to the preset difference threshold, that is, if the current signal is close to the safe current limit, then the increase in duty cycle can be stopped, that is, the increase in current signal can be stopped, and thus the increase in gripping force can be stopped, thereby ensuring the safety of the prosthetic hand and the object.

[0084] Based on this, in order to better understand the various technical solutions described above as a whole, the following examples will be provided for illustration. For example, see [link to example]. Figure 4 and Figure 5 The specific implementation process is as follows:

[0085] Step 1: Technicians locate several muscles with strong muscle activity on the forearm amputee's stump, wipe them with alcohol, and then attach a prosthetic hand equipped with a multi-channel electromyography (EMG) sensor to the stump, preparing for EMG signal acquisition. Optionally, in this example, the multi-channel EMG sensor may integrate a pressure sensor, enabling simultaneous acquisition of EMG and pressure signals.

[0086] Step 2: Open the serial port connection, ensure normal communication between the EMG sensor and the host computer software, calibrate the signal reception status, and begin real-time monitoring of EMG and pressure signals. After confirming stable signal transmission, technicians can begin collecting EMG and pressure signals and performing further processing before sending them to the host computer.

[0087] This example uses electromyography (EMG) signals; the same principle applies to pressure signals. In this example, the EMG signals can be processed by a signal processing unit, which consists of a signal acquisition module (AD7606) and a microcontroller (STM32F767IGT6). The signal acquisition module acquires the EMG signals and then sends them to the microcontroller via a Serial Peripheral Interface (SPI) for median-mean filtering. The processed EMG signals are then sent to a host computer via a Serial Communications Interface (SCI) for further processing. The host computer can display the EMG signals and perform preprocessing, such as extracting frequency domain features from the time-domain EMG signals using a Fast Fourier Transform.

[0088] The following explanation uses electromyography (EMG) signals as an example. However, in practical applications, the same steps can be used to process pressure signals, as both EMG signals and pressure signals are needed for model training and motion intention recognition.

[0089] Step 3: The prosthetic hand performs a preset muscle activity. At this time, the electromyography (EMG) sensor collects EMG signals, and the technician maps this muscle activity to a specific motion intention via host computer software. In this example, the motion intention can be understood as the prosthetic hand's movement, that is, the combination of position data of each servo in the prosthetic hand.

[0090] Step 4: Operate on the host computer software to preprocess the electromyographic signals corresponding to each collected movement intention to obtain frequency domain features. Then, train the model based on these features and monitor the model's loss reduction in real time through the software's visual interface. If the model learns normally, it can be saved for subsequent control of the prosthetic hand based on real-time electromyographic signals. If the model training effect is not good, you can start from Step 3 again to have the residual end generate electromyographic signals with greater differences in muscle activity so that the model can obtain higher quality electromyographic signals for model training.

[0091] Step 5: Using the motion intention recognition model saved in Step 4, the current motion intention of the forearm amputee is identified based on real-time electromyography signals. Based on the recognition results, a control signal is generated and sent to the microcontroller. The microcontroller sends a PWM signal to control the rotation angle of the servo motor of the traction tendon rope, thereby driving the fingers of the prosthetic hand to perform flexion and extension movements, realizing various movements of the prosthetic hand.

[0092] Step 6: During the real-time interaction between the prosthetic hand and objects in the environment, feedback signals collected by the angle sensor, IMU, and current sensor are uploaded to the host computer via the microcontroller. The motion intent is then recognized by the motion intent recognition model, and the servo motor is fine-tuned accordingly to make the control of the prosthetic hand more stable. Optionally, in this example, if an abnormality occurs during the gripping process (such as applying excessive or insufficient gripping force), the microcontroller can immediately adjust the PWM signal based on the feedback signal to avoid damaging the prosthetic hand or the object, thereby ensuring the safety and accuracy of the prosthetic hand's operation.

[0093] The above example has at least the following advantages:

[0094] 1) By integrating angle sensors, IMUs, and current sensors into the prosthetic hand, without adding additional sensors at the residual end for measuring feedforward signals, hardware complexity is reduced, while control signal acquisition becomes simpler and more accurate.

[0095] 2) The angle sensor can provide real-time feedback on the bending angle of the finger joints of the prosthetic hand, enabling the host computer to perceive the posture of the prosthetic hand when actually grasping an object, and accurately determine the shape of the object and whether the grasping action is successful through this posture, thereby significantly improving the accuracy of the prosthetic hand operation.

[0096] 3) The current sensor can monitor the change in the working current of the servo motor when the prosthetic hand comes into contact with an object, and provide real-time feedback on the gripping force applied by the prosthetic hand through this change. This ensures that the host computer can dynamically adjust the movement of the prosthetic hand based on the gripping force feedback, thereby achieving more precise gripping force control and improving the stability and safety of gripping.

[0097] 4) By multimodal fusion of inertial signals, angle signals, and current signals (i.e., various feedback signals) with electromyographic information and pressure signals (i.e., various feedforward signals), this example can provide more comprehensive feedback information to the host computer, improving the stability and responsiveness of the prosthetic hand in complex operating scenarios.

[0098] Figure 6 This is a structural block diagram of a prosthetic hand control device provided in an embodiment of the present invention. This device is used to execute the prosthetic hand control method provided in any of the above embodiments. This device and the prosthetic hand control methods of the above embodiments belong to the same inventive concept. Details not described in detail in the embodiments of the prosthetic hand control device can be found in the embodiments of the above prosthetic hand control methods. See also... Figure 6 The device may specifically include: a first motion intention recognition module 410, a first prosthetic hand control module 420, a second motion intention recognition module 430, and a second prosthetic hand control module 440.

[0099] The first motion intention recognition module 410 is used to acquire the electromyographic signals of the muscles on the wearing end of the prosthetic hand and the pressure signals generated by the muscles due to the bulge, and to recognize the first motion intention based on the electromyographic signals and the pressure signals.

[0100] The first prosthetic hand control module 420 can be used to generate a first control signal based on a first motion intention, so as to control the movement of the prosthetic hand based on the first control signal;

[0101] The second motion intention recognition module 430 can be used to acquire the angle signal of each finger joint on the prosthetic hand, the inertial signal of the prosthetic hand, and the gripping signal that reflects the gripping force applied by the prosthetic hand during the movement of the prosthetic hand, and to identify the second motion intention based on the angle signal, the inertial signal and the gripping signal.

[0102] The second prosthetic hand control module 440 can be used to generate a second control signal based on the first movement intention and the second movement intention, so as to control the movement of the prosthetic hand based on the second control signal.

[0103] Optionally, the first motion intent recognition module 410 may include:

[0104] The first motor intention acquisition unit is used to acquire a first motor intention recognition model by inputting electromyographic signals and pressure signals into the first motor intention recognition model to obtain the first motor intention; and / or,

[0105] The second motion intention recognition module 430 may include:

[0106] The second motion intent acquisition unit is used to acquire the second motion intent recognition model and input the angle signal, inertial signal and grip signal into the second motion intent recognition model to obtain the second motion intent.

[0107] Based on this, optionally, the first motion intent recognition model is pre-trained using the following modules:

[0108] The training sample acquisition module is used to acquire the sample electromyographic signals of the muscle and the sample pressure signals generated by the muscle due to bulging, as well as the motion intention labels corresponding to the sample electromyographic signals and sample pressure signals, and to use the sample electromyographic signals, sample pressure signals and motion intention labels as a set of training samples.

[0109] The first motion intent recognition model acquisition module is used to train the original motion intent recognition model based on the obtained multiple sets of training samples to obtain the first motion intent recognition model.

[0110] Optionally, the second prosthetic hand control module 440 may include:

[0111] The prosthetic hand control unit can be used to send a second control signal to a microcontroller, so that the microcontroller can generate a pulse width modulation signal based on the received second control signal, and control the movement of the prosthetic hand through the pulse width modulation signal.

[0112] Optionally, the gripping signal is represented by a current signal that describes the operating current of the servo motor located inside the prosthetic hand. The microcontroller also operates through the following modules:

[0113] The rate of change module is used to acquire angle and current signals, and to obtain the rate of change of the bending angle of the finger joint based on the angle signal.

[0114] A pulse width modulation (PWM) signal sustaining module is used to sustain the PWM signal when the rate of change is less than a preset rate of change threshold and the current signal is within a preset current range; and / or,

[0115] The duty cycle increase module is used to increase the duty cycle of the pulse width modulation signal when the current signal becomes smaller.

[0116] Based on this, optionally, when the duty cycle of the microcontroller is increased, the microcontroller also operates through the following modules:

[0117] The duty cycle stop increasing module is used to stop increasing the duty cycle when the current signal is less than the preset safe current upper limit and the difference between the current signal and the safe current upper limit is less than or equal to the preset difference threshold.

[0118] Based on any of the aforementioned prosthetic hand control devices, optionally, an angle signal is used to provide feedback on the motion state of the prosthetic hand and / or the shape of the object grasped by the prosthetic hand; and / or,

[0119] Inertial signals are used to provide feedback on at least one of the following: the direction of motion, speed of motion, and posture of the prosthetic hand.

[0120] The prosthetic hand control device provided in this embodiment of the invention acquires electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscle bulge through a first motion intention recognition module, and identifies a first motion intention based on the electromyographic signals and pressure signals, that is, the first motion intention is identified based on feedforward signals (i.e., electromyographic signals and pressure signals); the first prosthetic hand control module generates a first control signal based on the first motion intention, and controls the movement of the prosthetic hand based on the first control signal, that is, this movement process is controlled based on feedforward signals; the second motion intention recognition module acquires angle signals of each finger joint on the prosthetic hand, inertial signals of the prosthetic hand, and gripping signals reflecting the gripping force applied by the prosthetic hand during the movement of the prosthetic hand, and identifies a second motion intention based on the angle signals, inertial signals, and gripping signals, that is, the second motion intention is identified based on feedforward signals (i.e., angle signals, inertial signals, and gripping signals); furthermore, the second prosthetic hand control module generates a second control signal according to the first motion intention and the second motion intention, and controls the movement of the prosthetic hand based on the second control signal, that is, this movement process is controlled based on feedforward signals and feedback signals. The aforementioned device, during the movement of the prosthetic hand, especially during the interaction between the prosthetic hand and objects in the environment, corrects the first control signal of the prosthetic hand obtained based on the feedforward signal by collecting feedback signals from the prosthetic hand. This solves the problem of decreased stability of prosthetic hand control caused by factors such as the influence of sweat, electrode displacement and muscle fatigue on electromyographic signals, and improves the stability of prosthetic hand control.

[0121] The prosthetic hand control device provided in the embodiments of the present invention can execute the prosthetic hand control method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0122] It is worth noting that in the embodiments of the above-mentioned prosthetic hand control device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0123] Figure 7A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0124] like Figure 7 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0125] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0126] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as prosthetic hand control methods.

[0127] In some embodiments, the prosthetic hand control method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the prosthetic hand control method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the prosthetic hand control method by any other suitable means (e.g., by means of firmware).

[0128] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0130] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0131] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0132] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0133] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0134] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0135] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for controlling a prosthetic hand, characterized in that, include: Acquire electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscles due to bulging, and identify a first movement intention based on the electromyographic signals and the pressure signals; A first control signal is generated based on the first motion intention, so as to control the movement of the prosthetic hand based on the first control signal; During the movement of the prosthetic hand, the angle signal of each finger joint on the prosthetic hand, the inertia signal of the prosthetic hand, and the gripping signal reflecting the gripping force applied by the prosthetic hand are acquired, and the second movement intention is identified based on the angle signal, the inertia signal, and the gripping signal. The first motion intention is corrected according to the second motion intention, and a second control signal is generated according to the corrected motion intention. The second control signal is sent to the microcontroller so that the microcontroller generates a pulse width modulation signal based on the received second control signal and controls the movement of the prosthetic hand through the pulse width modulation signal. The gripping signal is represented by a current signal that describes the operating current of the servo motor located within the prosthetic hand. The microcontroller also performs the following steps: Acquire the angle signal and the current signal, and obtain the rate of change of the bending angle of the finger joint based on the angle signal; The pulse width modulation signal is maintained when the rate of change is less than a preset rate of change threshold and the current signal is within a preset current range.

2. The method according to claim 1, characterized in that, The process of identifying a first motor intention based on the electromyographic signal and the pressure signal includes: A first movement intention recognition model is obtained by inputting the electromyographic signal and the pressure signal into the first movement intention recognition model to obtain the first movement intention; and / or, The process of identifying the second movement intention based on the angle signal, the inertial signal, and the grip signal includes: A second motion intention recognition model is obtained, and the angle signal, the inertial signal, and the grip signal are input into the second motion intention recognition model to obtain the second motion intention.

3. The method according to claim 2, characterized in that, The first motion intent recognition model is pre-trained through the following steps: Acquire sample electromyographic signals of the muscle and sample pressure signals generated by the muscle due to bulging, as well as motion intention labels corresponding to the sample electromyographic signals and the sample pressure signals, and use the sample electromyographic signals, the sample pressure signals and the motion intention labels as a set of training samples; The original motion intention recognition model is trained based on the obtained training samples to obtain the first motion intention recognition model.

4. The method according to claim 1, characterized in that, The microcontroller also performs the following steps: When the current signal decreases, the duty cycle of the pulse width modulation signal is increased.

5. The method according to claim 4, characterized in that, The microcontroller also performs the following steps: If the current signal is less than the preset safe current limit and the difference between the current signal and the safe current limit is less than or equal to the preset difference threshold, the duty cycle is stopped from increasing.

6. The method according to any one of claims 1-5, characterized in that, The angle signal is used to provide feedback on the motion state of the prosthetic hand and / or the shape of the object grasped by the prosthetic hand; and / or, The inertial signal is used to provide feedback on at least one of the following: the direction of movement, the speed of movement, and the posture of movement of the prosthetic hand.

7. A prosthetic hand control device, characterized in that, include: The first motion intention recognition module is used to acquire electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscles due to bulging, and to identify the first motion intention based on the electromyographic signals and the pressure signals. The first prosthetic hand control module is used to generate a first control signal based on the first movement intention, so as to control the movement of the prosthetic hand based on the first control signal; The second motion intention recognition module is used to acquire angle signals of each finger joint on the prosthetic hand, inertial signals of the prosthetic hand, and grasping signals reflecting the grasping force applied by the prosthetic hand during the movement of the prosthetic hand, and to identify the second motion intention based on the angle signals, the inertial signals, and the grasping signals. The second prosthetic hand control module is used to correct the first motion intention according to the second motion intention, and generate a second control signal according to the corrected motion intention, so as to control the movement of the prosthetic hand based on the second control signal; The second prosthetic hand control module includes: The prosthetic hand control unit is used to send the second control signal to the microcontroller, so that the microcontroller generates a pulse width modulation signal based on the received second control signal, and controls the movement of the prosthetic hand through the pulse width modulation signal; The gripping signal is represented by a current signal describing the operating current of the servo motor installed in the prosthetic hand. The microcontroller also operates through the following modules: The rate of change module is used to acquire the angle signal and the current signal, and to obtain the rate of change of the bending angle of the finger joint based on the angle signal; A pulse width modulation signal sustaining module is used to sustain the pulse width modulation signal when the rate of change is less than a preset rate of change threshold and the current signal is within a preset current range.

8. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that is executed by the at least one processor to cause the at least one processor to perform the prosthetic hand control method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute and implement the prosthetic hand control method as described in any one of claims 1-6.

Images