Prosthetic hand control method and apparatus, electronic device, and storage medium

Abstract

Provided is a prosthetic hand control method, which comprises: acquiring an electromyographic signal of a muscle at a wearing end of a prosthetic hand and a pressure signal generated by the muscle bulging, and identifying a first motion intention on the basis of the electromyographic signal and the pressure signal (S110); generating a first control signal on the basis of the first motion intention, so as to control the motion of the prosthetic hand on the basis of the first control signal (S120); during the motion of the prosthetic hand, acquiring an angle signal of each finger joint on the prosthetic hand, an inertial signal of the prosthetic hand, and a gripping signal used for reflecting the gripping force applied by the prosthetic hand, and identifying a second motion intention on the basis of the angle signal, the inertial signal, and the gripping signal (S130); and generating a second control signal on the basis of the first motion intention and the second motion intention, so as to control the motion of the prosthetic hand on the basis of the second control signal (S140). A prosthetic hand control apparatus, a storage medium storing instructions for executing the prosthetic hand control method, and an electronic device with a memory are also provided.

Description

Prosthetic hand control methods, devices, electronic equipment and storage media

[0001] This application claims priority to Chinese Patent Application No. 202411825862.8, filed with the Chinese Patent Office on December 12, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of automatic control technology, such as a prosthetic hand control method, device, electronic device, and storage medium. Background Technology

[0003] Thanks to the rapid development of artificial intelligence and pattern recognition technologies, solutions for controlling prosthetic hands based on electromyography signals are becoming increasingly mature.

[0004] However, research has revealed that the prosthetic hand control scheme suffers from decreased stability due to factors such as the influence of sweat on electromyographic signals, electrode displacement, and muscle fatigue in the prosthetic hand wearer. This issue urgently needs to be addressed. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for controlling a prosthetic hand, thereby improving the stability of prosthetic hand control.

[0006] This application provides a method for controlling a prosthetic hand, comprising: acquiring electromyographic signals of muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscles due to bulging, and identifying a first movement intention based on the electromyographic signals and the pressure signals; generating a first control signal based on the first movement intention to control the movement of the prosthetic hand based on the first control signal; during the movement of the prosthetic hand, acquiring angle signals of each finger joint on the prosthetic hand, inertial signals of the prosthetic hand, and gripping signals reflecting the gripping force applied by the prosthetic hand, and identifying a second movement intention based on the angle signals, the inertial signals, and the gripping signals; and generating a second control signal according to the first movement intention and the second movement intention to control the movement of the prosthetic hand based on the second control signal.

[0007] This application provides a prosthetic hand control device, comprising: a first motion intention recognition module, configured to acquire electromyographic signals of muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscles due to bulging, and to identify a first motion intention based on the electromyographic signals and the pressure signals; a first prosthetic hand control module, configured to generate a first control signal based on the first motion intention, and to control the movement of the prosthetic hand based on the first control signal; a second motion intention recognition module, configured to acquire angle signals of each finger joint on the prosthetic hand, inertial signals of the prosthetic hand, and gripping signals reflecting the gripping force applied by the prosthetic hand during the movement of the prosthetic hand, and to identify a second motion intention based on the angle signals, the inertial signals, and the gripping signals; and a second prosthetic hand control module, configured to generate a second control signal according to the first motion intention and the second motion intention, and to control the movement of the prosthetic hand based on the second control signal.

[0008] This application provides an electronic device that may include: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to implement the prosthetic hand control method provided in any embodiment of this application when executed by the at least one processor.

[0009] This application provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the prosthetic hand control method provided in any embodiment of this application. Attached Figure Description

[0010] Figure 1 is a flowchart of a prosthetic hand control method according to an embodiment of this application;

[0011] Figure 2 is a flowchart of another prosthetic hand control method provided according to an embodiment of this application;

[0012] Figure 3 is a flowchart of another prosthetic hand control method provided according to an embodiment of this application;

[0013] Figure 4 is a first schematic diagram of an optional example of another prosthetic hand control method provided according to an embodiment of this application;

[0014] Figure 5 is a second schematic diagram of an optional example of another prosthetic hand control method provided according to an embodiment of this application;

[0015] Figure 6 is a structural block diagram of a prosthetic hand control device according to an embodiment of this application;

[0016] Figure 7 is a schematic diagram of the structure of an electronic device that implements the prosthetic hand control method of the present application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort should fall within the scope of protection of this application.

[0018] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The same applies to "target," "original," etc., and will not be repeated here. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] Figure 1 is a flowchart of a prosthetic hand control method provided in an embodiment of this application. This embodiment is applicable to prosthetic hand control, such as prosthetic hand control through the fusion of feedforward and feedback signals. This method can be executed by the prosthetic hand control device provided in this embodiment, which can be implemented in software and / or hardware. The device can be integrated into an electronic device (i.e., a host computer), which can be various user terminals or a server.

[0020] Referring to Figure 1, the method of this application embodiment includes the following steps:

[0021] S110. Acquire electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscles due to bulging, and identify the first movement intention based on the electromyographic signals and pressure signals.

[0022] The prosthetic hand can be understood as a prosthetic hand to be controlled. The wearer can be understood as the object wearing the prosthetic hand. In this embodiment, optionally, the object can be an amputee, such as a forearm amputee. The wearing end can be understood as the part of the wearer's body on which the prosthetic hand is worn. In this embodiment, optionally, the part can be a stump, that is, the stump of a forearm amputee.

[0023] Muscles can be understood as the muscles on the wearable end. Electromyography (EMG) signals can be understood as the electrical signals of the muscles. In this embodiment, optionally, an EMG sensor is integrated within the prosthetic hand, which can be used to collect the EMG signals. Pressure signals can be understood as the electrical signals generated by the muscles due to bulging. In this embodiment, optionally, a pressure sensor is integrated within the prosthetic hand, and the pressure sensor can also be integrated within the EMG sensor, in which case the pressure sensor can be used to collect the pressure signals.

[0024] The acquired electromyographic (EMG) and pressure signals can be used to identify the wearer's initial movement intention, or the movement intention of the prosthetic hand worn by the wearer. It is understood that, in this embodiment, the EMG and pressure signals are equivalent to feedforward signals.

[0025] S120. Generate a first control signal based on the first motion intention, so as to control the movement of the prosthetic hand based on the first control signal.

[0026] A first control signal is generated based on a first motion intention, thereby controlling the movement of the prosthetic hand based on the first control signal, for example, controlling the prosthetic hand to perform a movement process that conforms to the first motion intention.

[0027] S130. During the movement of the prosthetic hand, acquire the angle signal of each finger joint on the prosthetic hand, the inertial signal of the prosthetic hand, and the gripping signal to reflect the gripping force applied by the prosthetic hand, and identify the second movement intention based on the angle signal, the inertial signal, and the gripping signal.

[0028] The angle signal represents the bending angle of the finger joints on the prosthetic hand, for example, representing the change in this bending angle, thus providing feedback on the movement state of the prosthetic hand. Optionally, in this embodiment, the angle signal can be measured using an angle sensor or bending sensor mounted on the finger joints. Further, considering the application scenarios that this embodiment may involve, the prosthetic hand may interact with objects in the environment during movement; in this case, the angle signal can provide feedback on the posture of the prosthetic hand when actually grasping the object, and this posture corresponds to the shape of the object grasped by the prosthetic hand. Optionally, it can also provide feedback on whether the grasping action of the prosthetic hand matches the wearer's actual movement intention.

[0029] Inertial signals can be measured by an inertial measurement unit (IMU) mounted on the prosthetic hand; the IMU can also be called an inertial sensor. Optionally, in this embodiment, the IMU can be mounted on the back of the prosthetic hand. Further, the IMU can measure inertial signals such as acceleration, angular velocity, and tilt angle of the prosthetic hand. Therefore, at least one of the following can be obtained from these inertial signals: the direction of motion, the speed of motion, and the posture of motion. Optionally, the posture can be, for example, a tilt angle and / or a rotation angle, allowing the prosthetic hand to perceive its position in space.

[0030] A gripping signal can be understood as a signal used to characterize the gripping force applied by the prosthetic hand. In the context of possible applications in this application, this gripping force can be the force applied to an object when the prosthetic hand comes into contact with it. Optionally, in this application, the gripping signal can be acquired by a current sensor integrated within the prosthetic hand, a multi-point pressure sensor at the fingertips and palm, an ultrasonic sensor, or an optical sensor, etc., without limitation. Taking a current sensor as an example, the gripping signal is a current signal that reflects the change in current when the prosthetic hand comes into contact with an object. This current change can be a change in the operating current of a servo motor installed within the prosthetic hand, and the gripping force applied by the prosthetic hand is fed back through this current change.

[0031] During the movement of the prosthetic hand, angle signals, inertial signals, and grasping signals are acquired, and the wearer's second movement intention can be identified based on these three (i.e., three types of feedback signals).

[0032] S140. Generate a second control signal based on the first motion intention and the second motion intention, so as to control the movement of the prosthetic hand based on the second control signal.

[0033] A second control signal is generated based on the first and second movement intentions. For example, the wearer's actual movement intention X can be obtained by analyzing these two movement intentions; or the first movement intention can be corrected by the second movement intention to obtain the movement intention X; and so on. Then, the second control signal can be generated based on the movement intention X. Therefore, compared to controlling the movement of the prosthetic hand using a first control signal generated solely based on the first movement intention, controlling the movement of the prosthetic hand using a second control signal generated jointly by the first and second movement intentions can make the movement process more closely match the movement intention X.

[0034] Three points need to be clarified here. First, since the embodiments of this application do not use only feedforward signals for prosthetic hand control, but combine feedforward signals and feedback signals for prosthetic hand control, this allows for a suitable reduction in the number of electrodes of the electromyography sensor attached to the wearing end, thereby improving the convenience for the wearer when wearing the prosthetic hand. Second, the embodiments of this application integrate a sensor for measuring feedback signals at the prosthetic hand end, which eliminates the need to add an additional sensor for measuring feedforward signals at the wearing end, reducing hardware complexity and making the acquisition of control signals simpler and more accurate. Third, the embodiments of this application can be applied not only in the field of medical rehabilitation, but also in the industrial field to perform dangerous or delicate operational tasks, such as handling toxic chemicals or assembling precision instruments, etc., without limitation.

[0035] The technical solution of this application embodiment acquires electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscle bulge, and identifies a first movement intention based on the electromyographic signals and pressure signals, that is, the first movement intention is identified based on feedforward signals (i.e., electromyographic signals and pressure signals); generates a first control signal based on the first movement intention to control the movement of the prosthetic hand based on the first control signal, that is, this movement process is controlled based on feedforward signals; during the movement of the prosthetic hand, acquires the angle signals of each finger joint on the prosthetic hand, the inertial signals of the prosthetic hand, and the gripping signals reflecting the gripping force applied by the prosthetic hand, and identifies a second movement intention based on the angle signals, inertial signals, and gripping signals, that is, the second movement intention is identified based on feedback signals (i.e., angle signals, inertial signals, and gripping signals); generates a second control signal according to the first movement intention and the second movement intention to control the movement of the prosthetic hand based on the second control signal, that is, this movement process is controlled based on feedforward signals and feedback signals. The above-mentioned technical solution, during the movement of the prosthetic hand, such as during the interaction between the prosthetic hand and objects in the environment, corrects the first control signal of the prosthetic hand obtained based on the feedforward signal by using feedback signals collected from the prosthetic hand. This solves the problem of decreased stability of prosthetic hand control caused by factors such as the influence of sweat, electrode displacement and muscle fatigue on electromyographic signals, and improves the stability of prosthetic hand control.

[0036] Figure 2 is a flowchart of another prosthetic hand control method provided in an embodiment of this application. This embodiment is described based on the above-mentioned multiple technical solutions. In this embodiment, optionally, identifying a first movement intention based on electromyographic signals and pressure signals includes: acquiring a first movement intention recognition model, inputting electromyographic signals and pressure signals into the first movement intention recognition model, and obtaining the first movement intention; and / or, identifying a second movement intention based on angle signals, inertial signals, and grasping signals may include: acquiring a second movement intention recognition model, and inputting angle signals, inertial signals, and grasping signals into the second movement intention recognition model to obtain the second movement intention. The explanations of terms that are the same as or corresponding to those in the above-mentioned multiple embodiments will not be repeated here.

[0037] Referring to Figure 2, the method of this embodiment may include the following steps:

[0038] S210. Acquire electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscles due to bulging.

[0039] S220. Obtain the first motor intention recognition model by inputting electromyographic signals and pressure signals into the first motor intention recognition model to obtain the first motor intention.

[0040] S230. Generate a first control signal based on the first motion intention, so as to control the movement of the prosthetic hand based on the first control signal.

[0041] S240. During the movement of the prosthetic hand, acquire the angle signal of each finger joint on the prosthetic hand, the inertia signal of the prosthetic hand, and the grip signal to reflect the gripping force applied by the prosthetic hand.

[0042] S250. Obtain the second motion intention recognition model, and input the angle signal, inertial signal and grip signal into the second motion intention recognition model to obtain the second motion intention.

[0043] In this embodiment of the application, optionally, the second motion intention recognition model and the first motion intention recognition model can be the same motion intention recognition model or different motion intention recognition models, which helps to improve the accuracy of motion intention recognition.

[0044] S260. Generate a second control signal based on the first motion intention and the second motion intention, so as to control the movement of the prosthetic hand based on the second control signal.

[0045] The technical solution of this application embodiment achieves accurate recognition of motion intent by applying a motion intent recognition model.

[0046] An alternative technical solution involves pre-training the first motion intent recognition model through the following steps:

[0047] Acquire sample electromyographic signals of muscles and sample pressure signals generated by muscle bulging, as well as motion intention labels corresponding to sample electromyographic signals and sample pressure signals, and use sample electromyographic signals, sample pressure signals and motion intention labels as a set of training samples.

[0048] The original motion intent recognition model is trained based on the obtained multiple sets of training samples to obtain the first motion intent recognition model.

[0049] In this technical solution, optionally, a corresponding first motion intention recognition model can be trained for different wearers, that is, personalized training of the first motion intention recognition model can be carried out, thereby further improving the accuracy of motion intention recognition.

[0050] Figure 3 is a flowchart of another prosthetic hand control method provided in an embodiment of this application. This embodiment is described based on the above-mentioned multiple technical solutions. In this embodiment, optionally, controlling the movement of the prosthetic hand based on a second control signal includes: sending the second control signal to a microcontroller, so that the microcontroller generates a pulse width modulation signal based on the received second control signal, and controls the movement of the prosthetic hand through the pulse width modulation signal. The explanations of terms that are the same as or corresponding to those in the above-mentioned multiple embodiments will not be repeated here.

[0051] Referring to Figure 3, the method of this embodiment may include the following steps:

[0052] S310. Acquire electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscles due to bulging, and identify the first movement intention based on the electromyographic signals and pressure signals.

[0053] S320. Generate a first control signal based on the first motion intention, so as to control the movement of the prosthetic hand based on the first control signal.

[0054] In this embodiment of the application, optionally, the first control signal can be sent to the microcontroller. In this way, the microcontroller can generate a pulse width modulation (PWM) signal based on the received first control signal, and control the movement of the prosthetic hand through the PWM signal.

[0055] S330. During the movement of the prosthetic hand, acquire the angle signal of each finger joint on the prosthetic hand, the inertial signal of the prosthetic hand, and the gripping signal to reflect the gripping force applied by the prosthetic hand, and identify the second movement intention based on the angle signal, the inertial signal, and the gripping signal.

[0056] S340. Generate a second control signal based on the first motion intention and the second motion intention.

[0057] S350. Send the second control signal to the microcontroller so that the microcontroller generates a pulse width modulation signal based on the received second control signal and controls the movement of the prosthetic hand through the pulse width modulation signal.

[0058] The second control signal is sent to the microcontroller. In this way, the microcontroller can generate a PWM signal based on the received second control signal. It should be noted that the PWM signal generated in this step is the PWM signal corresponding to the second control signal, not the PWM signal corresponding to the first control signal generated in S320. Then, the microcontroller can control the movement of the prosthetic hand through the PWM signal.

[0059] In the technical solution of this application embodiment, the microcontroller achieves effective control of the prosthetic hand through PWM signals.

[0060] In one alternative technical solution, the gripping signal is represented by a current signal describing the operating current of a servo motor located within the prosthetic hand, and the microcontroller further performs the following steps:

[0061] Acquire angle and current signals;

[0062] Based on the angle signal, the rate of change of the bending angle of the finger joint is obtained;

[0063] The pulse width modulation signal is maintained when the rate of change is less than a preset rate of change threshold and the current signal is within a preset current range.

[0064] The current signal can be understood as a signal describing the operating current of the servo motor installed in the prosthetic hand, and the gripping signal is represented by this current signal. The rate of change of the bending angle, ΔAngle, is obtained from the angle signal. If ΔAngle is less than a preset rate of change threshold, it indicates that the bending angle is in a stable state; and if the current signal is within a preset current range, it indicates that the operating current is in a stable state. When both the bending angle and the operating current are in a stable state, it means that the prosthetic hand has gripped the object. The microcontroller can then maintain the current PWM signal to avoid unnecessary gripping force adjustments and ensure stable gripping of the object.

[0065] Another alternative technical solution is that the grip signal can be represented by a current signal that describes the operating current of the servo motor set inside the prosthetic hand. The microcontroller also performs the following steps:

[0066] When the current signal decreases, increase the duty cycle of the pulse width modulation signal.

[0067] If the current signal decreases, meaning the operating current decreases, this usually occurs when the grip weakens due to the object slipping. In this case, increasing the duty cycle of the PWM signal can increase the grip to compensate for the slipping tendency, thus preventing the object from slipping and being damaged. This is an anti-slip mechanism.

[0068] Based on this, optionally, the microcontroller also performs the following steps:

[0069] If the current signal is less than the preset safe current limit and the difference between the current signal and the safe current limit is less than or equal to the preset difference threshold, the duty cycle will stop increasing.

[0070] The safe current limit can be understood as the upper limit of the working current that can avoid damage to the object and the prosthetic hand. After increasing the duty cycle to increase the current signal, if the current signal is less than the preset safe current limit, and the difference between the current signal and the safe current limit is less than or equal to the preset difference threshold, that is, if the current signal is close to the safe current limit, then the increase in duty cycle can be stopped, that is, the increase in current signal can be stopped, and thus the increase in gripping force can be stopped, thereby ensuring the safety of the prosthetic hand and the object.

[0071] Based on this, in order to understand the above technical solutions as a whole, the following examples will illustrate the process. For example, referring to Figures 4 and 5, the implementation process is as follows:

[0072] Step 1: Technicians locate several muscles with strong muscle activity on the forearm amputee's stump, wipe them with alcohol, and then attach a prosthetic hand equipped with a multi-channel electromyography (EMG) sensor to the stump, preparing for EMG signal acquisition. Optionally, in this example, the multi-channel EMG sensor may integrate a pressure sensor, enabling simultaneous acquisition of EMG and pressure signals.

[0073] Step 2: Open the serial port connection, ensure normal communication between the EMG sensor and the host computer software, calibrate the signal reception status, and begin real-time monitoring of EMG and pressure signals. After confirming stable signal transmission, technicians can begin collecting EMG and pressure signals and performing further processing before sending them to the host computer.

[0074] This example uses electromyography (EMG) signals; the same principle applies to pressure signals. In this example, the EMG signals can be processed by a signal processing unit, which can consist of a signal acquisition module (AD7606) and a microcontroller (STM32F767IGT6). The signal acquisition module can be configured to acquire EMG signals and then send them to the microcontroller via a Serial Peripheral Interface (SPI) for median-mean filtering. The processed EMG signal is then sent to a host computer via a Serial Communications Interface (SCI) for further processing. The host computer can display the EMG signals and perform preprocessing, such as extracting frequency domain features from the time-domain EMG signals using Fast Fourier Transform.

[0075] The following explanation uses electromyography (EMG) signals as an example. However, in practical applications, the same steps can be used to process pressure signals, as both EMG signals and pressure signals are needed for model training and motion intention recognition.

[0076] Step 3: The prosthetic hand performs a preset muscle activity. At this time, the electromyography (EMG) sensor collects EMG signals, and the technician maps this muscle activity to a motion intention via host computer software. In this example, the motion intention can be understood as the prosthetic hand's movement, that is, the combination of position data from multiple servos in the prosthetic hand.

[0077] Step 4: Operate on the host computer software to preprocess the electromyographic signals corresponding to the multiple collected movement intentions to obtain frequency domain features. Then, train the model based on these features and monitor the model's loss reduction in real time through the software's visual interface. If the model learns normally, it can be saved for subsequent control of the prosthetic hand based on real-time electromyographic signals. If the model training effect is not good, you can start from Step 3 again to have the residual end generate electromyographic signals with greater differences in muscle activity so that the model can obtain higher quality electromyographic signals for model training.

[0078] Step 5: Using the motion intention recognition model saved in Step 4, the current motion intention of the forearm amputee is identified based on real-time electromyography signals. Based on the recognition results, a control signal is generated and sent to the microcontroller. The microcontroller sends a PWM signal to control the rotation angle of the servo motor of the traction tendon rope, thereby driving the fingers of the prosthetic hand to perform flexion and extension movements, realizing various movements of the prosthetic hand.

[0079] Step 6: During the real-time interaction between the prosthetic hand and objects in the environment, feedback signals collected by the angle sensor, IMU, and current sensor are uploaded to the host computer via the microcontroller. The motion intent is then recognized by the motion intent recognition model, and the servo motor is fine-tuned accordingly to make the control of the prosthetic hand more stable. Optionally, in this example, if an abnormality occurs during the gripping process (such as applying excessive or insufficient gripping force), the microcontroller can immediately adjust the PWM signal based on the feedback signal to avoid damaging the prosthetic hand or the object, thereby ensuring the safety and accuracy of the prosthetic hand's operation.

[0080] The above example has at least the following advantages:

[0081] 1) By integrating angle sensors, IMUs, and current sensors into the prosthetic hand, without adding additional sensors at the residual end for measuring feedforward signals, hardware complexity is reduced, while control signal acquisition becomes simpler and more accurate.

[0082] 2) The angle sensor can provide real-time feedback on the bending angle of the finger joints of the prosthetic hand, enabling the host computer to perceive the posture of the prosthetic hand when actually grasping an object, and accurately determine the shape of the object and whether the grasping action is successful through this posture, thereby significantly improving the accuracy of the prosthetic hand operation.

[0083] 3) The current sensor can monitor the change in the working current of the servo motor when the prosthetic hand comes into contact with an object, and provide real-time feedback on the gripping force applied by the prosthetic hand through this change. This ensures that the host computer can dynamically adjust the movement of the prosthetic hand based on the gripping force feedback, thereby achieving more precise gripping force control and improving the stability and safety of gripping.

[0084] 4) By multimodal fusion of inertial signals, angle signals, and current signals (i.e., multiple feedback signals) with electromyographic signals and pressure signals (i.e., multiple feedforward signals), this example can provide more comprehensive feedback information to the host computer, improving the stability and responsiveness of the prosthetic hand in complex operating scenarios.

[0085] Figure 6 is a structural block diagram of the prosthetic hand control device provided in an embodiment of this application. This device is configured to execute the prosthetic hand control method provided in any of the above embodiments. This device and the prosthetic hand control methods of the above embodiments belong to the same inventive concept. Details not described in detail in the embodiments of the prosthetic hand control device can be referred to the embodiments of the above prosthetic hand control methods. Referring to Figure 6, the device may include: a first motion intention recognition module 410, a first prosthetic hand control module 420, a second motion intention recognition module 430, and a second prosthetic hand control module 440.

[0086] The first motion intention recognition module 410 is configured to acquire electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscles due to bulging, and to recognize the first motion intention based on the electromyographic signals and pressure signals.

[0087] The first prosthetic hand control module 420 is configured to generate a first control signal based on a first motion intention, so as to control the movement of the prosthetic hand based on the first control signal.

[0088] The second motion intention recognition module 430 is configured to acquire angle signals of each finger joint on the prosthetic hand, inertial signals of the prosthetic hand, and grasping signals reflecting the grasping force applied by the prosthetic hand during the movement of the prosthetic hand, and to recognize the second motion intention based on the angle signals, inertial signals, and grasping signals.

[0089] The second prosthetic hand control module 440 is configured to generate a second control signal based on the first movement intention and the second movement intention, so as to control the movement of the prosthetic hand based on the second control signal.

[0090] Optionally, the first motion intent recognition module 410 may include:

[0091] The first motor intention acquisition unit is configured to acquire a first motor intention recognition model, input electromyographic signals and pressure signals into the first motor intention recognition model to obtain the first motor intention; and / or,

[0092] The second motion intention recognition module 430 may include:

[0093] The second motion intent acquisition unit is configured to acquire the second motion intent recognition model and input the angle signal, inertial signal, and grip signal into the second motion intent recognition model to obtain the second motion intent.

[0094] Based on this, optionally, the first motion intention recognition model is pre-trained through the following modules:

[0095] The training sample acquisition module is set to acquire the sample electromyographic signals of the muscle and the sample pressure signals generated by the muscle bulge, as well as the motion intention labels corresponding to the sample electromyographic signals and sample pressure signals, and use the sample electromyographic signals, sample pressure signals and motion intention labels as a set of training samples.

[0096] The module for obtaining the first motion intention recognition model is configured to train the original motion intention recognition model based on the obtained multiple sets of training samples, thereby obtaining the first motion intention recognition model.

[0097] Optionally, the second prosthetic hand control module 440 may include:

[0098] The prosthetic hand control unit is configured to send a second control signal to a microcontroller, so that the microcontroller generates a pulse width modulation signal based on the received second control signal, and controls the movement of the prosthetic hand through the pulse width modulation signal.

[0099] Optionally, the gripping signal is represented by a current signal that describes the operating current of the servo motor located inside the prosthetic hand. The microcontroller also operates through the following modules:

[0100] The rate of change module is configured to acquire angle and current signals, and obtain the rate of change of the bending angle of the finger joint based on the angle signal.

[0101] The pulse width modulation signal sustaining module is configured to sustain the pulse width modulation signal when the rate of change is less than a preset rate of change threshold and the current signal is within a preset current range; and / or,

[0102] The duty cycle increase module is configured to increase the duty cycle of the pulse width modulation signal when the current signal decreases.

[0103] Based on this, optionally, when the duty cycle of the microcontroller is increased, the microcontroller also operates through the following modules:

[0104] The duty cycle stop increasing module is set to stop increasing the duty cycle when the current signal is less than the preset safe current limit and the difference between the current signal and the safe current limit is less than or equal to the preset difference threshold.

[0105] Based on any of the above-mentioned prosthetic hand control devices, optionally, an angle signal is used to provide feedback on the motion state of the prosthetic hand and / or the shape of the object grasped by the prosthetic hand; an inertial signal is used to provide feedback on at least one of the motion direction, motion speed, and motion posture of the prosthetic hand.

[0106] The prosthetic hand control device provided in this application embodiment acquires electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscle bulge through a first motion intention recognition module, and identifies a first motion intention based on the electromyographic signals and pressure signals, that is, the first motion intention is identified based on feedforward signals (i.e., electromyographic signals and pressure signals); the first prosthetic hand control module generates a first control signal based on the first motion intention, and controls the movement of the prosthetic hand based on the first control signal, that is, this movement process is controlled based on feedforward signals; the second motion intention recognition module acquires angle signals of each finger joint on the prosthetic hand, inertial signals of the prosthetic hand, and gripping signals reflecting the gripping force applied by the prosthetic hand during the movement of the prosthetic hand, and identifies a second motion intention based on the angle signals, inertial signals, and gripping signals, that is, the second motion intention is identified based on feedback signals (i.e., angle signals, inertial signals, and gripping signals); furthermore, the second prosthetic hand control module generates a second control signal according to the first motion intention and the second motion intention, and controls the movement of the prosthetic hand based on the second control signal, that is, this movement process is controlled based on feedforward signals and feedback signals. The aforementioned device, during the movement of the prosthetic hand, such as during the interaction between the prosthetic hand and objects in the environment, corrects the first control signal of the prosthetic hand obtained based on the feedforward signal by using feedback signals collected from the prosthetic hand. This solves the problem of decreased stability of prosthetic hand control caused by factors such as the influence of sweat on electromyographic signals, electrode displacement, and muscle fatigue, and improves the stability of prosthetic hand control.

[0107] The prosthetic hand control device provided in this application embodiment can execute the prosthetic hand control method provided in any embodiment of this application, and has the corresponding functional modules for executing the method.

[0108] It is worth noting that in the above embodiments of the prosthetic hand control device, the multiple units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the names of the multiple functional units are only for easy differentiation and are not used to limit the scope of protection of this application.

[0109] Figure 7 illustrates a schematic diagram of the structure of an electronic device 10 that can be used to implement embodiments of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0110] As shown in Figure 7, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0111] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0112] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs several of the methods and processes described above, such as prosthetic hand control methods.

[0113] In some embodiments, the prosthetic hand control method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the prosthetic hand control method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the prosthetic hand control method by any other suitable means (e.g., by means of firmware).

[0114] The various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), systems-on-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0115] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0116] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. Examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fiber, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0117] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be configured to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0118] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0119] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0120] It should be understood that the various processes shown above can be used to rearrange, add, or delete steps. For example, the multiple steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

Claims

1. A method for controlling a prosthetic hand, comprising: Acquire electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscles due to bulging, and identify a first movement intention based on the electromyographic signals and the pressure signals; A first control signal is generated based on the first motion intention, so as to control the movement of the prosthetic hand based on the first control signal; During the movement of the prosthetic hand, the angle signal of each finger joint on the prosthetic hand, the inertia signal of the prosthetic hand, and the gripping signal reflecting the gripping force applied by the prosthetic hand are acquired, and the second movement intention is identified based on the angle signal, the inertia signal, and the gripping signal. A second control signal is generated based on the first movement intention and the second movement intention, so as to control the movement of the prosthetic hand based on the second control signal.

2. The method according to claim 1, wherein, The process of identifying a first motor intention based on the electromyographic signal and the pressure signal includes: A first movement intention recognition model is obtained by inputting the electromyographic signal and the pressure signal into the first movement intention recognition model to obtain the first movement intention; or, The process of identifying the second movement intention based on the angle signal, the inertial signal, and the grip signal includes: Obtain a second motion intention recognition model, and input the angle signal, the inertial signal, and the gripping signal into the second motion intention recognition model to obtain the second motion intention; or, The step of identifying a first movement intention based on the electromyographic signal and the pressure signal includes: acquiring a first movement intention recognition model, inputting the electromyographic signal and the pressure signal into the first movement intention recognition model to obtain the first movement intention; and the step of identifying a second movement intention based on the angle signal, the inertial signal and the grip signal includes: acquiring a second movement intention recognition model, and inputting the angle signal, the inertial signal and the grip signal into the second movement intention recognition model to obtain the second movement intention.

3. The method according to claim 2, wherein, The first motion intent recognition model is pre-trained in the following manner: Acquire sample electromyographic signals of the muscle and sample pressure signals generated by the muscle due to bulging, as well as motion intention labels corresponding to the sample electromyographic signals and the sample pressure signals, and use the sample electromyographic signals, the sample pressure signals and the motion intention labels as a set of training samples; The original motion intention recognition model is trained based on the obtained training samples to obtain the first motion intention recognition model.

4. The method according to claim 1, wherein, The control of the prosthetic hand movement based on the second control signal includes: The second control signal is sent to the microcontroller, so that the microcontroller generates a pulse width modulation signal based on the received second control signal, and controls the movement of the prosthetic hand through the pulse width modulation signal.

5. The method according to claim 4, wherein, The gripping signal is represented by a current signal describing the operating current of a servo motor located within the prosthetic hand, and the microcontroller also performs: Acquire the angle signal and the current signal, and obtain the rate of change of the bending angle of the finger joint based on the angle signal; When the rate of change is less than a preset rate of change threshold and the current signal is within a preset current range, the pulse width modulation signal is maintained; or, When the current signal decreases, the duty cycle of the pulse width modulation signal is increased.

6. The method according to claim 5, wherein, When the microcontroller increases the duty cycle of the pulse width modulation signal, the microcontroller also performs the following: When the current signal is less than the preset safe current limit and the difference between the current signal and the safe current limit is less than or equal to the preset difference threshold, the duty cycle of the pulse width modulation signal is stopped from increasing.

7. The method according to any one of claims 1-6, wherein, The angle signal is used to provide feedback on at least one of the motion state of the prosthetic hand and the shape of the object grasped by the prosthetic hand; The inertial signal is used to provide feedback on at least one of the following: the direction of movement, the speed of movement, and the posture of movement of the prosthetic hand.

8. A prosthetic hand control device, comprising: The first motion intention recognition module is configured to acquire electromyographic signals of the muscles on the wearing end of the prosthetic hand and pressure signals generated by the muscles due to bulging, and to recognize the first motion intention based on the electromyographic signals and the pressure signals. The first prosthetic hand control module is configured to generate a first control signal based on the first movement intention, so as to control the movement of the prosthetic hand based on the first control signal; The second motion intention recognition module is configured to acquire angle signals of each finger joint on the prosthetic hand, inertial signals of the prosthetic hand, and grasping signals reflecting the grasping force applied by the prosthetic hand during the movement of the prosthetic hand, and to identify the second motion intention based on the angle signals, the inertial signals, and the grasping signals. The second prosthetic hand control module is configured to generate a second control signal based on the first movement intention and the second movement intention, so as to control the movement of the prosthetic hand based on the second control signal.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to cause the at least one processor to perform the prosthetic hand control method as described in any one of claims 1-7.

10. A computer-readable storage medium storing computer instructions, said computer instructions being configured to cause a processor to execute and implement the prosthetic hand control method as described in any one of claims 1-7.

Images