Bionic hand and switching method of playing mode thereof, control device

Abstract

The application discloses a switching method of a bionic hand playing mode, the bionic hand comprises a plurality of fingers, and the switching method of the bionic hand playing mode comprises the following steps: when the bionic hand is in a semi-automatic playing mode, controlling the corresponding fingers to play according to a musical score and an electromyographic signal on a residual arm surface; when it is detected that notes played by the bionic hand in the semi-automatic playing mode meet preset requirements, controlling the bionic hand to switch from the semi-automatic playing mode to a full-automatic playing mode; and controlling the fingers to play according to the musical score in the full-automatic playing mode. The switching method of the bionic hand playing mode can solve the problem of slow response speed when the bionic hand plays the piano. In addition, the application further discloses a control device of the bionic hand and the bionic hand.

Description

Technical Field

[0001] This invention relates to the field of bionic hand technology, and in particular to a bionic hand and a method and control device for switching its playing modes. Background Technology

[0002] In existing bionic hands, each finger is controlled by an independent drive motor to flex and extend, and the bending angle of the finger is determined by electromyographic signals that control flexion and extension.

[0003] However, in certain specific application scenarios, such as playing the piano, the bionic hand cannot quickly respond to all the wearer's electromyographic signals to control all the fingers to move quickly. It can only respond to the electromyographic signals of a single finger in turn, simply controlling a single finger to press the piano key. This makes playing the piano very strenuous, with low accuracy and slow rhythm. As a result, it is difficult for the wearer to continue using the bionic hand for piano training, resulting in a poor user experience. Summary of the Invention

[0004] The main objective of this invention is to propose a bionic hand and a method and control device for switching its playing modes, aiming to solve the problem of slow response speed when the bionic hand plays the piano.

[0005] To achieve the above objectives, this invention proposes a method for switching the playing mode of a bionic hand, wherein the bionic hand includes several fingers, and the method for switching the playing mode of the bionic hand includes:

[0006] When the bionic hand is in semi-automatic playing mode, it controls the corresponding fingers to play according to the musical score and the electromyographic signals on the surface of the residual arm.

[0007] When it is detected that the notes played by the bionic hand in the semi-automatic playing mode meet the preset requirements, the bionic hand is controlled to switch from the semi-automatic playing mode to the fully automatic playing mode; and

[0008] In the fully automatic playing mode, the fingers are controlled to play according to the musical score.

[0009] Preferably, the step of controlling the fingers to play according to the sheet music in the fully automatic playing mode includes:

[0010] Based on the musical score, corresponding preset playing information is obtained, including bending angle, bending speed, and bending force; and

[0011] Control the fingers to play according to the musical score and the corresponding preset playing information.

[0012] Preferably, the method for switching the bionic hand playing mode further includes:

[0013] When the musical score is recognized based on the notes played, the bionic hand is controlled to enter the fully automatic playing mode based on the received trigger information, which includes electromyographic data, motion data and / or control commands corresponding to specific actions.

[0014] Preferably, before controlling the bionic hand to switch from the semi-automatic playing mode to the fully automatic playing mode, the method for switching the playing mode of the bionic hand further includes:

[0015] When it is detected that a series of consecutive notes played by the bionic hand in the semi-automatic playing mode match the musical score, it is confirmed that the notes played by the bionic hand in the semi-automatic playing mode meet the preset requirements.

[0016] Preferably, after controlling the fingers to play according to the musical score in the fully automatic playing mode, the method for switching the bionic hand playing mode further includes:

[0017] When it is detected that the notes played by the bionic hand in the fully automatic playing mode do not meet the preset requirements, the bionic hand is controlled to switch from the fully automatic playing mode to the semi-automatic playing mode.

[0018] Preferably, after controlling the bionic hand to switch from the fully automatic playing mode to the semi-automatic playing mode, the method for switching the playing mode of the bionic hand further includes:

[0019] When it is detected that the notes played by the bionic hand in the semi-automatic playing mode do not meet the preset requirements, the bionic hand is controlled to switch from the semi-automatic playing mode to the normal playing mode; and

[0020] In the normal playing mode, the fingers are controlled to play according to the electromyographic signals.

[0021] Preferably, after controlling the bionic hand to switch from the semi-automatic playing mode to the fully automatic playing mode, the method for switching the playing mode of the bionic hand further includes:

[0022] Send a notification message.

[0023] The present invention further proposes a control device for a bionic hand, the bionic hand comprising a plurality of fingers, the control device for the bionic hand comprising:

[0024] The first control module is used to control the corresponding fingers to play music based on the sheet music and electromyographic signals on the surface of the residual arm when the bionic hand is in semi-automatic playing mode.

[0025] A switching module is configured to control the bionic hand to switch from the semi-automatic playing mode to the fully automatic playing mode when it detects that the notes played by the bionic hand in the semi-automatic playing mode meet preset requirements; and

[0026] The second control module is used to control the fingers to play according to the sheet music in the fully automatic playing mode.

[0027] Preferably, the second control module includes:

[0028] The acquisition module is used to acquire corresponding preset playing information based on the musical score, the preset playing information including bending angle, bending speed, and bending force; and

[0029] The sub-control module is used to control the fingers to play according to the musical score and the corresponding preset playing information.

[0030] The present invention further proposes a bionic hand, which includes a bionic hand body and a control device for the bionic hand as described above. The bionic hand body includes a plurality of fingers, and the control device is disposed on the bionic hand body for driving the fingers to play.

[0031] The technical solution of this invention is as follows: In semi-automatic playing mode, the fingers are controlled to play according to the sheet music and electromyographic signals. The played notes are detected in semi-automatic mode, and when the detected notes meet preset requirements, the bionic hand is controlled to switch from semi-automatic playing mode to fully automatic playing mode. In fully automatic playing mode, the bionic hand can directly control the fingers to play the corresponding notes according to the sheet music, greatly reducing the wearer's control burden and making instrument playing easy and simple, with high accuracy and smooth rhythm. Attached Figure Description

[0032] Figure 1 A flowchart of the method for switching the bionic hand playing mode provided in the first embodiment of the present invention;

[0033] Figure 2 A sub-flowchart of the method for switching the bionic hand playing mode provided in an embodiment of the present invention;

[0034] Figure 3 A flowchart of a method for switching bionic hand playing modes provided in the second embodiment of the present invention;

[0035] Figure 4 A flowchart of a method for switching bionic hand playing modes provided in the third embodiment of the present invention;

[0036] Figure 5 A flowchart of the method for switching the bionic hand playing mode provided in the fourth embodiment of the present invention;

[0037] Figure 6 A schematic diagram of a bionic hand provided in an embodiment of the present invention;

[0038] Figure 7This is a schematic diagram of the structure of the control device provided in an embodiment of the present invention;

[0039] Figure 8 This is a schematic diagram of the control device for the bionic hand provided in the first embodiment of the present invention;

[0040] Figure 9 for Figure 8 A schematic diagram of the second control module of the control device for the bionic hand shown.

[0041] Figure 10 This is a schematic diagram of the control device for the bionic hand provided in the second embodiment of the present invention;

[0042] Figure 11 This is a schematic diagram of the control device for the bionic hand provided in the third embodiment of the present invention;

[0043] Figure 12 This is a schematic diagram of a bionic hand module provided in an embodiment of the present invention.

[0044] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0045] The solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0047] It should also be noted that when a component is described as "fixed to" or "set on" another component, it can be directly on the other component or there may be an intervening component present. When a component is described as "connected to" another component, it can be directly connected to the other component or there may be an intervening component present.

[0048] Furthermore, the use of terms such as "first" and "second" in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed by this invention.

[0049] Please refer to the following: Figure 1 and Figure 6 , Figure 1 The flowchart shows the method for switching the bionic hand playing mode according to the first embodiment of the present invention. Figure 6 This is a schematic diagram of a bionic hand provided in an embodiment of the present invention. The bionic hand playing mode switching method is applied to a bionic hand 1, which includes several fingers 21. The bionic hand playing mode switching method is used to control the fingers 21 of the bionic hand 1 to play musical instruments. The musical instruments include, but are not limited to, keyboard instruments such as pianos, electronic keyboards, and accordions.

[0050] In this embodiment, the bionic hand 1 is provided with a receiving cavity 30. The bionic hand 1 is fixed to the wearer's arm (residual limb) through the receiving cavity 30, and the cavity wall of the receiving cavity 30 is in close contact with the wearer's arm muscles. The bionic hand 1 has multiple electromyographic electrodes (not shown) on the cavity wall of the receiving cavity 30. The electromyographic electrodes are used to collect the action potential generated by the muscles and form electromyographic signals on the surface of the residual limb.

[0051] The bionic hand 1 also includes a control device 10 for performing a method of switching the bionic hand playing modes, the control device 10 being electrically connected to electromyographic electrodes. The relevant functions of the control device 10 can be implemented by a single device, by multiple devices working together, or by one or more functional modules within a single device; no specific limitation is made here. It is understood that the aforementioned functions can be network elements within hardware devices, software functions running on dedicated hardware, a combination of hardware and software, or virtualized functions instantiated on a platform (e.g., a cloud platform).

[0052] The specific steps for switching the bionic hand playing mode are as follows.

[0053] Step S102: When the bionic hand is in semi-automatic playing mode, the corresponding fingers are controlled to play according to the musical score and the electromyographic signals on the surface of the residual arm.

[0054] In this embodiment, when the bionic hand 1 is in semi-automatic playing mode, the control device 10 controls the corresponding fingers 21 to play according to the sheet music and the electromyographic signals on the surface of the residual arm. Specifically, during the process of the wearer controlling the bionic hand 1 to play a musical instrument, the wearer generates a motor intention through the brain, which excites the peripheral nervous system and controls the contraction of the forearm muscle group. The electromyographic electrodes on the surface of the residual limb receive stimulation and generate corresponding electromyographic signals; the control device 10 receives the electromyographic signals sent by the electromyographic electrodes and controls the fingers 21 to play in combination with the sheet music. After the bionic hand 1 enters the semi-automatic playing mode, the control device 10 can obtain the corresponding playing speed according to the interval of the notes in the analyzed sheet music, and controls the fingers 21 to play in combination with the playing speed and electromyographic signals. Among them, the control device 10 can control the bending speed of the fingers 21 according to the playing speed.

[0055] The specific process of controlling the corresponding fingers to play music based on the sheet music and electromyographic signals on the surface of the residual arm will be described in detail below.

[0056] In some embodiments, before controlling the corresponding fingers to play according to the musical score and the electromyographic signals on the surface of the stump, the method for switching the playing mode of the bionic hand further includes: when the musical score is recognized based on the notes played, controlling the bionic hand to enter a semi-automatic playing mode.

[0057] In this embodiment, before the bionic hand 1 enters the semi-automatic playing mode, it is in the normal playing mode, and the control device 10 can only control the fingers 21 to play based on electromyographic signals. Specifically, the control device 10 receives electromyographic signals sent by the electromyographic electrodes, controls the corresponding fingers 21 to bend according to the electromyographic signals, and controls the bending angle of the fingers 21 according to the duration of the electromyographic signals, so that the fingers 21 press the piano keys to play the instrument. The bending angle can represent the angle formed between the distal phalanx (corresponding to the distal phalanx) of the finger 21 and the palm plane when the finger 21 bends; the larger the angle, the greater the degree of bending of the finger 21. In some feasible embodiments, the bending angle can also represent the angle formed between other parts of the finger 21 (corresponding to the middle or proximal phalanx) and the palm plane when the finger 21 bends.

[0058] The control device 10 is equipped with a sound recognition system for detecting the notes being played. During normal playing mode, the control device 10 performs sound recognition detection on the played notes, matching them with scores in a preset music library to obtain a matching score. In other words, the control device 10 identifies the score the wearer is currently playing based on the played notes. After identifying the score based on the played notes, the control device 10 controls the bionic hand 1 to enter semi-automatic playing mode.

[0059] For example, when the wearer controls the bionic hand 1 to play a musical piece, the control device 10 controls the fingers 21 to play the notes 123455543 based on the received electromyographic signals and corresponding durations. During the playing, the control device 10 detects the played notes. When the fingers 21 play the second 3, the control device 10 recognizes that the music being played is "The Doll and the Bear Dancing," and then controls the bionic hand 1 to enter a semi-automatic playing mode. After the control device 10 controls the bionic hand 1 to enter the semi-automatic playing mode, it controls the corresponding fingers 21 to play the following notes of "The Doll and the Bear Dancing" based on the music score and electromyographic signals: 44432135...

[0060] Step S104: When it is detected that the notes played by the bionic hand in the semi-automatic playing mode meet the preset requirements, control the bionic hand to switch from the semi-automatic playing mode to the fully automatic playing mode.

[0061] In this embodiment, after the bionic hand 1 enters the semi-automatic playing mode, the control device 10, while controlling the fingers 21 to play according to the musical score and electromyographic signals, also needs to determine whether the notes played in the semi-automatic playing mode meet the preset requirements. Specifically, the control device 10 detects the played notes according to the identified musical score. When the notes played in the semi-automatic playing mode meet the preset requirements, it means that the control device 10 can combine the musical score and electromyographic signals to control the fingers 21 to play the corresponding music well. Then, the control device 10 controls the bionic hand 1 to switch from the semi-automatic playing mode to the fully automatic playing mode.

[0062] When the notes played in the semi-automatic playing mode do not meet the preset requirements, it means that the music played by the control device 10 in the process of controlling the finger 21 to play according to the electromyographic signal and the musical score is not in line with the musical score. Then the control device 10 continues to control the finger 21 to play according to the musical score and electromyographic signal in the semi-automatic playing mode.

[0063] Understandably, as long as it is in semi-automatic playing mode, the control device 10 will detect the played notes in real time. When it detects that a series of consecutive notes meet the preset requirements, the control device 10 will switch the playing mode of the bionic hand 1 from semi-automatic playing mode to fully automatic playing mode.

[0064] In some embodiments, before controlling the bionic hand to switch from a semi-automatic playing mode to a fully automatic playing mode, the method for switching the bionic hand playing mode further includes: when it is detected that a number of consecutive notes played by the bionic hand in the semi-automatic playing mode match the musical score, confirming that the notes played by the bionic hand in the semi-automatic playing mode meet preset requirements.

[0065] In this embodiment, the control device 10 detects several consecutive musical notes and determines whether the consecutive musical notes match the musical score. Specifically, the control device 10 can determine whether the pitch, playing speed, etc., of the several consecutive musical notes match the musical score. When the pitch, playing speed, etc., of the several consecutive musical notes match the musical score, the control device 10 confirms that the notes played by the bionic hand 1 in the semi-automatic playing mode meet the preset requirements; when the pitch or playing speed of the several consecutive musical notes does not match the musical score, the control device 10 confirms that the notes played by the bionic hand 1 in the semi-automatic playing mode do not meet the preset requirements.

[0066] In some embodiments, after controlling the bionic hand to switch from a semi-automatic playing mode to a fully automatic playing mode, the method for switching the playing mode of the bionic hand further includes: sending a prompt message.

[0067] In this embodiment, the control device 10 can send a prompt message to remind the wearer. The prompt message may include, but is not limited to, voice, light, vibration, or other forms.

[0068] Step S106: In fully automatic playing mode, control your fingers to play according to the sheet music.

[0069] In this embodiment, in fully automatic playing mode, the control device 10 controls the fingers 21 to play according to the sheet music. Specifically, after entering fully automatic playing mode, the control device 10 no longer needs electromyographic signals to directly control the fingers 21 to play according to the sheet music. Even in fully automatic playing mode, the control device 10 still receives electromyographic signals, but it can block these signals and not execute the corresponding control operations until the bionic hand 1 exits fully automatic playing mode or the sheet music is played. It is understood that in fully automatic playing mode, the wearer does not need to provide electromyographic signals, and the bionic hand 1 can autonomously and independently complete the playing of the sheet music.

[0070] The specific process of controlling your fingers to play according to the sheet music in fully automatic playing mode will be described in detail below.

[0071] In this embodiment, after recognizing the sheet music being played by the bionic hand based on the notes played in normal playing mode, the bionic hand is controlled to enter a semi-automatic playing mode. In this mode, the fingers are controlled to play based on the recognized sheet music and electromyographic signals. Controlling the bionic hand into semi-automatic playing mode improves its response speed when playing instruments, allowing it to produce rhythms that match the sheet music, while also reducing the wearer's control burden and improving their experience. In semi-automatic playing mode, the played notes are detected. If a note does not meet preset requirements, the bionic hand remains in semi-automatic mode; if a note meets the preset requirements, the bionic hand switches from semi-automatic to fully automatic playing mode. In fully automatic playing mode, the bionic hand can directly control its fingers to play the corresponding notes based on the sheet music and preset playing information, greatly reducing the wearer's control burden and making instrument playing easy, simple, accurate, and rhythmically smooth. At the same time, a prompt message is sent when switching playing modes to remind the wearer, giving the wearer ample preparation time to change playing strategies.

[0072] Please refer to the following: Figure 2 This is a sub-flowchart of the bionic hand playing mode switching method provided in the embodiment of the present invention. Step S106 specifically includes the following steps.

[0073] Step S202: Obtain the corresponding preset playing information based on the musical score.

[0074] In this embodiment, the control device 10 can obtain corresponding preset playing information based on the identified musical score. Specifically, the control device 10 pre-stores preset playing information corresponding to each musical score, with one preset playing information per score. The preset playing information may include bending angle, bending speed, and bending force, etc.

[0075] Step S204: Control the fingers to play according to the musical score and the corresponding preset playing information.

[0076] In this embodiment, the control device 10 controls the fingers 21 to play according to the musical score and the preset playing information corresponding to the score. It should be noted that each note in the musical score can correspond to a bending angle, bending speed, and bending force. Specifically, the control device 10 controls the fingers 21 to press the keys according to the playing order of the musical score, based on the bending angle, bending speed, and bending force corresponding to the notes, thereby playing music that matches the volume of the note and the rhythm of the musical score.

[0077] For example, if the bionic hand 1 enters the semi-automatic playing mode, the control device 10 controls the corresponding finger 21 to play the following notes of "The Doll and the Bear Dancing" according to the electromyographic signals: 44432135. When finger 21 plays 5, the control device 10 detects that the played note meets the preset requirements, and then the control device 10 controls the bionic hand 1 to enter the fully automatic playing mode. The control device 10 controls the corresponding finger 21 to play the following notes according to the sheet music of "The Doll and the Bear Dancing" and the corresponding preset playing information: 123455543...

[0078] For example, if the bionic hand 1 enters the semi-automatic playing mode, the control device 10 controls the corresponding finger 21 to play the following notes of "The Doll and the Bear Dancing" according to the electromyographic signals: 44432136. When finger 21 plays 6, the control device 10 detects that the played note does not meet the preset requirements, so the control device 10 continues to control finger 21 to play the following notes of "The Doll and the Bear Dancing" according to the musical score and electromyographic signals: 123455543...

[0079] Accordingly, in the semi-automatic playing mode, controlling the corresponding fingers to play according to the sheet music and electromyographic signals on the surface of the residual arm includes: obtaining the corresponding preset playing information according to the sheet music, and controlling the corresponding fingers to play according to the preset playing information based on the electromyographic signals.

[0080] In this embodiment, the control device 10 controls the corresponding finger 21 to bend according to the received electromyographic signals, and at the same time, the control device 10 controls the finger 21 to press the piano keys according to the corresponding preset playing information. Specifically, the control device 10 controls the movement of the finger 21 according to the bending angle, bending speed and bending force corresponding to the notes, based on the received electromyographic signals and the playing order of the musical score, thereby playing music that matches the volume of the note and the rhythm of the musical score.

[0081] Understandably, in semi-automatic playing mode, as long as the control device 10 receives an electromyographic signal, it can control the corresponding finger 21 to bend according to the corresponding bending angle, bending speed, and bending force, thereby playing the corresponding musical score. After receiving the electromyographic signal, the control device 10 does not need to detect the duration of the electromyographic signal; it can directly control the corresponding finger 21 based on the electromyographic signal.

[0082] In this embodiment, after the bionic hand enters the fully automatic playing mode, it obtains the corresponding preset playing information based on the identified musical score. Electromyography signals are no longer needed. By controlling the corresponding fingers according to the musical score and following the preset playing information, music corresponding to the musical score can be played. This makes it easier for the wearer to control the bionic hand to play musical instruments, while also effectively improving the accuracy of playing and greatly enhancing the user experience.

[0083] Please refer to the following: Figure 3 This is a flowchart of the method for switching the bionic hand playing mode according to the second embodiment of the present invention. After executing step S106, the method for switching the bionic hand playing mode further includes the following steps.

[0084] Step S108: When it is detected that the notes played by the bionic hand in the fully automatic playing mode do not meet the preset requirements, the bionic hand is controlled to switch from the fully automatic playing mode to the semi-automatic playing mode.

[0085] In this embodiment, after the bionic hand 1 enters the fully automatic playing mode, the control device 10, while controlling the fingers 21 to play according to the musical score, also needs to determine whether the notes played in the fully automatic playing mode meet the preset requirements. The specific process of determining whether the notes played in the fully automatic playing mode meet the preset requirements is basically the same as the specific process of determining whether the notes played in the semi-automatic playing mode meet the preset requirements, and will not be described in detail here.

[0086] When the notes played in the fully automatic playing mode do not meet the preset requirements, it means that the music played by the control device 10 in the process of controlling the fingers 21 to play according to the score does not correspond to the score. Then the control device 10 controls the bionic hand 1 to switch from the fully automatic playing mode to the semi-automatic playing mode, and controls the fingers 21 to play according to the score and electromyographic signals.

[0087] When the notes played in the fully automatic playing mode meet the preset requirements, it means that the control device 10 can control the fingers 21 to play the corresponding music well according to the score. Then the control device 10 continues to play in the fully automatic playing mode, controlling the fingers 21 to play according to the score.

[0088] In this embodiment, as long as the fully automatic playing mode is in operation, the control device 10 can detect the played notes in real time or at preset intervals. When several consecutive notes are detected as not meeting the preset requirements, the control device 10 switches the playing mode of the bionic hand 1 from the fully automatic playing mode to the semi-automatic playing mode. The preset time can be set according to the actual control situation and is not limited here.

[0089] In some embodiments, when the bionic hand 1 switches from a fully automatic playing mode to a semi-automatic playing mode, the control device 10 can send a prompt message to remind the wearer. This prompt message may include, but is not limited to, voice, light, or vibration. It is understood that the prompt message sent by the control device 10 when the bionic hand 1 switches from a fully automatic playing mode to a semi-automatic playing mode is different from the prompt message sent by the control device 10 when the bionic hand 1 switches from a semi-automatic playing mode to a fully automatic playing mode.

[0090] In this embodiment, the notes played are detected in fully automatic playing mode. When a note is detected that does not meet the preset requirements, the bionic hand is controlled to return from fully automatic playing mode to semi-automatic playing mode. When a note is detected that meets the preset requirements, the bionic hand is controlled to continue in fully automatic playing mode. This allows for timely switching of the bionic hand's playing mode when problems occur, enabling the wearer to better control the bionic hand and effectively improve the accuracy of playing.

[0091] Please refer to the following: Figure 4 This is a flowchart of the method for switching the bionic hand playing mode according to the third embodiment of the present invention. After executing step S108, the method for switching the bionic hand playing mode further includes the following steps.

[0092] Step S110: When it is detected that the notes played by the bionic hand in the semi-automatic playing mode do not meet the preset requirements, the bionic hand is controlled to switch from the semi-automatic playing mode to the normal playing mode.

[0093] In this embodiment, after the bionic hand 1 returns from the fully automatic playing mode to the semi-automatic playing mode, the control device 10, while controlling the fingers 21 to play according to the musical score and electromyographic signals, also needs to determine whether the played notes meet the preset requirements. The specific process of determining whether the played notes meet the preset requirements in the semi-automatic playing mode is detailed above and will not be repeated here.

[0094] If the notes played in the semi-automatic playing mode do not meet the preset requirements, it means that after the control device 10 returns from the fully automatic playing mode to the semi-automatic playing mode, during the process of controlling the finger 21 to play according to the electromyographic signal and the preset playing information, the notes played still do not correspond to the musical score. In this case, the control device 10 controls the bionic hand 1 to switch from the semi-automatic playing mode to the normal playing mode.

[0095] When the notes played in the semi-automatic playing mode meet the preset requirements, it means that after the control device 10 returns from the fully automatic playing mode to the semi-automatic playing mode, in the process of controlling the finger 21 to play according to the electromyographic signal and the preset playing information, the control device 10 can combine the score and the electromyographic signal to control the finger 21 to play the corresponding music well. Then the control device 10 continues to control the finger 21 to play according to the score and the electromyographic signal in the semi-automatic playing mode.

[0096] In some embodiments, when the bionic hand 1 switches from a semi-automatic playing mode to a normal playing mode, the control device 10 can send a prompt message to remind the wearer. This prompt message may include, but is not limited to, voice, light, or vibration. It is understood that the prompt message sent by the control device 10 when the bionic hand 1 switches from a semi-automatic playing mode to a normal playing mode is different from the prompt message sent by the control device 10 when the bionic hand 1 switches from a fully automatic playing mode to a semi-automatic playing mode. That is, the prompt messages sent by the control device 10 are different when switching between different playing modes, so that the wearer can easily distinguish between them.

[0097] It should be noted that during the semi-automatic playing mode, if the control device 10 detects that the notes played within a certain period of time all meet the preset requirements, the control device 10 can switch the playing mode of the bionic hand 1 from the semi-automatic playing mode to the fully automatic playing mode. Correspondingly, during the normal playing mode, if the control device 10 detects that the notes played within a certain period of time all meet the preset requirements, the control device 10 can switch the playing mode of the bionic hand 1 from the normal playing mode to the semi-automatic playing mode.

[0098] Step S112: In normal playing mode, control the fingers to play according to electromyographic signals.

[0099] In this embodiment, in normal playing mode, the control device 10 can only control the finger 21 to play based on the electromyographic signal and the duration corresponding to the electromyographic signal. Specifically, the control device 10 receives the electromyographic signal sent by the electromyographic electrode, controls the corresponding finger 21 to bend according to the electromyographic signal, and controls the bending angle of the finger 21 according to the duration of the electromyographic signal, so that the finger 21 presses the piano key to play the instrument.

[0100] In this embodiment, when the fully automatic playing mode is switched back to the semi-automatic playing mode, the played notes are detected. When the detected notes do not meet the preset requirements, the bionic hand is controlled to switch from the semi-automatic playing mode to the normal playing mode. When the detected notes meet the preset requirements, the bionic hand is controlled to continue in the semi-automatic playing mode. This allows for timely switching of the bionic hand's playing mode when problems occur, enabling the wearer to better control the bionic hand and effectively improving the user experience.

[0101] The bionic hand 1 is pre-set with multiple playing modes, including normal playing mode, semi-automatic playing mode, and fully automatic playing mode. The control device 10 switches between playing modes based on the detected notes. In normal playing mode, the control device 10 controls the finger 21 to play based on electromyographic signals and their duration. In semi-automatic playing mode, the control device 10 no longer needs to detect the duration of the electromyographic signals; it only needs to control the finger 21 to play based on the electromyographic signals and the corresponding preset playing information. In fully automatic playing mode, the control device 10 can directly control the finger 21 to play based on the sheet music and preset playing information without requiring any electromyographic signals. The specific conditions for switching between playing modes can be set according to the actual playing situation or the wearer's personal preference, and are not limited here.

[0102] Please refer to the following: Figure 5 This is a flowchart of a method for switching the playing mode of a bionic hand provided in the fourth embodiment of the present invention. The bionic hand control method also includes the following steps.

[0103] In step S101, when the musical score is recognized based on the notes played, the bionic hand is controlled to enter the fully automatic playing mode based on the received trigger information.

[0104] In this embodiment, the initial playing mode of the bionic hand 1 is the normal playing mode. While the fingers 21 are playing according to electromyographic signals, the control device 10 performs sound recognition and detection on the played notes. When the control device 10 identifies the sheet music being played by the wearer based on the played notes, it determines whether it has received a trigger message. If the control device 10 receives a trigger message, it identifies the trigger message to control the bionic hand 1 to enter the fully automatic playing mode, that is, to switch from the normal playing mode to the fully automatic playing mode.

[0105] Trigger information includes electromyographic (EMG) data, motion data, and / or control commands corresponding to specific actions. Trigger information can be any one of EMG data, motion data, and control commands, or a combination of any two or three of these. Specific actions can be specific hand gestures, such as all fingers 21 of the bionic hand 1 rapidly making two consecutive clenching and opening fist movements, a single specific finger 21 of the bionic hand 1 repeatedly making rapid bending and opening movements, or wrist rotation movements of the bionic hand 1. The wearer can generate EMG data as trigger information by performing specific hand gestures; the wearer can generate motion data as trigger information by performing specific hand gestures; and the wearer can also send control commands via external devices as trigger information.

[0106] Specifically, when the wearer makes a specific hand gesture, the electromyographic electrodes collect the electromyographic signals and aggregate them into electromyographic data, which is then sent to the control device 10 as trigger information. The bionic hand 1 also includes an inertial measurement unit (IMU) that is communicatively connected to the control device 10. When the wearer makes a specific hand gesture, the inertial sensor collects the data and aggregates it into motion data, which is then sent to the control device 10 as trigger information. The control device 10 is also communicatively connected to an external device. When the wearer generates a control command through the external device, the external device sends the control command as trigger information to the control device 10.

[0107] In some embodiments, the control device 10 can control the bionic hand 1 to enter a fully automatic playing mode when it detects that the trigger information matches a specific action. Specifically, the control device 10 matches the received trigger information with a pre-set specific action. When the trigger information matches the specific action, the control device 10 controls the bionic hand 1 to enter the fully automatic playing mode. It is understood that the control device 10 only needs to match the trigger information with a specific action when the trigger information is electromyographic data or motion data; when the trigger information is a control command, the control device 10 directly controls the bionic hand 1 to enter the fully automatic playing mode according to the control command. That is to say, the control command can directly control the bionic hand 1 to enter the fully automatic playing mode. The specific action can be different from the bionic hand 1's daily routine actions to avoid the wearer accidentally triggering the fully automatic playing mode during daily activities.

[0108] In this embodiment, after identifying the sheet music being played by the bionic hand based on the notes played in the normal playing mode, the bionic hand can be controlled to directly enter the fully automatic playing mode based on the received trigger information, so as to quickly enter the fully automatic playing mode in specific situations and assist the wearer in playing the instrument better.

[0109] Please refer to the following: Figure 7 This is a schematic diagram of the structure of the control device provided in an embodiment of the present invention. The present invention also proposes a control device 10, which can be a computing device such as a desktop computer, laptop, handheld computer, or server. The control device 10 may include: a processor 1001 (e.g., CPU), a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize communication between these components. The user interface 1003 may include a display screen and an input unit, such as a keyboard; the user interface 1003 may also include standard wired interfaces and wireless interfaces. The network interface 1004 may include standard wired interfaces and wireless interfaces (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or stable non-volatile memory, such as disk storage; the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0110] Those skilled in the art will understand that Figure 7 The structure of the control device 10 shown does not constitute a limitation on the control device 10. The control device 10 may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0111] like Figure 7 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and computer-executable instructions.

[0112] exist Figure 7 In the control device 10 shown, the network interface 1004 is mainly used to connect to the backend server and communicate with the backend server; the user interface 1003 is mainly used to connect to the client (user end) and communicate with the client; and the processor 1001 can be used to call the computer-executable instructions stored in the memory 1005. When the computer-executable instructions are called and executed by the processor 1001, the steps of the above-mentioned method for switching the bionic hand playing mode are implemented.

[0113] Based on the computer-executable instructions proposed in the foregoing embodiments, the present invention also proposes a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are executed by a processor, the method for switching the bionic hand playing mode described in the foregoing embodiments is implemented.

[0114] Please refer to the following: Figure 8This is a schematic diagram of the control device for the bionic hand provided in the first embodiment of the present invention. The control device 40 for the bionic hand includes a first control module 41, a switching module 42, and a second control module 43.

[0115] The first control module 41 is used to control the corresponding fingers to play music based on the sheet music and electromyographic signals on the surface of the residual arm when the bionic hand is in semi-automatic playing mode.

[0116] In this embodiment, when the bionic hand 1 is in semi-automatic playing mode, the first control module 41 controls the corresponding fingers 21 to play according to the sheet music and the electromyographic signals on the surface of the residual arm. Specifically, during the process of the wearer controlling the bionic hand 1 to play a musical instrument, the wearer generates a motor intention through the brain, which excites the peripheral nervous system and controls the contraction of the forearm muscle group. The electromyographic electrodes on the surface of the residual limb receive stimulation and generate corresponding electromyographic signals; the control device 40 receives the electromyographic signals sent by the electromyographic electrodes and controls the fingers 21 to play in combination with the sheet music. After the bionic hand 1 enters the semi-automatic playing mode, the first control module 41 can obtain the corresponding playing speed according to the interval of the notes in the analyzed sheet music, and controls the fingers 21 to play in combination with the playing speed and electromyographic signals. Among them, the first control module 41 can control the bending speed of the fingers 21 according to the playing speed.

[0117] The switching module 42 is used to control the bionic hand to switch from the semi-automatic playing mode to the fully automatic playing mode when it detects that the notes played by the bionic hand in the semi-automatic playing mode meet the preset requirements.

[0118] In this embodiment, after the bionic hand 1 enters the semi-automatic playing mode, while the first control module 41 controls the fingers 21 to play according to the musical score and electromyographic signals, the switching module 42 needs to determine whether the notes played in the semi-automatic playing mode meet the preset requirements. Specifically, the switching module 42 detects the played notes according to the identified musical score. When the notes played in the semi-automatic playing mode meet the preset requirements, it means that the first control module 41 can combine the musical score and electromyographic signals to control the fingers 21 to play the corresponding music well. Then, the switching module 42 controls the bionic hand 1 to switch from the semi-automatic playing mode to the fully automatic playing mode.

[0119] When the notes played in the semi-automatic playing mode do not meet the preset requirements, it indicates that the first control module 41 is controlling the fingers 21 to play according to the electromyographic signals and the musical score, but the music played does not correspond to the musical score. In this case, the first control module 41 continues to control the fingers 21 to play according to the musical score and electromyographic signals in the semi-automatic playing mode.

[0120] Understandably, as long as it is in semi-automatic playing mode, the switching module 42 will detect the played notes in real time. When it detects that several consecutive notes meet the preset requirements, the switching module 42 will switch the playing mode of the bionic hand 1 from semi-automatic playing mode to fully automatic playing mode.

[0121] The second control module 43 is used to control the fingers to play according to the sheet music in fully automatic playing mode.

[0122] In this embodiment, in the fully automatic playing mode, the second control module 43 controls the fingers 21 to play according to the sheet music. Specifically, after entering the fully automatic playing mode, the second control module 43 no longer needs electromyographic signals to directly control the fingers 21 to play according to the sheet music. Even in the fully automatic playing mode, the second control module 43 still receives electromyographic signals. The second control module 43 can block the electromyographic signals and not execute the control operations corresponding to the electromyographic signals until the bionic hand 1 exits the fully automatic playing mode or the sheet music is played. It can be understood that in the fully automatic playing mode, the wearer does not need to provide electromyographic signals, and the bionic hand 1 can autonomously and independently complete the playing of the sheet music.

[0123] Please refer to the following: Figure 9 This is a schematic diagram of the second control module of the bionic hand control device provided in the first embodiment of the present invention. The second control module 43 includes an acquisition module 431 and a sub-control module 432.

[0124] The acquisition module 431 is used to acquire the corresponding preset playing information based on the musical score.

[0125] In this embodiment, the acquisition module 431 can obtain corresponding preset playing information based on the identified musical score. Specifically, the acquisition module 431 pre-stores preset playing information corresponding to each musical score, with each score corresponding to one preset playing information. The preset playing information may include bending angle, bending speed, and bending force, etc.

[0126] The sub-control module 432 is used to control the fingers to play according to the musical score and the corresponding preset playing information.

[0127] In this embodiment, the sub-control module 432 controls the fingers 21 to play according to the musical score and the preset playing information corresponding to the score. It should be noted that each note in the musical score can correspond to a bending angle, bending speed, and bending force. Specifically, the sub-control module 432 controls the fingers 21 to press the keys according to the playing order of the musical score, based on the bending angle, bending speed, and bending force corresponding to the notes, thereby playing music that matches the volume of the note and the rhythm of the musical score.

[0128] Please refer to the following: Figure 10This is a schematic diagram of the control device for the bionic hand provided in the second embodiment of the present invention. The control device 40 for the bionic hand also includes a confirmation module 44.

[0129] The confirmation module 44 is used to confirm that the notes played by the bionic hand in the semi-automatic playing mode meet the preset requirements when it detects that several consecutive notes played by the bionic hand in the semi-automatic playing mode match the musical score.

[0130] In this embodiment, the confirmation module 44 detects several consecutive notes and determines whether the consecutive notes match the musical score. Specifically, the confirmation module 44 can determine whether the pitch, playing speed, etc., of the several consecutive notes match the musical score. When the pitch, playing speed, etc., of the several consecutive notes match the musical score, the confirmation module 44 confirms that the notes played by the bionic hand 1 in the semi-automatic playing mode meet the preset requirements; when the pitch or playing speed of the several consecutive notes does not match the musical score, the confirmation module 44 confirms that the notes played by the bionic hand 1 in the semi-automatic playing mode do not meet the preset requirements.

[0131] Please refer to the following: Figure 11 This is a schematic diagram of the control device for the bionic hand provided in the third embodiment of the present invention. The control device 40 for the bionic hand also includes a third control module 45.

[0132] The third control module 45 is used to control the bionic hand to enter a fully automatic playing mode based on the received trigger information when the musical score is recognized based on the notes played.

[0133] In this embodiment, the initial playing mode of the bionic hand 1 is the normal playing mode. While the fingers 21 are playing according to electromyographic signals, the third control module 45 performs sound recognition detection on the played notes. When the wearer identifies the sheet music being played based on the played notes, the third control module 45 determines whether a trigger message has been received. If a trigger message is received, the third control module 45 identifies the trigger message to control the bionic hand 1 to enter the fully automatic playing mode, that is, to switch from the normal playing mode to the fully automatic playing mode.

[0134] Trigger information includes electromyographic (EMG) data, motion data, and / or control commands corresponding to specific actions. Trigger information can be any one of EMG data, motion data, and control commands, or a combination of any two or three of these. Specific actions can be specific hand gestures, such as all fingers 21 of the bionic hand 1 rapidly making two consecutive clenching and opening fist movements, a single specific finger 21 of the bionic hand 1 repeatedly making rapid bending and opening movements, or wrist rotation movements of the bionic hand 1. The wearer can generate EMG data as trigger information by performing specific hand gestures; the wearer can generate motion data as trigger information by performing specific hand gestures; and the wearer can also send control commands via external devices as trigger information.

[0135] Specifically, when the wearer makes a specific hand gesture, the electromyographic electrodes collect the electromyographic signals and aggregate them into electromyographic data, which is then sent to the third control module 45 as trigger information. The bionic hand 1 also includes an inertial measurement unit (IMU) that is communicatively connected to the third control module 45. When the wearer makes a specific hand gesture, the inertial sensor collects the data and aggregates it into motion data, which is then sent to the third control module 45 as trigger information. The third control module 45 is also communicatively connected to external devices. When the wearer generates control commands through external devices, the external devices send the control commands as trigger information to the third control module 45.

[0136] In some embodiments, the third control module 45 can control the bionic hand 1 to enter a fully automatic playing mode when it detects that the trigger information matches a specific action. Specifically, the third control module 45 matches the received trigger information with a pre-set specific action. When the trigger information matches the specific action, the third control module 45 controls the bionic hand 1 to enter the fully automatic playing mode. It is understood that the third control module 45 only needs to match the trigger information with a specific action when the trigger information is electromyographic data or motion data; when the trigger information is a control command, the third control module 45 directly controls the bionic hand 1 to enter the fully automatic playing mode according to the control command. That is to say, the control command can directly control the bionic hand 1 to enter the fully automatic playing mode. The specific action can be different from the bionic hand 1's daily routine actions to avoid the wearer accidentally triggering the fully automatic playing mode during daily activities.

[0137] Please refer to the following: Figure 12 This is a schematic diagram of the bionic hand module provided in an embodiment of the present invention. The bionic hand 1 includes a bionic hand body 20 and a bionic hand control device 40. The bionic hand body 20 includes a plurality of fingers 21. The control device 40 is disposed on the bionic hand body 20 and is used to drive the fingers 21 to play.

[0138] In this embodiment, the control device 40 can drive the fingers 21 to play musical instruments. These instruments include, but are not limited to, keyboard instruments such as pianos, electronic keyboards, and accordions. The bionic hand body 20 has a receiving cavity, and multiple electromyographic electrodes are disposed on the cavity wall. The control device 40 is electrically connected to these electromyographic electrodes. The bionic hand 1 is fixed to the wearer's arm (residual limb) through the receiving cavity. The cavity wall is in contact with the wearer's arm muscles. The electromyographic electrodes can collect the action potentials generated by the muscles, forming electromyographic signals on the surface of the residual limb, and then send these signals to the control device 40.

[0139] In some embodiments, the bionic hand 1 further includes an inertial sensor disposed on the bionic hand body 20. The inertial sensor is communicatively connected to the control device 40. The inertial sensor can collect motion data generated by the wearer controlling the bionic hand 1 to move, and send the motion data to the control device 40.

[0140] In some embodiments, the control device 40 can also communicate with an external device. The wearer can send control commands to the control device 40 via the external device. The external device can be a smartphone, tablet, laptop, desktop computer, or smartwatch, etc.

[0141] The specific structure of the control device 40 is as described in the above embodiments. Since the bionic hand 1 adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, and will not be described in detail here.

[0142] The above description is only a part or preferred embodiment of the present invention. Neither the text nor the drawings should limit the scope of protection of the present invention. All equivalent structural transformations made using the content of the present invention specification and drawings under the overall concept of the present invention, or direct / indirect applications in other related technical fields, are included within the scope of protection of the present invention.

Claims

1. A method for switching playing modes of a bionic hand, wherein the bionic hand comprises several fingers, characterized in that, The method for switching the bionic hand playing mode includes: When the bionic hand is in semi-automatic playing mode, it controls the corresponding fingers to play according to the musical score and the electromyographic signals on the surface of the residual arm. When it is detected that the notes played by the bionic hand in the semi-automatic playing mode meet the preset requirements, the bionic hand is controlled to switch from the semi-automatic playing mode to the fully automatic playing mode; and In the fully automatic playing mode, the fingers are controlled to play according to the musical score; The step of controlling the fingers to play according to the musical score in the fully automatic playing mode includes: Based on the musical score, corresponding preset playing information is obtained, including bending angle, bending speed, and bending force; and Control the fingers to play according to the musical score and the corresponding preset playing information.

2. The method for switching the bionic hand playing mode according to claim 1, characterized in that, The method for switching the bionic hand playing mode also includes: When the musical score is recognized based on the notes played, the bionic hand is controlled to enter the fully automatic playing mode based on the received trigger information, which includes electromyographic data, motion data and / or control commands corresponding to specific actions.

3. The method for switching the bionic hand playing mode according to claim 1, characterized in that, Before controlling the bionic hand to switch from the semi-automatic playing mode to the fully automatic playing mode, the method for switching the playing mode of the bionic hand further includes: When it is detected that a series of consecutive notes played by the bionic hand in the semi-automatic playing mode match the musical score, it is confirmed that the notes played by the bionic hand in the semi-automatic playing mode meet the preset requirements.

4. The method for switching the bionic hand playing mode according to claim 1, characterized in that, After controlling the fingers to play according to the sheet music in the fully automatic playing mode, the method for switching the bionic hand playing mode further includes: When it is detected that the notes played by the bionic hand in the fully automatic playing mode do not meet the preset requirements, the bionic hand is controlled to switch from the fully automatic playing mode to the semi-automatic playing mode.

5. The method for switching the bionic hand playing mode according to claim 4, characterized in that, After controlling the bionic hand to switch from the fully automatic playing mode to the semi-automatic playing mode, the method for switching the bionic hand playing mode further includes: When it is detected that the notes played by the bionic hand in the semi-automatic playing mode do not meet the preset requirements, the bionic hand is controlled to switch from the semi-automatic playing mode to the normal playing mode; and In the normal playing mode, the fingers are controlled to play according to the electromyographic signals.

6. The method for switching the bionic hand playing mode according to claim 1, characterized in that, After controlling the bionic hand to switch from the semi-automatic playing mode to the fully automatic playing mode, the method for switching the playing mode of the bionic hand further includes: Send a notification message.

7. A control device for a bionic hand, the bionic hand comprising a plurality of fingers, characterized in that, The control device for the bionic hand includes: The first control module is used to control the corresponding fingers to play music based on the sheet music and electromyographic signals on the surface of the residual arm when the bionic hand is in semi-automatic playing mode. A switching module is configured to control the bionic hand to switch from the semi-automatic playing mode to the fully automatic playing mode when it detects that the notes played by the bionic hand in the semi-automatic playing mode meet preset requirements; and The second control module is used to control the fingers to play according to the sheet music in the fully automatic playing mode; The second control module includes: The acquisition module is used to acquire corresponding preset playing information based on the musical score, the preset playing information including bending angle, bending speed, and bending force; and The sub-control module is used to control the fingers to play according to the musical score and the corresponding preset playing information.

8. A bionic hand, characterized in that, The bionic hand includes a bionic hand body and a control device for the bionic hand as described in claim 7. The bionic hand body includes a plurality of fingers, and the control device is disposed on the bionic hand body for driving the fingers to play.

Images