Bionic hand and method of playing it, control device

Abstract

The application discloses a playing method of a bionic hand. The bionic hand comprises a plurality of fingers. The playing method of the bionic hand comprises the following steps: when a musical score is recognized according to a played note, the bionic hand is controlled to enter a semi-automatic playing mode; and in the semi-automatic playing mode, corresponding fingers are controlled to play according to the musical score and a myoelectric signal on a residual arm surface. The playing method of the bionic hand disclosed by the application can solve the problem of slow response speed when the bionic hand plays a musical instrument. In addition, the application further discloses a control device of the bionic hand and the bionic hand.

Description

Technical Field

[0001] This invention relates to the field of bionic hand technology, and in particular to a bionic hand and its playing method and control device. Background Technology

[0002] In existing bionic hands, each finger is controlled by an independent drive motor to flex and extend, and the bending angle of the finger is determined by electromyographic signals that control flexion and extension.

[0003] However, in certain specific application scenarios, such as playing the piano, the bionic hand cannot quickly respond to all the wearer's electromyographic signals to control all the fingers to move quickly. It can only respond to the electromyographic signals of a single finger in turn, simply controlling a single finger to press the piano key. This makes playing the piano very strenuous, with low accuracy and slow rhythm. As a result, it is difficult for the wearer to continue using the bionic hand for piano training, resulting in a poor user experience. Summary of the Invention

[0004] The main objective of this invention is to propose a bionic hand and its playing method and control device, aiming to solve the problem of slow response speed when the bionic hand plays the piano.

[0005] To achieve the above objectives, the present invention proposes a method for playing music using a bionic hand, the bionic hand comprising a plurality of fingers, and the method for playing music using the bionic hand comprising:

[0006] When the musical score is recognized based on the notes played, the bionic hand is controlled to enter a semi-automatic playing mode; and

[0007] In the semi-automatic playing mode, the corresponding fingers are controlled to play according to the musical score and the electromyographic signals on the surface of the residual arm.

[0008] Preferably, the step of controlling the corresponding fingers to play according to the sheet music and the electromyographic signals on the surface of the residual arm in the semi-automatic playing mode includes:

[0009] Based on the musical score, corresponding preset playing information is obtained, including bending angle, bending speed, and bending force; and

[0010] The corresponding fingers are controlled to play according to the preset playing information based on the electromyographic signals.

[0011] Preferably, after controlling the corresponding fingers to play according to the musical score and the electromyographic signals on the surface of the residual arm in the semi-automatic playing mode, the playing method of the bionic hand further includes:

[0012] Determine whether the played notes meet the preset requirements;

[0013] When the notes played meet the preset requirements, the bionic hand is controlled to switch from the semi-automatic playing mode to the fully automatic playing mode.

[0014] In the fully automatic playing mode, the fingers are controlled to play according to the musical score; and

[0015] If the notes played do not meet the preset requirements, continue to control the fingers to play in the semi-automatic playing mode.

[0016] Preferably, after controlling the corresponding fingers to play according to the musical score and the electromyographic signals on the surface of the residual arm in the semi-automatic playing mode, the playing method of the bionic hand further includes:

[0017] Determine whether the played notes meet the preset requirements;

[0018] When the notes played meet the preset requirements, continue to control the fingers to play in the semi-automatic playing mode;

[0019] When the notes played do not meet the preset requirements, the bionic hand is controlled to switch from the semi-automatic playing mode to the normal playing mode; and

[0020] In the normal playing mode, the fingers are controlled to play according to the electromyographic signals.

[0021] Preferably, determining whether the played notes meet preset requirements includes:

[0022] Determine whether a series of consecutive notes match the musical score;

[0023] When a series of consecutive notes match the musical score, it is confirmed that the played notes meet the preset requirements; and

[0024] If a series of notes do not match the musical score, it is confirmed that the notes played do not meet the preset requirements.

[0025] The present invention further proposes a control device for a bionic hand, the bionic hand comprising a plurality of fingers, the control device for the bionic hand comprising:

[0026] The first control module is used to control the bionic hand to enter a semi-automatic playing mode when recognizing the notes being played and determining the musical score; and

[0027] The second control module is used to control the corresponding fingers to play according to the sheet music and electromyographic signals on the surface of the residual arm in the semi-automatic playing mode.

[0028] Preferably, the second control module includes:

[0029] The acquisition module is used to acquire corresponding preset playing information based on the musical score, the preset playing information including bending angle, bending speed, and bending force; and

[0030] The first sub-control module is used to control the corresponding fingers to play according to the preset playing information based on the electromyographic signals.

[0031] Preferably, the control device for the bionic hand further includes:

[0032] The first judgment module is used to determine whether the played notes meet the preset requirements;

[0033] The first switching module is used to control the bionic hand to switch from the semi-automatic playing mode to the fully automatic playing mode when the notes played meet the preset requirements.

[0034] The third control module is used to control the fingers to play according to the sheet music in the fully automatic playing mode; and

[0035] The fourth control module is used to continue controlling the fingers to play in the semi-automatic playing mode when the notes played do not meet the preset requirements.

[0036] Preferably, the control device for the bionic hand further includes:

[0037] The second judgment module is used to determine whether the played notes meet the preset requirements;

[0038] The fifth control module is used to continue controlling the fingers to play in the semi-automatic playing mode when the played notes meet the preset requirements.

[0039] The second switching module is used to control the bionic hand to switch from the semi-automatic playing mode to the normal playing mode when the played notes do not meet the preset requirements; and

[0040] The sixth control module is used to control the fingers to play according to the electromyographic signals in the normal playing mode.

[0041] The present invention further proposes a bionic hand, which includes a bionic hand body and a control device for the bionic hand as described above. The bionic hand body includes a plurality of fingers, and the control device is disposed on the bionic hand body for driving the fingers to play.

[0042] The technical solution of this invention is as follows: After recognizing the sheet music currently being played by the bionic hand based on the notes being played, the bionic hand is controlled to enter a semi-automatic playing mode. In this mode, the fingers are controlled to play according to the recognized sheet music and electromyographic signals. Controlling the bionic hand into a semi-automatic playing mode can improve the response speed of the bionic hand when playing musical instruments, produce rhythms that match the sheet music, and at the same time reduce the wearer's control burden, thus improving the wearer's experience. Attached Figure Description

[0043] Figure 1 A flowchart illustrating the playing method of the bionic hand provided in an embodiment of the present invention;

[0044] Figure 2 This is a first sub-flowchart of the bionic hand playing method provided in an embodiment of the present invention;

[0045] Figure 3 This is a second sub-flowchart of the bionic hand playing method provided in an embodiment of the present invention;

[0046] Figure 4 The third sub-flowchart of the bionic hand playing method provided in the embodiment of the present invention;

[0047] Figure 5 The fourth sub-flowchart of the bionic hand playing method provided in the embodiment of the present invention;

[0048] Figure 6 A schematic diagram of a bionic hand provided in an embodiment of the present invention;

[0049] Figure 7 This is a schematic diagram of the structure of the control device provided in an embodiment of the present invention;

[0050] Figure 8 This is a schematic diagram of the control device for the bionic hand provided in the first embodiment of the present invention;

[0051] Figure 9 for Figure 8 A schematic diagram of the second control module of the control device for the bionic hand shown.

[0052] Figure 10 This is a schematic diagram of the control device for the bionic hand provided in the second embodiment of the present invention;

[0053] Figure 11 This is a schematic diagram of the control device for the bionic hand provided in the third embodiment of the present invention;

[0054] Figure 12 This is a schematic diagram of a bionic hand module provided in an embodiment of the present invention.

[0055] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0056] The solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0058] It should also be noted that when a component is described as "fixed to" or "set on" another component, it can be directly on the other component or there may be an intervening component present. When a component is described as "connected to" another component, it can be directly connected to the other component or there may be an intervening component present.

[0059] Furthermore, the use of terms such as "first" and "second" in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed by this invention.

[0060] Please refer to the following: Figure 1 and Figure 6 , Figure 1 This is a flowchart of a bionic hand playing method provided in an embodiment of the present invention. Figure 6 This is a schematic diagram of a bionic hand provided in an embodiment of the present invention. The bionic hand playing method is applied to a bionic hand 1, which includes a plurality of fingers 21. The bionic hand playing method is used to control the fingers 21 of the bionic hand 1 to play musical instruments. The musical instruments include, but are not limited to, keyboard instruments such as pianos, electronic keyboards, and accordions.

[0061] In this embodiment, the bionic hand 1 is provided with a receiving cavity 30. The bionic hand 1 is fixed to the wearer's arm (residual limb) through the receiving cavity 30, and the cavity wall of the receiving cavity 30 is in close contact with the wearer's arm muscles. The bionic hand 1 has multiple electromyographic electrodes (not shown) on the cavity wall of the receiving cavity 30. The electromyographic electrodes are used to collect the action potential generated by the muscles and form electromyographic signals on the surface of the residual limb.

[0062] The bionic hand 1 also includes a control device 10 for performing the playing method of the bionic hand, and the control device 10 is electrically connected to electromyographic electrodes. The relevant functions of the control device 10 can be implemented by a single device, multiple devices working together, or one or more functional modules within a single device; no specific limitation is made here. It is understood that the aforementioned functions can be network elements in hardware devices, software functions running on dedicated hardware, a combination of hardware and software, or virtualized functions instantiated on a platform (e.g., a cloud platform).

[0063] The specific steps involved in playing music with the bionic hand are as follows.

[0064] In step S102, when the sheet music is recognized based on the notes played, the bionic hand is controlled to enter a semi-automatic playing mode.

[0065] In this embodiment, during the process of the wearer controlling the bionic hand 1 to play a musical instrument, the wearer generates a motor intention through the brain, which excites the peripheral nervous system and thus controls the contraction of the forearm muscle group. The electromyographic electrodes on the surface of the residual limb receive stimulation and generate corresponding electromyographic signals; the control device 10 receives the electromyographic signals sent by the electrodes, controls the bending of the fingers 21 of the bionic hand 1, and controls the bending angle of the fingers 21 according to the duration of the electromyographic signals, so that the fingers 21 press the piano keys to play the instrument. The bending angle can represent the angle formed between the distal phalanx (corresponding to the distal phalanx) of the finger 21 and the palm plane when the finger 21 is bent; the larger the angle, the greater the degree of bending of the finger 21. In some feasible embodiments, the bending angle can also represent the angle formed between other parts of the finger 21 (corresponding to the middle or proximal phalanx) and the palm plane when the finger 21 is bent.

[0066] The control device 10 is equipped with a sound recognition system for detecting the notes being played. During playing, the control device 10 performs sound recognition detection on the played notes and matches them with scores in a preset music score library to obtain a score that matches the played notes. In other words, the control device 10 identifies the score the wearer is currently playing based on the played notes. After identifying the score based on the played notes, the control device 10 controls the bionic hand 1 to enter a semi-automatic playing mode.

[0067] Step S104: In semi-automatic playing mode, control the corresponding fingers to play according to the sheet music and electromyographic signals on the surface of the residual arm.

[0068] In this embodiment, in semi-automatic playing mode, the control device 10 controls the corresponding fingers 21 to play based on the sheet music and electromyographic signals on the surface of the residual arm. It is understood that before entering semi-automatic playing mode, the control device 10 can only control the fingers 21 to play based on electromyographic signals; after entering semi-automatic playing mode, the control device 10 can obtain the corresponding playing speed based on the intervals of notes in the analyzed sheet music, and combines the playing speed and electromyographic signals to jointly control the fingers 21 to play. Specifically, the control device 10 can control the bending speed of the fingers 21 based on the playing speed.

[0069] The specific process of controlling the corresponding fingers to play music based on the sheet music and electromyographic signals on the surface of the residual arm in semi-automatic playing mode will be described in detail below.

[0070] In this embodiment, after recognizing the sheet music being played by the bionic hand based on the notes being played, the bionic hand is controlled to enter a semi-automatic playing mode. In this mode, the fingers are controlled to play according to the recognized sheet music and electromyographic signals. Controlling the bionic hand into semi-automatic playing mode improves the response speed when playing instruments, produces rhythms that match the sheet music, and reduces the wearer's control burden, thus enhancing the wearer's experience.

[0071] Please refer to the following: Figure 2 This is the first sub-flowchart of the bionic hand playing method provided in this embodiment of the invention. Step S104 specifically includes the following steps.

[0072] Step S202: Obtain the corresponding preset playing information based on the musical score.

[0073] In this embodiment, the control device 10 can obtain corresponding preset playing information based on the identified musical score. Specifically, the control device 10 pre-stores preset playing information corresponding to each musical score, with one preset playing information per score. The preset playing information may include bending angle, bending speed, and bending force, etc.

[0074] Step S204: Control the corresponding fingers to play according to preset playing information based on electromyographic signals.

[0075] In this embodiment, the control device 10 controls the corresponding finger 21 to bend according to the received electromyographic signals, and at the same time, the control device 10 controls the finger 21 to press the piano keys according to the corresponding preset playing information. It should be noted that each note in the musical score can correspond to a bending angle, bending speed, and bending force. The control device 10 controls the finger 21 to move according to the bending angle, bending speed, and bending force corresponding to the note, based on the received electromyographic signals and in combination with the playing order of the musical score, thereby playing music that matches the volume of the note and the rhythm of the music.

[0076] For example, when the wearer controls the bionic hand 1 to play a musical piece, the control device 10 controls the fingers 21 to play the notes 123455543 based on the received electromyographic signals and corresponding durations. During the playing process, the control device 10 detects the played notes. When the fingers 21 play the second 3, the control device 10 recognizes that the music being played is "The Doll and the Bear Dancing," and then controls the bionic hand 1 to enter a semi-automatic playing mode. The control device 10 acquires the preset playing information corresponding to "The Doll and the Bear Dancing," and controls the corresponding fingers 21 to play the following notes of "The Doll and the Bear Dancing": 44432135... based on the electromyographic signals.

[0077] Understandably, in semi-automatic playing mode, as long as the control device 10 receives an electromyographic signal, it can control the corresponding finger 21 to bend according to the corresponding bending angle, bending speed, and bending force, thereby playing the corresponding musical score. After receiving the electromyographic signal, the control device 10 does not need to detect the duration of the electromyographic signal; it can directly control the corresponding finger 21 based on the electromyographic signal.

[0078] In this embodiment, after the bionic hand enters the semi-automatic playing mode, it obtains the corresponding preset playing information based on the identified musical score. Based on the received electromyographic signals, it controls the corresponding fingers to play the notes corresponding to the musical score according to the preset playing information. It is no longer necessary to detect the duration of the electromyographic signals, which makes it easier for the wearer to control the bionic hand to play musical instruments. At the same time, it can effectively improve the accuracy of playing and greatly enhance the user experience.

[0079] Please refer to the following: Figure 3 This is the second sub-flowchart of the bionic hand playing method provided in this embodiment of the invention. After executing step S104, the bionic hand playing method further includes the following steps.

[0080] Step S302: Determine whether the played notes meet the preset requirements.

[0081] In this embodiment, after the bionic hand 1 enters the semi-automatic playing mode, the control device 10, while controlling the fingers 21 to play according to the musical score and electromyographic signals, also needs to determine whether the played notes meet the preset requirements. Specifically, the control device 10 detects the played notes according to the identified musical score.

[0082] The specific process of determining whether the played notes meet the preset requirements will be described in detail below.

[0083] If the played notes meet the preset requirements, proceed to step S304; if the played notes do not meet the preset requirements, proceed to step S308.

[0084] Step S304: Control the bionic hand to switch from semi-automatic playing mode to fully automatic playing mode.

[0085] When the notes played meet the preset requirements, it means that the control device 10 can combine the preset playing information and electromyographic signals to control the finger 21 to play the corresponding musical score well. Then the control device 10 controls the bionic hand 1 to switch from the semi-automatic playing mode to the fully automatic playing mode.

[0086] In some embodiments, when the bionic hand 1 switches from a semi-automatic playing mode to a fully automatic playing mode, the control device 10 can send a prompt message to remind the wearer. The prompt message may include, but is not limited to, voice, light, vibration, or other forms.

[0087] Step S306: In fully automatic playing mode, control your fingers to play according to the sheet music.

[0088] In this embodiment, in fully automatic playing mode, the control device 10 controls the fingers 21 to play according to the sheet music. Specifically, the control device 10 controls the fingers 21 to play according to the sheet music and the corresponding preset playing information. It is understood that after entering fully automatic playing mode, the control device 10 no longer needs electromyographic signals to directly control the fingers 21 according to the sheet music and the corresponding preset playing information. Even in fully automatic playing mode, the control device 10 still receives electromyographic signals. The control device 10 can block the electromyographic signals and not execute the control operations corresponding to the electromyographic signals until the bionic hand 1 exits the fully automatic playing mode or the sheet music is played.

[0089] Step S308: Continue to control your fingers to play in semi-automatic playing mode.

[0090] If the notes played do not meet the preset requirements, it means that during the process of the control device 10 controlling the finger 21 to play according to the electromyographic signal and the preset playing information, there is a situation where the notes played do not correspond to the score. Then the control device 10 continues to control the finger 21 to play according to the score and electromyographic signal in the semi-automatic playing mode.

[0091] For example, if the bionic hand 1 enters semi-automatic playing mode, the control device 10 controls the corresponding finger 21 to play the following notes of "The Doll and the Bear Dancing" according to electromyographic signals: 44432135. When finger 21 plays note 5, the control device 10 detects that the played note meets the preset requirements, and then the control device 10 controls the bionic hand 1 to enter fully automatic playing mode. The control device 10 controls the corresponding finger 21 to play the following notes according to the sheet music of "The Doll and the Bear Dancing" and the corresponding preset playing information: 123455543... If the bionic hand 1 enters semi-automatic playing mode, the control device 10 controls the corresponding finger 21 to play the following notes of "The Doll and the Bear Dancing" according to electromyographic signals: 44432136. When finger 21 plays 6, the control device 10 detects that the played note does not meet the preset requirements. Then, the control device 10 continues to control finger 21 to play the following notes of "The Doll and the Bear Dancing" according to the musical score and electromyographic signals: 123455543...

[0092] In this embodiment, as long as the semi-automatic playing mode is in operation, the control device 10 detects the played notes in real time. When several consecutive notes are detected to meet the preset requirements, the control device 10 switches the mode of the bionic hand 1 from the semi-automatic playing mode to the fully automatic playing mode.

[0093] In this embodiment, the played notes are detected in semi-automatic playing mode. When a note does not meet the preset requirements, the bionic hand remains in semi-automatic playing mode. When a note meets the preset requirements, the bionic hand switches from semi-automatic playing mode to fully automatic playing mode. In fully automatic playing mode, the bionic hand can directly control its fingers to play the corresponding notes based on the sheet music and preset playing information, greatly reducing the wearer's control burden and making instrument playing easy and simple, with high accuracy and smooth rhythm.

[0094] Please refer to the following: Figure 4 This is the third sub-flowchart of the bionic hand playing method provided in this embodiment of the invention. After executing step S104, the bionic hand playing method further includes the following steps.

[0095] Step S402: Determine whether the played notes meet the preset requirements.

[0096] In this embodiment, after the bionic hand 1 enters the semi-automatic playing mode, the control device 10, while controlling the fingers 21 to play according to the musical score and electromyographic signals, also needs to determine whether the played notes meet the preset requirements. Specifically, the control device 10 detects the played notes according to the identified musical score.

[0097] The specific process of determining whether the played notes meet the preset requirements will be described in detail below.

[0098] If the played notes meet the preset requirements, proceed to step S404; if the played notes do not meet the preset requirements, proceed to step S406.

[0099] Step S404: Continue to control your fingers to play in semi-automatic playing mode.

[0100] When the notes played meet the preset requirements, it means that the control device 10 can combine the preset playing information and electromyographic signals to control the finger 21 to play the corresponding musical score well. Then the control device 10 continues to control the finger 21 to play in the semi-automatic playing mode according to the musical score and electromyographic signals.

[0101] Step S406: Control the bionic hand to switch from semi-automatic playing mode to normal playing mode.

[0102] If the notes played do not meet the preset requirements, it means that during the process of the control device 10 controlling the finger 21 to play according to the electromyographic signal and the preset playing information, there is a situation where the notes played do not correspond to the musical score. Then the control device 10 controls the bionic hand 1 to switch from the semi-automatic playing mode to the normal playing mode.

[0103] In some embodiments, when the bionic hand 1 switches from a semi-automatic playing mode to a normal playing mode, the control device 10 can send a prompt message to remind the wearer. This prompt message may include, but is not limited to, voice, light, or vibration. It is understood that the prompt message sent by the control device 10 when the bionic hand 1 switches from a semi-automatic playing mode to a fully automatic playing mode is different from the prompt message sent by the control device 10 when the bionic hand 1 switches from a semi-automatic playing mode to a normal playing mode.

[0104] Step S408: In normal playing mode, control the fingers to play according to electromyographic signals.

[0105] In this embodiment, in normal playing mode, the control device 10 can only control the finger 21 to play based on the electromyographic signal and the duration corresponding to the electromyographic signal. It can be understood that when the wearer puts on the bionic hand 1 and begins to play, the bionic hand 1 is in normal playing mode.

[0106] In this embodiment, as long as the semi-automatic playing mode is in operation, the control device 10 detects the played notes in real time. When several consecutive notes are detected as not meeting the preset requirements, the control device 10 switches the mode of the bionic hand 1 from the semi-automatic playing mode to the normal playing mode.

[0107] In this embodiment, the notes played are detected in semi-automatic playing mode. When the detected notes meet the preset requirements, the bionic hand remains in semi-automatic playing mode. When the detected notes do not meet the preset requirements, the bionic hand is controlled to switch from semi-automatic playing mode to normal playing mode. This allows for timely switching of the bionic hand's playing mode when problems occur, enabling the wearer to better control the bionic hand and effectively improving the user experience.

[0108] The bionic hand 1 is pre-set with multiple playing modes, including normal playing mode, semi-automatic playing mode, and fully automatic playing mode. The control device 10 switches between playing modes based on the recognized notes. In normal playing mode, the control device 10 controls the finger 21 to play based on electromyographic signals and their duration. In semi-automatic playing mode, the control device 10 no longer needs to detect the duration of the electromyographic signals; it only needs to control the finger 21 to play based on the electromyographic signals and the corresponding preset playing information. In fully automatic playing mode, the control device 10 can directly control the finger 21 to play based on the sheet music and preset playing information without requiring any electromyographic signals. The conditions for the bionic hand to switch to normal or fully automatic playing mode in semi-automatic mode can be set according to the actual playing situation or the wearer's personal preference, and are not limited here.

[0109] Please refer to the following: Figure 5 This is the fourth sub-flowchart of the bionic hand playing method provided in the embodiments of the present invention. Step S302 or step S402 specifically includes the following steps.

[0110] Step S502: Determine whether a series of consecutive notes match the musical score.

[0111] In this embodiment, the control device 10 detects a series of consecutive musical notes and determines whether the notes match the musical score. Specifically, the control device 10 can determine whether the pitch, playing speed, etc., of the consecutive musical notes match the musical score.

[0112] When a series of consecutive notes match the score, proceed to step S504; when a series of consecutive notes do not match the score, proceed to step S506.

[0113] Step S504: Confirm that the notes played meet the preset requirements.

[0114] When a series of notes match the musical score, the control device 10 confirms that the notes played meet the preset requirements.

[0115] Step S506: Confirm that the notes played do not meet the preset requirements.

[0116] When several consecutive notes do not match the score, the control device 10 confirms that the notes played do not meet the preset requirements.

[0117] Please refer to the following: Figure 7 This is a schematic diagram of the structure of the control device provided in an embodiment of the present invention. The present invention also proposes a control device 10, which can be a computing device such as a desktop computer, laptop, handheld computer, or server. The control device 10 may include: a processor 1001 (e.g., CPU), a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize communication between these components. The user interface 1003 may include a display screen and an input unit, such as a keyboard; the user interface 1003 may also include standard wired interfaces and wireless interfaces. The network interface 1004 may include standard wired interfaces and wireless interfaces (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or stable non-volatile memory, such as disk storage; the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0118] Those skilled in the art will understand that Figure 7 The structure of the control device 10 shown does not constitute a limitation on the control device 10. The control device 10 may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0119] like Figure 7 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and computer-executable instructions.

[0120] exist Figure 7 In the control device 10 shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the computer-executable instructions stored in the memory 1005. When the computer-executable instructions are called and executed by the processor 1001, the steps of the above-mentioned bionic hand playing method are implemented.

[0121] Based on the computer-executable instructions proposed in the foregoing embodiments, the present invention also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the bionic hand playing method described in the foregoing embodiments.

[0122] Please refer to the following: Figure 8 This is a schematic diagram of the control device for the bionic hand provided in the first embodiment of the present invention. The control device 40 for the bionic hand includes a first control module 41 and a second control module 42.

[0123] The first control module 41 is used to control the bionic hand to enter a semi-automatic playing mode when the musical score is recognized based on the notes played.

[0124] In this embodiment, during the process of the wearer controlling the bionic hand 1 to play a musical instrument, the wearer generates a motor intention through the brain, which excites the peripheral nervous system and thus controls the contraction of the forearm muscle group. The electromyographic electrodes on the surface of the residual limb receive stimulation and generate corresponding electromyographic signals; the first control module 41 receives the electromyographic signals sent by the electromyographic electrodes, controls the fingers 21 of the bionic hand 1 to bend, and controls the bending angle of the fingers 21 according to the duration of the electromyographic signals, so that the fingers 21 press the piano keys to play the instrument. The bending angle can represent the angle formed between the distal phalanx (corresponding to the distal phalanx) of the finger 21 and the palm plane when the finger 21 is bent; the larger the angle, the greater the degree of bending of the finger 21. In some feasible embodiments, the bending angle can also represent the angle formed between other parts of the finger 21 (corresponding to the middle or proximal phalanx) and the palm plane when the finger 21 is bent.

[0125] The first control module 41 is equipped with a sound recognition system for detecting the notes being played. During playing, the first control module 41 performs sound recognition detection on the played notes and matches them with scores in a preset music score library to obtain a score that matches the played notes. In other words, the first control module 41 identifies the score the wearer is currently playing based on the played notes. After identifying the score based on the played notes, the first control module 41 controls the bionic hand 1 to enter a semi-automatic playing mode.

[0126] The second control module 42 is used to control the corresponding fingers to play according to the sheet music and electromyographic signals on the surface of the residual arm in semi-automatic playing mode.

[0127] In this embodiment, in the semi-automatic playing mode, the second control module 42 controls the corresponding fingers 21 to play based on the sheet music and the electromyographic signals on the surface of the residual arm. It is understood that before entering the semi-automatic playing mode, the first control module 41 can only control the fingers 21 to play based on the electromyographic signals; after entering the semi-automatic playing mode, the second control module 42 can obtain the corresponding playing speed based on the intervals of the notes in the analyzed sheet music, and combines the playing speed and the electromyographic signals to jointly control the fingers 21 to play. Specifically, the second control module 42 can control the bending speed of the fingers 21 based on the playing speed.

[0128] Please refer to the following: Figure 9This is a schematic diagram of the second control module of the control device for the bionic hand provided in the first embodiment of the present invention. The second control module 42 includes an acquisition module 421 and a first sub-control module 422.

[0129] The acquisition module 421 is used to acquire the corresponding preset playing information based on the musical score.

[0130] In this embodiment, the acquisition module 421 can obtain corresponding preset playing information based on the identified musical score. Specifically, the acquisition module 421 pre-stores preset playing information corresponding to each musical score, with each score corresponding to one preset playing information. The preset playing information may include bending angle, bending speed, and bending force, etc.

[0131] The first sub-control module 422 is used to control the corresponding fingers to play according to preset playing information based on electromyographic signals.

[0132] In this embodiment, the first sub-control module 422 controls the corresponding finger 21 to bend according to the received electromyographic signals. Simultaneously, the first sub-control module 422 controls the finger 21 to press the piano keys according to the corresponding preset playing information. It should be noted that each note in the musical score can correspond to a bending angle, bending speed, and bending force. The first sub-control module 422, based on the received electromyographic signals and in conjunction with the playing order of the musical score, controls the movement of the finger 21 according to the bending angle, bending speed, and bending force corresponding to the note, thereby playing music that matches the volume of the note and the rhythm of the musical score.

[0133] Understandably, in semi-automatic playing mode, as long as the first sub-control module 422 receives an electromyographic signal, it can control the corresponding finger 21 to bend according to the corresponding bending angle, bending speed, and bending force, thereby playing the corresponding musical score. After receiving the electromyographic signal, the first sub-control module 422 does not need to detect the duration of the electromyographic signal; it can directly control the corresponding finger 21 according to the electromyographic signal.

[0134] Please refer to the following: Figure 10 This is a schematic diagram of the control device for the bionic hand provided in the second embodiment of the present invention. The control device 40 for the bionic hand also includes a first judgment module 431, a first switching module 432, a third control module 433, and a fourth control module 434.

[0135] The first judgment module 431 is used to judge whether the played notes meet the preset requirements.

[0136] In this embodiment, after the bionic hand 1 enters the semi-automatic playing mode, while the first sub-control module 422 controls the fingers 21 to play according to the musical score and electromyographic signals, the first judgment module 431 needs to determine whether the played notes meet the preset requirements. Specifically, the first judgment module 431 detects the played notes according to the identified musical score.

[0137] The first switching module 432 is used to control the bionic hand to switch from semi-automatic playing mode to fully automatic playing mode when the played notes meet the preset requirements.

[0138] When the notes played meet the preset requirements, it means that the first sub-control module 422 can combine the preset playing information and electromyographic signals to control the finger 21 to play the corresponding musical score well. Then the first switching module 432 controls the bionic hand 1 to switch from the semi-automatic playing mode to the fully automatic playing mode.

[0139] In some embodiments, when the bionic hand 1 switches from a semi-automatic playing mode to a fully automatic playing mode, the first switching module 432 can send a prompt message to remind the wearer. The prompt message may include, but is not limited to, voice, light, vibration, or other forms.

[0140] The third control module 433 is used to control the fingers to play according to the sheet music in fully automatic playing mode.

[0141] In this embodiment, in the fully automatic playing mode, the third control module 433 controls the finger 21 to play according to the musical score. Specifically, the third control module 433 controls the finger 21 to play according to the musical score and the corresponding preset playing information. It can be understood that after entering the fully automatic playing mode, the third control module 433 no longer needs electromyographic signals to directly control the finger 21 according to the musical score and the corresponding preset playing information. Even in the fully automatic playing mode, the third control module 433 still receives electromyographic signals. The third control module 433 can block the electromyographic signals and not execute the control operations corresponding to the electromyographic signals until the bionic hand 1 exits the fully automatic playing mode or the musical score is played.

[0142] The fourth control module 434 is used to continue controlling the fingers to play in semi-automatic mode when the notes played do not meet the preset requirements.

[0143] If the notes played do not meet the preset requirements, it means that the first sub-control module 422 is controlling the finger 21 to play according to the electromyographic signal and the preset playing information, but the notes played do not correspond to the score. Then the fourth control module 434 continues to control the finger 21 to play according to the score and electromyographic signal in the semi-automatic playing mode.

[0144] In this embodiment, as long as the semi-automatic playing mode is in operation, the first judgment module 431 detects the played notes in real time. When several consecutive notes are detected to meet the preset requirements, the first switching module 432 switches the mode of the bionic hand 1 from the semi-automatic playing mode to the fully automatic playing mode.

[0145] Please refer to the following: Figure 11 This is a schematic diagram of the control device for the bionic hand provided in the third embodiment of the present invention. The control device 40 for the bionic hand also includes a second judgment module 441, a fifth control module 442, a second switching module 443, and a sixth control module 444.

[0146] The second judgment module 441 is used to judge whether the played notes meet the preset requirements.

[0147] In this embodiment, after the bionic hand 1 enters the semi-automatic playing mode, while the first sub-control module 422 controls the fingers 21 to play according to the musical score and electromyographic signals, the second judgment module 441 needs to determine whether the played notes meet the preset requirements. Specifically, the second judgment module 441 detects the played notes according to the identified musical score.

[0148] The fifth control module 442 is used to continue controlling the fingers to play in semi-automatic playing mode.

[0149] When the notes played meet the preset requirements, it means that the first sub-control module 422 can combine the preset playing information and electromyographic signals to control the finger 21 to play the corresponding musical score well. Then the fifth control module 442 continues to control the finger 21 to play in the semi-automatic playing mode according to the musical score and electromyographic signals.

[0150] The second switching module 443 is used to control the bionic hand to switch from semi-automatic playing mode to normal playing mode when the notes played do not meet the preset requirements.

[0151] When the notes played do not meet the preset requirements, it means that during the process of the first sub-control module 422 controlling the finger 21 to play according to the electromyographic signal and the preset playing information, there is a situation where the notes played do not correspond to the musical score. Then the second switching module 443 controls the bionic hand 1 to switch from the semi-automatic playing mode to the normal playing mode.

[0152] In some embodiments, when the bionic hand 1 switches from a semi-automatic playing mode to a normal playing mode, the second switching module 443 can send a prompt message to remind the wearer. This prompt message may include, but is not limited to, voice, light, or vibration. It is understood that the prompt message sent by the first switching module 432 when the bionic hand 1 switches from a semi-automatic playing mode to a fully automatic playing mode is different from the prompt message sent by the second switching module 443 when the bionic hand 1 switches from a semi-automatic playing mode to a normal playing mode.

[0153] The sixth control module 444 is used to control the fingers to play according to electromyographic signals in normal playing mode.

[0154] In this embodiment, in normal playing mode, the sixth control module 444 can only control the finger 21 to play based on the electromyographic signal and the duration corresponding to the electromyographic signal. It can be understood that when the wearer puts on the bionic hand 1 and starts playing, the bionic hand 1 is in normal playing mode.

[0155] In this embodiment, as long as the semi-automatic playing mode is in operation, the second judgment module 441 detects the played notes in real time. When several consecutive notes are detected as not meeting the preset requirements, the second switching module 443 switches the mode of the bionic hand 1 from the semi-automatic playing mode to the normal playing mode.

[0156] Please refer to the following: Figure 12 This is a schematic diagram of the bionic hand module provided in an embodiment of the present invention. The bionic hand 1 includes a bionic hand body 20 and a bionic hand control device 40. The bionic hand body 20 includes a plurality of fingers 21. The control device 40 is disposed on the bionic hand body 20 and is used to drive the fingers 21 to play.

[0157] In this embodiment, the control device 40 can drive the fingers 21 to play musical instruments. These instruments include, but are not limited to, keyboard instruments such as pianos, electronic keyboards, and accordions. The bionic hand body 20 has a receiving cavity, and multiple electromyographic electrodes are disposed on the cavity wall. The control device 40 is electrically connected to these electromyographic electrodes. The bionic hand 1 is fixed to the wearer's arm (residual limb) through the receiving cavity. The cavity wall is in contact with the wearer's arm muscles. The electromyographic electrodes can collect the action potentials generated by the muscles, forming electromyographic signals on the surface of the residual limb, and then send these signals to the control device 40.

[0158] The specific structure of the control device 40 is as described in the above embodiments. Since the bionic hand 1 adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, and will not be described in detail here.

[0159] The above description is only a part or preferred embodiment of the present invention. Neither the text nor the drawings should limit the scope of protection of the present invention. All equivalent structural transformations made using the content of the present invention specification and drawings under the overall concept of the present invention, or direct / indirect applications in other related technical fields, are included within the scope of protection of the present invention.

Claims

1. A method for playing music using a bionic hand, the bionic hand comprising a plurality of fingers, characterized in that, The playing method of the bionic hand includes: When the sheet music is recognized based on the notes played, the bionic hand is controlled to enter a semi-automatic playing mode. In the semi-automatic playing mode, the corresponding fingers are controlled to play according to the musical score and the electromyographic signals on the surface of the residual arm. Determine whether the played notes meet the preset requirements; When the notes played meet the preset requirements, the bionic hand is controlled to switch from the semi-automatic playing mode to the fully automatic playing mode. In the fully automatic playing mode, the fingers are controlled to play according to the musical score; and If the notes played do not meet the preset requirements, continue to control the fingers to play in the semi-automatic playing mode.

2. The method for playing music with a bionic hand according to claim 1, characterized in that, The process of controlling the corresponding fingers to play according to the sheet music and electromyographic signals on the surface of the residual arm in the semi-automatic playing mode includes: Based on the musical score, corresponding preset playing information is obtained, including bending angle, bending speed, and bending force; and The corresponding fingers are controlled to play according to the preset playing information based on the electromyographic signals.

3. The method for playing music using a bionic hand according to claim 1, characterized in that, The determination of whether the played notes meet the preset requirements includes: Determine whether a series of consecutive notes match the musical score; When a series of consecutive notes match the musical score, it is confirmed that the played notes meet the preset requirements; and If a series of notes do not match the musical score, it is confirmed that the notes played do not meet the preset requirements.

4. A method for playing music using a bionic hand, characterized in that, The playing method of the bionic hand includes: When the sheet music is recognized based on the notes played, the bionic hand is controlled to enter a semi-automatic playing mode. In the semi-automatic playing mode, the corresponding fingers are controlled to play according to the musical score and the electromyographic signals on the surface of the residual arm. Determine whether the played notes meet the preset requirements; When the notes played meet the preset requirements, continue to control the fingers to play in the semi-automatic playing mode; When the notes played do not meet the preset requirements, the bionic hand is controlled to switch from the semi-automatic playing mode to the normal playing mode; and In the normal playing mode, the fingers are controlled to play according to the electromyographic signals.

5. The method for playing music with a bionic hand according to claim 4, characterized in that, The determination of whether the played notes meet the preset requirements includes: Determine whether a series of consecutive notes match the musical score; When a series of consecutive notes match the musical score, it is confirmed that the played notes meet the preset requirements; and If a series of notes do not match the musical score, it is confirmed that the notes played do not meet the preset requirements.

6. A control device for a bionic hand, the bionic hand comprising a plurality of fingers, characterized in that, The control device for the bionic hand includes: The first control module is used to control the bionic hand to enter a semi-automatic playing mode when recognizing the notes being played and determining the musical score. The second control module is used to control the corresponding fingers to play according to the sheet music and the electromyographic signals on the surface of the residual arm in the semi-automatic playing mode. The first judgment module is used to determine whether the played notes meet the preset requirements; The first switching module is used to control the bionic hand to switch from the semi-automatic playing mode to the fully automatic playing mode when the notes played meet the preset requirements. The third control module is used to control the fingers to play according to the sheet music in the fully automatic playing mode; and The fourth control module is used to continue controlling the fingers to play in the semi-automatic playing mode when the notes played do not meet the preset requirements.

7. The control device for the bionic hand according to claim 6, characterized in that, The second control module includes: The acquisition module is used to acquire corresponding preset playing information based on the musical score, the preset playing information including bending angle, bending speed, and bending force; and The first sub-control module is used to control the corresponding fingers to play according to the preset playing information based on the electromyographic signals.

8. A control device for a bionic hand, characterized in that, The control device for the bionic hand includes: The first control module is used to control the bionic hand to enter a semi-automatic playing mode when recognizing the notes being played and determining the musical score. The second control module is used to control the corresponding fingers to play according to the sheet music and the electromyographic signals on the surface of the residual arm in the semi-automatic playing mode. The second judgment module is used to determine whether the played notes meet the preset requirements; The fifth control module is used to continue controlling the fingers to play in the semi-automatic playing mode when the played notes meet the preset requirements. The second switching module is used to control the bionic hand to switch from the semi-automatic playing mode to the normal playing mode when the played notes do not meet the preset requirements; and The sixth control module is used to control the fingers to play according to the electromyographic signals in the normal playing mode.

9. A bionic hand, characterized in that, The bionic hand includes a bionic hand body and a control device for the bionic hand as described in claim 6, 7 or 8. The bionic hand body includes a plurality of fingers, and the control device is disposed on the bionic hand body for driving the fingers to play.

Images