Computer-implemented method for controlling the operation of a robotic prothesis

Abstract

The present invention relates to a computer-implemented method for controlling the operation of a robotic prothesis. The method is carried out by receiving and processing EMG and EEG signals received from a user and sending said signals to a nanocomputer, wherein artificial intelligence and specific algorithms interpret the received signals and activate different servomotors, thus allowing the robotic prothesis to move. Furthermore, additional sensors for protecting the user, an inertial measurement unit (IMU), a virtual assistant and a failure diagnosis and loading system are included. The method is managed via a computer program that is loaded into the nanocomputer.

Description

[0001] COMPUTER-IMPLEMENTED METHOD FOR CONTROLLING THE OPERATION OF A ROBOTIC PROSTHESIS

[0002] A computer-implemented method for controlling the operation of a robotic prosthesis using an algorithm. The method allows a subject to control the prosthesis using electrical impulses captured by EMG and EEG sensors. The information obtained is processed by artificial intelligence, emulating the functions of an upper limb.

[0003] BACKGROUND OF THE INVENTION

[0004] When conducting the research, several documents related to the present invention were identified.

[0005] Document MX2017009980A describes an articulated arm prosthesis mechanism controlled by toe movements, using gears and pulleys. However, this invention is complex, as each toe must perform a different action, which involves a considerable learning curve, and for people without lower limbs (this system could not be controlled). In contrast, the present invention allows operation through brain signals and does not depend on specific movements of other limbs.

[0006] For its part, patent document CN103892945B advocates a myographic joint system, without EMG sensors, however, this invention could not work if the patient does not have an arm, it also lacks safety sensors for the protection of the user and the prosthesis, and requires constant maintenance due to its gear system. The present invention overcomes these problems by integrating protection and self-diagnosis sensors that alert the user to failures and allow adjustments without the need for equipment.

[0007] Document US10980466B2 discusses a system based on EEG sensors to control a prosthesis, but it does not consider possible errors in the precision of patient movements. This is because it uses only one type of sensor and sweat, when in contact with the skin for two hours, tends to generate erroneous signals captured from delta and alpha waves. However, the computer-implemented method that is intended to be protected controls the operation of a robotic prosthesis because it uses not only EEG sensors but also EMG sensors, in addition to a complete list of anti-hostile environment sensors for the well-being of both the patient and a set of algorithms and robotic system.

[0008] DETAILED DESCRIPTION OF THE INVENTION

[0009] The computer-implemented method revealed in the present invention, for controlling the operation of a robotic prosthesis, seeks to reintegrate amputee citizens into the workplace and society.

[0010] Furthermore, the computer-implemented method for controlling the operation of a robotic prosthesis controls said prosthesis using EMG and EEG bioelectrical signals from a subject.

[0011] First, an initial configuration is performed using a computer and a customized algorithm based on the user's profile. This algorithm is then loaded into a nanocomputer located inside the robotic prosthesis. EMG and EEG sensors placed on the user send signals to the nanocomputer via dedicated communication channels. At this stage, artificial intelligence algorithms are used to process these signals, interpreting the user's movement intentions. This allows servomotors in the prosthesis to be activated to perform the desired actions. Furthermore, the method can integrate various additional sensors, for example, temperature, humidity, and heart rate sensors, as well as an inertial measurement unit (IMU), which aids in the precise detection of position and movement.A virtual assistant can also be available to interact with the user and facilitate control of the robotic prosthesis, while an internal diagnostic area allows for monitoring the prosthesis's status and energy level. The entire method is managed through a specially designed computer program loaded onto the nanocomputer, which coordinates and executes each function to provide an optimized and adaptable user experience for each subject.

[0012] Specifically, the computer-implemented method for controlling the operation of a robotic prosthesis, which is the subject of protection in this document, basically consists of the following steps:

[0013] (a) perform initial algorithm configuration on a computer according to user;

[0014] (b) load the algorithm obtained in step (a) into the nano-computer;

[0015] (c) receiving EMG and EEG signals from sensors placed on a user;

[0016] (d) process the signals from step (c) with algorithms and artificial intelligence; and

[0017] (e) The processed step (d) signals activate the servo motors.

[0018] In another embodiment, the present invention provides a computer-implemented method for controlling the operation of a robotic prosthesis, wherein only EEG signals from sensors placed on a user can be received.

[0019] Preferably, the computer-implemented method of the present invention involves additional signals received from sensors such as humidity, temperature, heart rate, and master sensors. Furthermore, an inertial measurement unit (IMU) may be included when implementing the computer-implemented method.

[0020] Another preferred embodiment of the computer-implemented method for controlling the operation of a robotic prosthesis comprises including a virtual assistant, a fault diagnosis system and energy charging.

[0021] It is important to note that the execution of the method for controlling the operation of a robotic prosthesis involves a computer program, which is loaded into the nanocomputer. This nanocomputer also controls the devices detailed in the disclosed method.

[0022] Likewise, it is important to emphasize that before introducing the algorithm into the nanocomputer, it is necessary to know who will be using the robotic prosthesis, as each algorithm must be programmed and adapted to the user who will be using it. Therefore, each algorithm is unique and will differ for an adult, a young adult, or a child. Furthermore, the algorithm is programmed in editable programming code, but once introduced into the nanocomputer, it cannot be modified.

[0023] BRIEF DESCRIPTION OF THE FIGURES

[0024] Fig. 1: Front-top three-dimensional view of the robotic prosthesis.

[0025] Fig. 2: EMG signal operation graph.

[0026] Fig. 3: Explanatory diagram of the internal and external components of the nanocomputer. Fig. 4: Diagram of the method for controlling the operation of a robotic prosthesis.

[0027] Fig. 5: Flowchart of the computer program.

[0028] Fig. 6: Location of EMG and EEG sensors on a user.

[0029] Fig. 7: Brain waves identified in an electroencephalogram.

[0030] Figure 1 shows the robotic prosthesis from a top-front three-dimensional view, displaying at least one EMG sensor (1) that captures bioelectrical signals from the user's muscles. The robotic prosthesis has a lower and upper housing (2). The servomotor area (5) is also protected by a housing (7). All the internal components described above that are part of the robotic prosthesis are embedded within a silicone protector (6) in the shape of a hand, which is very similar to human skin and is achieved by scanning the user's existing hand with a 3D scanner to obtain a hand similar to the real one. Below the silicone hand are the fingers and joints printed on a 3D printer. The diagnostic screen (3) and an electrical branch (4) can also be seen.Furthermore, in Figure 1 there is located at least one rechargeable battery (8), and also a solar panel (9), the latter as an alternative energy supply. This entire set of devices constitutes the housing of the robotic prosthesis, and are responsible for the operation of said prosthesis.

[0031] Figure 2 shows the EMG signal performance graph of the computer-implemented method for controlling the operation of a robotic prosthesis. The y-axis of the Cartesian plane shows a maximum threshold of 0.8, and the x-axis of the Cartesian plane shows the different threshold activation times. The algorithm loaded into the nanocomputer determines, using artificial intelligence, which signals are useful and which are not. For example, if the threshold reaches 0.2, the servomotor is activated with a 100-degree rotation. The graph shows the different thresholds (0.2, 0.4, and 0.6) depending on the EMG signal received; the last signal observed corresponds to the gyroscope.

[0032] Figure 3 comprises an explanatory diagram of the internal and external components of the nano-computer (15), which comprises the main master sensor area (11), in addition to including EMG electromyography sensors, EEG electroencephalogram sensors and an ECG heart rate sensor. Also, the wireless connection area (10) can be seen, which receives signals from the EEG sensor. The wireless connection area also includes abilities to control smart devices, such as televisions, Smartphones, among others. In the nano-computer (15) the inertial measurement unit or IMU (13) can be displayed, said IMU can be transformed to add a virtual assistant, microphone and speaker. In addition, the IMU (13) measures and reports on the speed, orientation, gravitational forces of the robotic prosthesis.The IMU (13) may be directly connected to a wireless connection or a wired connection, and smart devices may be controlled by user gestures. For example, to move a slide on a computer via Bluetooth, the robotic prosthesis may simply be linked to a computer or other device that has a wireless connection. The robotic prosthesis uses the gyroscope and accelerometers of the IMU (13) to rotate the prosthesis or to determine whether the prosthesis is moving forward or backward, or is rotating. The inertial measurement unit or IMU (13) may have a speaker and a microphone connected to it so that the prosthesis user can communicate with the virtual assistant. In addition, the ECG sensor is observed, which has the function of detecting the number of heartbeats per minute of the user.For example, the normal range of heart rate is 120 rpm, but if the ECG sensor detects that the heart rate has dropped to 100 rpm and then drops further to 60 rpm, or in the case that the heart rate is higher than 140 rpm, it is determined that the person is minutes away from presenting a cardiac arrest, and the horn is turned on to alert people who are near the user, who is wearing the robotic prosthesis, that he needs urgent medical attention. If the user's heart rate continues to drop, a signal can be sent through the nano-computer (15) located in the robotic prosthesis to a cell phone of a relative of the user, but it is necessary to previously register said emergency phone number within the algorithm.Thanks to the fact that the IMU (13) is transformed, it has the ability to connect to the Internet, and can be linked to a virtual assistant, and also thanks to the fact that a microphone and a speaker can be integrated into the IMU (13), these last devices will be activated by recognizing the voice of the user of the robotic prosthesis, thus the user can make an Internet query or play music, which is possible by virtue of the internal memory of the nano-computer. In addition, the user of the robotic prosthesis can control the movements of said prosthesis by means of voice and make basic configurations. For example, if the user wants to see something on the diagnostic screen (3), such as the status of the algorithm or watch a video or video game, the user must press "play" on said screen. Also, the nano-computer has nano microcontrollers, for example the nano 33ble sense, and with it can execute one of the aforementioned tasks.In addition, in the area of ​​anti-hostile environment sensors (12) more sensors can be found such as "IR flame" that detects temperatures, and a humidity sensor, which are other master sensors (11) and that provide security to the user of the robotic prosthesis.

[0033] Figure 4 shows a diagram of the method for controlling the operation of a robotic prosthesis. The diagram shows different areas, among which the sensors and connectors area (14), the nano-computer area (15), the artificial intelligence area (16), the EMG threshold signals and EEG waves area (17), and the servomotor activation area (18) can be mentioned. In the first sensors and connectors area (14), signals are constantly received from all the sensors, that is, for example, signals are received from the EMG, EEG and ECG sensors placed on the user, and said signals are then passed on to the nano-computer area (15) for diagnosis. The nano-computer area (15) also receives signals from the wireless connection area (10), for example EEG signals. The nano-computer area (15) sends the preprocessed signals to the artificial intelligence area (16) to process and classify them.Subsequently, when these signals are classified, they pass to the EMG threshold area and EEG waves (17), and the corresponding movement is carried out in the servomotors (18). If, for example, the user wants to open and close the fingers of the robotic prosthesis, and needs to hold a glass, the threshold for each finger will be 0.2 and each servomotor (18) will be activated with a rotation of 100 degrees.

[0034] Figure 5 reveals a flowchart of the computer program that executes the computer-implemented method for controlling the operation of a robotic prosthesis. The flow is carried out as follows: the sensors collect readings (19), which are sent to the nanocomputer (20), the latter searches for matches by comparing the signals received from the sensors (19) and when it finds a match it sends it to the artificial intelligence area (21), which analyzes and classifies in more detail the signals pre-processed by the nano-computer (20). Subsequently, the artificial intelligence area (21) classifies based on the 4 programmed options (22) which option it should take.For example, if "option 1" is chosen, for this purpose, EMG threshold 1 or EEG wave 1 is activated, and an action 1 is executed in the servomotors (23), which is visualized by moving the robotic prosthesis to 100 degrees of rotation, this is to be able to hold large objects such as notebooks or apples. If, on the other hand, "option 4" is chosen, EMG threshold 4 or EEG wave 4 is activated, and an action 4 is executed in the servomotors (23), resulting in a 300 degree movement, thus causing the thumb and little finger to touch each other, this option is suitable for holding small objects such as a pencil or a page with the robotic prosthesis.

[0035] Figure 6 shows the location of the EMG (1) and EEG (24) sensors in a user, which send signals to the nano-computer to control the operation of the robotic prosthesis.

[0036] Figure 7 shows the brain waves identified in an electroencephalogram. The following wave types are displayed: delta, theta, alpha, and beta, which normally occur in different sleep states, that is, when the user is asleep. However, in the present invention, it has been discovered that delta, theta, alpha, and beta waves can also be replicated when the user is awake.

[0037] One advantage of the computer-implemented method for controlling the operation of a robotic prosthesis is that the EMG and EEG sensors work together to provide more precise movement of the robotic prosthesis, but they can also operate independently. For example, if the user has a partial upper limb missing, both the EMG and EEG sensors are connected. Conversely, if a user is missing a complete upper limb, the EMG sensor cannot be connected, but the EEG sensor will be connected and send brainwave signals to the nanocomputer. Another advantage is that if the user walks for three to five hours, they may sweat, and the EMG sensor may receive inaccurate signals. However, by having two sensors, the EMG and EEG sensors, the EEG sensor validates the received signal and the prosthesis functions correctly.It is important to note that if any sensor fails, surgery is not required to replace it, as is done in other existing technologies.

Claims

CLAIMS 1. Computer-implemented method for controlling the operation of a robotic prosthesis characterized by the following steps: (a) perform initial algorithm configuration on a computer according to user; (b) load the algorithm obtained in step (a) into the nano-computer; (c) receiving EMG and EEG signals from sensors placed on a user; (d) process the signals from step (c) with algorithms and artificial intelligence; and (e) The processed step (d) signals activate the servo motors.

2. Computer-implemented method for controlling the operation of a robotic prosthesis according to claim 1, characterized in that in step (c) only EEG signals from sensors placed on a user can be received.

3. Computer-implemented method for controlling the operation of a robotic prosthesis according to claim 1, characterized in that it also comprises humidity, temperature, heart rate sensors, and master sensors.

4. Computer-implemented method for controlling the operation of a robotic prosthesis according to claim 1, characterized in that it comprises an inertial measurement unit (IMU).

5. Computer-implemented method for controlling the operation of a robotic prosthesis according to claim 1, characterized in that it comprises a virtual assistant.

6. Computer-implemented method for controlling the operation of a robotic prosthesis according to claim 1, characterized in that it comprises a fault diagnosis and energy loading system.

7. Computer program characterized in that it executes the method of claim 1, wherein said computer program is loaded into the nano-computer.

8. Computer program according to claim 6, characterized in that said nanocomputer controls the devices detailed in claims 2-5.

Images