Robotic hand controlled by brain signals (EEG)

Abstract

The invention relates to a robotic hand that can be controlled by brain signals (EEG), which operates with brain signals, which can make power, torque and position control thanks to motors that do not make a person feel the lack of a limb to meet the needs of a person who has lost a limb from any accident or birth, which can control finger pressure, and which can analyze the data under the thought of the individual and transfer it to the robotic hand.

Description

[0001] ROBOTIC HAND CONTROLLED BY BRAIN SIGNALS (EEG)

[0002] Technical Field

[0003] The invention relates to a robotic hand that can be controlled by brain signals (EEG), which are used as prosthetic hands, which can be controlled by brain signals, which are prosthetic hands that work with the muscular system.

[0004] Background of the Invention

[0005] Bionic Hand is not a well-established practice that has come from ancient times to the present day. In ancient times, there was not a structure that gave as much importance and offered different applications as today. Technology has developed and changed over time, creating its current structure. Technology can affect a wide area. In this way, artificial gloves such as bionic hands are used for people who have hand diseases and lost their hands. The development of bionic hands has advanced significantly in recent years as technology has advanced, and researchers are constantly working to improve their functionality, responsiveness and integration with the user's nervous system. These advances aim to provide greater independence, dexterity and quality of life for individuals with hand disabilities or limb loss.

[0006] The bionic hand used today is applied in accordance with the body based on its ability to grasp objects well, movement speed and grip sensitivity. The aim of the bionic hand is to ensure that patients do not have to live dependent on others, to be able to move freely individually and to prevent pediatric patients from becoming withdrawn due to physical deficiencies.

[0007] A bionic hand, also known as a robotic hand or prosthetic hand, is an advanced artificial limb designed to replace a missing hand or provide improved functionality to individuals with hand disabilities or limb loss. It is an example of bioengineering and combines mechanical, electronic and sensory components to replicate the functions and appearance of a natural hand. The bionic hand includes a frame or exoskeleton that mimics the anatomical structure of the human hand. It can have articulated fingers, knuckles and a palm, allowing natural movements and dexterity.

[0008] Prostheses are medical devices that are produced for individuals who have suffered amputation of their limbs such as hands, arms, feet or legs as a result of a disease or loss of function and replace these limbs and enable amputees to perform basic physical movements. Hand prosthesis studies, which have a more complex structure than other body protrusions, are given as an example of prosthetic studies. For the hand prostheses used today, three types of prostheses are mentioned. The first one is products that have almost no cosmetic function and only support the individual visually. Second one is body-controlled prostheses that are operated by more than one cable and these cables are connected to the individual's body. The biggest advantage of these prostheses is that there is no need for an electrical power source. The third one is myoelectric bionic hand prostheses.

[0009] The myoelectricbionic hand is an electro-mechanical system that combines mechanical, electronic and invasive or non-invasive EMG signals of the amputee. In such a system, mechanical parts are driven together with the control of electrical parts. The biggest advantages of these prostheses are that they have multiple degrees of freedom and can perform multiple movements at the same time. Examples of these are OttoBock, TouchBionics, DekaArm, Bebionic and similar products, which also have a commercial dimension. In the development of these prostheses, the processing of EMG signals plays a major role in providing hand movements.

[0010] Electromyography (EMG) is a method based on the recording of electrical activity that occurs in muscles and nerves as a result of muscle movements controlled by the nervous system. EMG signals are received from the body with the help of electrodes and processed through a voltage amplifier. By analyzing the signals obtained from the contraction and relaxation states of the muscles, it is possible to identify the desired movements with various classification algorithms. Brain Computer Interface (BCI) is a research field developed to implement and collect human brainwaves. This technology is based on senses that transmit, analyze and apply the language of neurons. BCI is one of the most exciting technologies to explore as it has a bright future for open communication, especially for people who have lost control over their limbs and have difficulty using their muscles to move their arms or hands. BCI is also known as brain machine interface (BMI) or sometimes known as direct neural interface. BCI is a direct communication pathway between the brain and an external device. There are four methods that record microvoltage-level extracellular potentials generated by neurons in cortical layers namely Electroencephalography (EEG), Electrocorticography (ECoG), local field potentials (LFPs) and single neuron action potential recording (single unit).

[0011] With the development of technology, especially in the field of computer and communication, the "Brain Computer Interface (BCI)" technology is gradually maturing. It starts by capturing small brain waves on the scalp until it records a specific area of brain activity and commands an external device such as a robot or prosthesis. Research and development on BCI is focusing not only on prosthetic hands but also on other applications such as controlling wheelchairs, humanoid robots and mobile robots.

[0012] Studies have shown the advantages of using EEG technology to offer new solutions in prosthetic devices. Typically, most researchers use Electroencephalography (EEG) for rapid communication because it is a simple and inexpensive equipment and does not require too many procedures. It is also more practical for patients and researchers. In addition, the surface of EEG does not require any surgery to take the neuron source from the human brain. EEG is preferred by many BCI researchers and is used in this study as a source of neuronal information. EEG uses non-invasive techniques as it records electrical activity from electrodes attached to the scalp with gel. This technology creates new alternatives without using muscle control (EMG). Both EEG and EMG technologies have limitations. EMG can be controlled by a person with strong active muscles, while EEG can be controlled using brainwaves without using active muscles. A complex BCI system allows users to control a system other than possibly a prosthetic device by generating the output of a specific EEG frequency. The electroencephalogram (EEG) records a signal by placing electrodes on the scalp with a conductive gel. The electrodes measure voltage differences across the skin in the microvolt (pV) range. The first electrical activity was recorded by Hans Berger in 1924. After analyzing the EEG signal, Berger identified the first brain wave called Berger, also known as the Alpha wave (8-12Hz), which has a very low signal.

[0013] The results of several years of research are promising with solutions with BCI applications such as BCI systems for environmental control, spelling systems, interfaces for robotic wheelchair steering, gaming. In the existing literature, several studies have been published reporting the results of the evaluation of the Emotiv EPOC+ device and its application. For example, the EPOC+ device was used to design and develop a BCI-based system for the generation of synthesized speech that works on eye blinks detected from EEG signals.

[0014] The EPOC+ device was also used to develop a speech interaction system for parallelized humans by integrating the BCI device to associate brain signals with commands to control wheelchairs. Research conducted by Vourvopoulos and Liarokapis has shown that commercial BCIs such as EPOC+ can be used for effective and natural robot navigation in both real and virtual environments.

[0015] In the field of video games, the EPOC+ device can be used to assess the user experience and cognitive processes induced by various stimulus modalities and game events when producing video games.

[0016] The working principle of EEG is simple; it is used to collect the signals or potential voltage produced by the brain, but the disadvantage is that the potential voltage produced is small and measured in microvolts (pV). Therefore, it requires accurate and sensitive equipment for measurement. With advanced technology, EEG has become more advanced and is widely used, especially in neurology, physiology and psychology. Therefore, there are various tools on the market such as NeuroSkyMindset, QuickAmp, G.tecAmplifier, Enobio, gO.MOBIIab and Tracklt system. Emotiv EEG Headset, a wireless EEG device, was developed in 2008. It is a low-cost device for recording EEG from the human brain. The Emotiv EEG Headset reference electrodes, which come preconfigured with 14 electrodes located at 10-20 International System positions with AF3, F7, F3, FC5, T7, P7, 01 , 02, P8, T8, FC6, F4, F8 and AF4, provide good coverage of the pre-motor and anterior regions of the brain, which is the sole interest of this study. A clinical EEG headset provides better coverage of the motor cortex, but its cost, long setup time and lack of portability prevent this approach from being a practical commercial solution. The headset aligns; band-pass filters and digitizes the signal at 128 Hz before transmitting wirelessly to the laptop.

[0017] There are many robotic hands in the current art. These existing robot hands work with the Muscle (EMG) system. This system works with old technology and it is necessary to work with new technology. Prostheses that work with the classical muscle system that exists today are becoming inefficient with the developing technology. Due to the weakening of the muscles in some disabled individuals, robot hands working with the muscular system may not be suitable for these people.

[0018] Bionic hands that can be controlled by the muscular system are widely available. They are also known as myoelectric prostheses. These prostheses work with the individual's muscular system. But in some individuals, the muscular system is lazy and cannot be used. For this reason, in some people, muscle data is taken from the back behind the shoulder, and in some people, prosthetic arms are produced mechanically since the muscles are not used at all.

[0019] The invention works with brain signals. Thus, it can be used more cost-effectively in every individual. A robot hand controlled by brain signals is made. After analyzing this data with the data we receive from the brain, it can be transferred to the robot hand. Thus, it can be used in every disabled individual.

[0020] For an optimal solution, the muscular system (EMG) is preferred as movement control. However, with today's technology, the EMG system has been left behind and has become a control method that is not used in individuals with lazy muscles. With the invention, the control method with brain waves (EEG) is now used. A prosthetic hand was made using the brainwave control method. The most important feature of the bionic hand of the invention that distinguishes it from other projects is that it is controlled by brain signals, sensitive grip, innovative and original design. However, in addition to design, functionality was also given importance. In this direction, a bionic robot hand was designed to meet the needs of a person who has lost a limb due to an accident or birth. In the design, it is aimed to be useful that will be needed in daily life and will not interfere with the operations to be performed. After analyzing and processing the electrical activity coming from the electrodes connected to the scalp with gel, the robot hand can be controlled by machine learning method by classifying it.

[0021] The invention is designed in a completely original way. DC motors are used in the working mechanism of the systems. Thanks to these motors, power, torque and position control can be performed. There is a system control card that will control the system of the motors with the data we will receive from the brain. The current sensors to be used in the system control card will be able to control the finger pressure without connecting any sensors to the fingertips. In other words, it will be able to hold an egg or a cardboard cup without crushing I breaking.

[0022] The robotic hand, which can be controlled by brain signals, receives signals from the brain with an electroencephalography (EEG) sensor. The signals from the brain are processed with a certain algorithm. The processed data is transferred to the robot hand and takes action. Thus, the data under the thought of the individual is analyzed and transferred to the robot hand and the robot hand movement is realized with brain signals.

[0023] The structural and characteristic features and all the advantages of the invention will be more clearly understood by means of the figures and the detailed description given below, and therefore the evaluation should be made by taking these figures and the detailed description into consideration.

[0024] Auxiliary Figures for the Description of the Invention

[0025] Figure 1: Side perspective view of the wrist and forearm design of a robotic hand controlled by brain signals (EEG) Figure 2: Side perspective view of the index finger sample design of the robotic hand controlled by brain signals (EEG)

[0026] Figure 3: General view of the sample design of all fingers and palm of the robotic hand controlled by brain signals (EEG)

[0027] Figure 4: Profile and top view of the arrangement of electrodes according to the robotic hand system controlled by brain signals (EEG)

[0028] Figure 5: General view of sample EEG recordings taken from different channels through robotic hand surface electrodes controlled by brain signals (EEG)

[0029] Figure 6: General view of the robotic hand controlled by brain signals (EEG), showing the Emotiv Epoc device electrode positions with green channels and red channels

[0030] Figure 7: Schematic view of the general approach to analyzing EEG data on hand function data of a robotic hand controlled by brain signals (EEG)

[0031] Figure 8: Schematic view of the operation of the robotic hand controlled by brain signals (EEG)

[0032] Auxiliary References for the Description of the Invention

[0033] 1. The idea of moving the hand

[0034] 2. Making wireless analog signals

[0035] 3. Recording of EEG signals

[0036] 4. Pre-processing

[0037] 5. Attributes

[0038] Description of the Invention

[0039] In the process steps of the method of obtaining a robotic hand that can be controlled by brain signals (EEG); the finger skeleton system will be printed from the 3D printer in the first stage shown in Figure 2. As a result of the experiment on a single finger, sensitive opening-closing, grasping movements are performed smoothly.

[0040] Precise closing-opening, grasping movement operations on one finger will be applied to all fingers. And by using one motor for each finger but two motors for the thumb, the thumb will move both on the X-axis and on the Y-axis, coinciding in parallel with the index finger, and thus, it is designed by paying attention to the mounting parts of the palm motors so that it can easily hold thin and small materials with the thumb and index finger. The size and torque criteria of the DC micro motors to be used are the most important issue, and the motors with a ratio of 298:1 that we use provide 4Kg.cm torque for each finger in terms of torque. Due to its small size in terms of size, it is designed to mount 6 motors in the palm of the hand.

[0041] After the previous stages are completed, the wrist joint shown in Figure 3 is started. The wrist joint has the ability to move on a single axis. With the servo motor and reducer hidden in the wrist, it will provide roll (rotation) movement on the Z axis and precise movement capability is provided as in the 2nd stages.

[0042] After the wrist joint is completed in a healthy and successful way, the forearm part is started. As seen in Figure 1 , when designing the forearm, the electronic card will be included in the forearm section. Since there is no need for any external energy, the necessary energy will be met with the internal Li-ion batteries on it. For this reason, since the electronic PCB (printed circuit board) cards will be placed in this area, the electronic PCB will be designed by considering the dimensions of the cards.

[0043] Elbow joint and biceps will be designed and produced. Since there will be electronic components in the forearm section, the elbow motor must be in the biceps area. Therefore, the elbow joint and biceps were designed with the elbow motor hidden in the biceps.

[0044] At the mechanical stage, a demo manual control system will be installed via computer. With the established demo system, mechanical openness and problem control will be performed by applying the opening-closing, grasping movements of the fingers, wrist on the roll (rotation) axis and elbow opening-closing operations. If a mechanical problem is encountered, a solution will be produced and developed.

[0045] At this stage, data will be read from Electroencephalography (EEG). EEG is a medical imaging technique based on the reading of changes in electrical activity produced by the brain through the skull. Electrical current changes collected and recorded from the skull surface with the help of electrodes and a conductive medium are defined as EEG recordings. Surface electrodes are placed on the skull according to the international standard 10-20 system. In this system, the electrodes are arranged between the nose and the back of the head as 10 - 20 - 20 - 20 - 20 - 10%. The arrangement of the electrodes is shown in Figure 4. The measurements taken from the electrodes are brought to the microvolt (pV) level with the help of amplifier circuits and become readable. The signals are recorded in a digital environment with an analog-to-digital (A / D) converter. A sample recording taken from the human head is shown in Figure 5. The vertical axis shows the magnitude of the signal at the pV level and the horizontal axis shows the time information.

[0046] In the invention, Emotiv Epoc x EEG (Electroencephalography) device is used. It is shown in Figure 6. Emotiv Epoc is a portable, cheap, consumer level EEG device with 16 channels, 2 of which are reference, battery operated, transferring the signals to the computer via bluetooth communication method. In Figure 8, the EEG sensor is attached to the head of the individual and raw brain signals are received. Data is transferred from our device to the computer via bluetooth connection. It analyzes this data according to the person's thought state of controlling his / her hand and the data is analyzed by machine learning method. The processed data is then classified and transferred to our control card. Thus, when similar data comes from the person with the classified data, the control card detects it and sends commands to the output units. The output units are fingers, wrist, elbow and shoulder.

[0047] The robotic hand, which can be controlled by brain signals, receives signals from the brain with an electroencephalography (EEG) sensor. The signals from the brain are processed with a certain algorithm. The processed data is transferred to the robot hand and it takes action. Thus, the data under the thought of the individual is analyzed and transferred to the robot hand and the robot hand movement is realized with brain signals.

[0048] The general working principle of the device is shown in Figure 7. First, the idea of moving the hand (1 ) is made of wireless analog signals (2). EEG signals are recorded (3). The EEG data of the functions are attributed and separated from noise and the EEG sensor sends hamdelta, theta, alpha and beta bands for analysis. To facilitate the interpretation of the EEG data and to reduce the dimension space, features (5) are used. Thus, only the data from the electrodes in the desired regions of the brain are used. Finally, the obtained features (5) are classified by various machine learning algorithms. The classified data are defined to the control card as functions. When the person sends thought data similar to the classified data, the control card detects this and controls the output units. The output units are fingers, wrist, elbow and shoulder.

[0049] There is a system control card that will control the system of the motors with the data we will receive from the brain. The current sensors to be used in the system control card will be able to control the finger pressure without connecting any sensors to the fingertips.

[0050] After the processing of the data received from the EEG is finished, the functions of the EEG will be assigned.

[0051] The invention aims to reach the appropriate level of visual comfort both physiologically and psychologically. In this direction, visual beauty, aesthetic appearance has been developed because it has a great impact on human psychology, self-confidence and socialization. In response to these, there is a sheath on the device that gives an aesthetic appearance. The case is made using the casting method. The mold for the casting method will be manufactured from a 3D printer.

[0052] The technical and all other features mentioned in each claim are followed by a reference number, which reference number is used only to facilitate understanding of the claims and therefore should not be considered as limiting the scope of the element indicated by this reference number for illustrative purposes.

[0053] It is clear that a person skilled in the art can also demonstrate the innovation in the invention by using similar embodiments and / or apply this embodiment to other areas with similar purposes used in the relevant art. Therefore, it is also obvious that such embodiments will lack the criterion of exceeding the novelty.

Claims

CLAIMS1. A robotic hand that can be controlled by brain signals (EEG), wherein it comprises single finger, all fingers, wrist, forearm, wrist joint, elbow, sheath and biceps sections using casting method giving aesthetic appearance.

2. A robotic hand that can be controlled by brain signals (EEG) according to Claim 1 , wherein it comprises 6 DC micro palm motors with a ratio of 298:1 , one motor for each finger, but two motors for the thumb for both X-axis and Y- axis movement of the thumb, which are applied to all fingers for precise closing-opening, grasping movement on a single finger.

3. A robotic hand that can be controlled by brain signals (EEG) according to Claim 1 , wherein it comprises a wrist joint which is capable of movement on a single axis and which provides roll (rotation) movement on the Z-axis with a servo motor and reducer hidden in the wrist.

4. A robotic hand that can be controlled by brain signals (EEG) according to Claim 1 , wherein it comprises electronic PCB (printed circuit board) cards that are powered by internal Li-ion batteries placed in the forearm.

5. A robotic hand that can be controlled by brain signals (EEG) according to Claim 1 , wherein it comprises electronic components inside the forearm section and an elbow motor in the biceps area.

6. A robotic hand that can be controlled by brain signals (EEG) according to Claim 1 , wherein it comprises electrodes that are arranged between the nose and the back of the head as 10 - 20 - 20 - 20 - 20 - 10 % and are brought to the microvolt (pV) level with the help of amplifying circuits and that become readable, and a robotic handheld electroencephalography (EEG) sensor that can be controlled by brain signals processed by electrodes and a certain algorithm.

7. A robotic hand that can be controlled by brain signals (EEG) according to Claim 1 or 6, wherein it comprises Emotiv Epoc x EEG (Electroencephalography) device, which is portable, has 16 channels, 2 of which are reference, battery operated, and transfers the signals to the computer via bluetooth communication method.

8. The operation of robotic hand that can be controlled by brain signals (EEG), and characterized in that it comprises the following process steps:• The idea of moving the hand (1 ) making wireless analog signals (2),• Receiving raw brain signals by attaching the EEG sensor to the individual's head,• Transfer of data to the computer via bluetooth connection,• Recording of EEG signals (3),• EEG sensor sends raw delta, theta, alpha and beta bands so that the EEG data of the functions can be attributed, noise separated and analyzed,• Utilizing attributes (5) to reduce the dimension space and facilitate the interpretation of EEG data to work with data from electrodes in only the desired regions of the brain,• Classification of the obtained attributes (5) with various machine learning algorithms,• Defining the classified data as functions on the control card,• When the person sends thought data that is similar to the data that is instantaneously classified, the control card detects this and controls the output units (fingers, wrist, elbow and shoulder),• Ensuring the control of the system of motors with the data received from the brain.

Images