Neurologically impaired patient voice classification system with deep learning techniques

Abstract

The invention relates to a system for classifying the voice of a neurologically impaired patient by using deep learning techniques to determine whether the individual has a neurological disease by classifying the voice signals converted into spectrogram images using the deep learning method.

Description

[0001] NEUROLOGICALLY IMPAIRED PATIENT VOICE CLASSIFICATION SYSTEM WITH DEEP LEARNING TECHNIQUES

[0002] Technical Field Related to the Invention:

[0003] The invention relates to a system for classifying the voice of a neurologically impaired patient by using deep learning techniques to determine whether the individual has a neurological disease by classifying the voice signals converted into spectrogram images using deep learning techniques.

[0004] State of the Art:

[0005] People with neurological diseases often have voice disorders. In cases where the patient has a voice disorder, it is necessary to determine whether the patient has a neurological disease. Voice analysis is a study based on objective parameters. Voice signals of individuals with neurological diseases may be affected over time. This effect changes the speech of the patients. In order to examine this change in voice signals, these voice signals are converted into images called spectrograms in the computer environment.

[0006] Currently, there is no system that classifies patients with neurological diseases according to voice signals. Only the relevant specialist can diagnose diseases. Considering the difficulty of reaching the clinic, the workload of the specialists and all the problems that may be experienced in the clinic during this process, it is concluded that it is not always possible to reach objective results. The assessments made by the specialist concerned may differ depending on the person making the assessment and even over time. An objective assessment is essential for a good assessment of the risk of disease.

[0007] The invention subject to the application numbered "KR1936302B1", in the state of the art, comprises a method for diagnosing a degenerative neurological disease based on a deep learning network, the steps of converting user voice data into image data, inputting the image data into a learnt neural network and diagnosing a user based on the value output from the user. The invention subject to the application numbered "W02020013296A1 ", in the known state of the art, provides a medical apparatus for predicting a mental / neurological disease with high precision. This medical apparatus is provided with a computational processing device and a recording device on which a prediction programme executed by the computational processing device is recorded.

[0008] In the invention subject to the application numbered "WO2021250854", in the state of the art, an information processing device extracts a feature value, an acoustic parameter, from voice data. The information processing device produces a spectrogram image of the audio data. The information processing device calculates, on the basis of the feature value and a calculation model, a first score indicating the degree of a user's psychiatric-based disorder or neurological-based disorder or mental disorder symptom or cognitive dysfunction symptom.

[0009] The documents in the known state of the art evaluated the sounds of diagnosed neurodegenerative diseases. There is no system that analyses the voices compatible with each age range and gender by voice analysis in healthy people, and identifies the affected anatomical system and then the diagnostic process in all neurological diseases. Therefore, there is a need for a system that analyses the patient's voice recorded via a mobile application and determines which neurological disease the patient has through artificial intelligence. In addition, the present invention enables the follow-up of the prognostic processes of diagnosed individuals and the changes of the effects of the disease on the speech voice over time. Additionally, it is possible to determine the effect of the treatments applied to the diagnosed individuals on the voice signals.

[0010] Brief Description and Objectives of the Invention:

[0011] The invention relates to a system for classifying the voice of a neurologically impaired patient by using deep learning techniques to determine whether the individual has a neurological disease by classifying the voice signals converted into spectrogram images using deep learning techniques.

[0012] The most important aim of the invention is to analyse the voices of healthy people and the voices compatible with each age range and gender. A further object of the invention is to enable the identification of both healthy and voice-affecting diseases and the neuronal anatomical structures affected.

[0013] Another purpose of the invention is to obtain more objective results and provide early diagnosis thanks to the system created based on artificial intelligence.

[0014] Another aim of the invention is to provide a more comfortable diagnostic process by analysing voice recordings that can be obtained even from individuals' homes.

[0015] Another object of the invention is to provide remote monitoring for diagnosed individuals.

[0016] Another aim of the invention is to provide artificial intelligence-based analyses and recommendations on prognostic processes and drug regulations.

[0017] Description of Figures:

[0018] FIGURE-1 : Drawing giving a schematic view of the system subject to the invention.

[0019] Reference Numbers:

[0020] 110. Application

[0021] 120. Microfon

[0022] 130. Data set

[0023] 140. Server

[0024] Description of the Invention

[0025] The invention relates to a system for classifying the voice of a neurologically affected patient by using deep learning techniques to determine whether the individual has a neurological disease by classifying the voice signals converted into spectrogram images using deep learning techniques.

[0026] The application (110) comprises an electronic device interface allowing users to record voices with a microphone (120). The application (110) provides an interface for the labelling of voices in the data set (130) by doctors. Labelling is done with patient diagnoses. The microphone (120), located on the electronic device, enables users to record the voices.

[0027] The data set (130) was obtained by having patients diagnosed with MS and other neurological diseases (ALS, SCA, Alzheimer's, Epilepsy, Parkinson's, Myasthenia, Myelitis, Motor Aphasia, Psychological, Fiedreich's Ataxia, and Language Deficiencies) utter a common sentence. The preferred application of the invention contains audio files with .wav extension belonging to 99 individuals with MS and 96 individuals with other neurological diseases. The data set was formed by eliminating misleading data (data with confusion in voices or problems in recording) from the data obtained.

[0028] The server (140) extracts the spectrograms of the audio data in the data set by Fourier transform. The server (140) classifies the spectrogram images obtained with convolutional neural networks. Classifications are made according to neurological diseases and the brain region affected by these diseases. Spectrogram images are obtained by applying this process to each audio file. 80% of these images are used to create a classification model with a transfer-based deep learning technique. 20% of these images are used to test the accuracy of the model. The training and test data in the data set are randomly determined.

[0029] The server (140) receives the voice data of individuals who are thought to be healthy (individuals without any other diagnosis) entered from the application (110) and extracts spectrograms by Fourier transform of the voice data. The server (140) classifies the audio signals converted into spectrogram images using the deep learning method and determines whether the individual has a neurological disease. The server (140) provides artificial intelligence-based analyses and recommendations on the prognostic process and drug regulations. For disease detection, a gold standard is established by making disease diagnoses and labelling the affected area from previously recorded sounds. Recommendations are based on the changes realised with the treatment applied to individuals. Recommendations are presented through clinical evaluations and treatments for the body region in the spectrum of the affected central nervous system region. The server (140) identifies the neuronal anatomical structures primarily affected in all neurological diseases from spectrograms. When identifying the affected neuronal anatomical structures, the affected anatomical systems in the central nervous system (cortical, basal ganglia, brain stem, cranial nerve) or peripheral nervous system (neuromuscular junction, muscle) are determined in relation to the relevant neurological diagnosis of the patients. Over time, the prognostic significance of the natural course of the disease can be determined by determining the improvement and deterioration of the affected neurological systems from the voice recordings of the same patient.

Claims

CLAIMS1. It is a system that classifies the voice of neurologically impaired patients with deep learning techniques, characterised by the following:- at least one application (110) executed on an electronic device, providing an interface that allows users to record voices with a microphone (120) and allows doctors to label the voices in the data set (130),- at least one microphone (120) located on the electronic device, enabling users to record voices,- at least one dataset (130) containing audio files of individuals with neurological disorders,- at least one server (140) that extracts the spectrograms of the audio data in the data set (130) by Fourier transform, classifies the spectrogram images obtained with convolutional neural networks and determines whether the individual has a neurological disease by classifying the audio signals converted into spectrogram images using the deep learning method2. A system for classifying the voice of a neurologically impaired patient using deep learning techniques according to claim 1 , characterised in that it comprises a server (140) that provides artificial intelligence-based analyses and recommendations on prognostic process and medication regulations.

3. A system for classifying the voice of a neurologically impaired patient with deep learning techniques according to claim 1 , characterised in that it comprises a server (140) that detects neuronal anatomical structures primarily affected in all neurological diseases from spectrograms.

4. A system for classifying neurologically impaired patient voice with deep learning techniques according to claim 1 , characterised in that the neurological diseases are MS, ALS, SCA, Alzheimer's, Epilepsy, Parkinson's, Myasthenia, Myelitis, Motor Aphasia, Psychological, Fiedreich's Ataxia, and Language Deficiencies.

Images