MULTIMODAL ARTIFICIAL INTELLIGENCE SYSTEM AND METHOD FOR EARLY DIAGNOSIS OF IDIOPATHIC PARKINSON'S DISEASE

Abstract

The invention encompasses a multimodal artificial intelligence-based clinical decision support system and its operating methodology, which enables the simultaneous processing of visual, auditory, and kinematic data obtained from four different clinical modalities—gait, facial expression, voice, and posture—for the early detection of idiopathic Parkinson's disease.

Description

1 TARIFF MULTIMODAL ARTIFICIAL INTELLIGENCE FOR EARLY DIAGNOSIS OF IDIOPATHIC PARKINSON'S DISEASE. SYSTEM AND METHOD TECHNICAL AREA 5 The invention aims to detect idiopathic Parkinson's disease in its early stages by analyzing gait and facial features. This expression is derived from visual, auditory, and posture data obtained from four different clinical modalities. A multimodal AI-based clinic that enables the simultaneous processing of kinematic data. It includes the decision support system and how this system works. STATE OF THE ART Studies have shown that efforts are being made to detect idiopathic Parkinson's disease in its early stages. Current practices largely rely on clinical observation, the experience of the specialist neurologist, and scale. It appears to be based on subjective scoring systems. Combined Parkinson's Disease Clinical scales such as the Assessment Scale are widely used, and they measure motor symptoms from 15 to 15. The severity is assessed by the physician through observational scoring. These subjective scores are based on: It varies depending on the observer and consistency is lacking due to vague symptoms in the early stages. This can create problems. However, digital approaches developed in the literature and clinical practice are mostly... focusing on a single data source or biomarker, and only performing gait analysis. image processing-based systems, acoustic models that analyze only sound recordings, or 20 Deep learning-based solutions that evaluate only facial expressions are available separately. Furthermore, A significant portion of current techniques include magnetic resonance imaging and positron emission. CT scans, optical coherence tomography, or polysomnography are expensive and not always performed in every clinical setting. some rely on imaging devices that are not available, while others use blood, saliva, and tears. Invasive or 25 requiring the analysis of biological samples such as plasma, exosomes, or genetic material. These systems consist of semi-invasive biochemical tests. All of these systems are generally single-module. It provides an assessment and makes a decision that considers different motor and vocal parameters together. It does not involve a mechanism. As a result, current techniques have low sensitivity and low accuracy. generalizability, high cost, need for invasive procedures, or the need for specialized equipment, etc. They work with disadvantages and their capacity to support clinical decision-making processes remains limited. 30 The most fundamental technical challenge in the clinical diagnosis of Parkinson's disease is the difference in symptoms between individuals. It exhibits a heterogeneous distribution. Some patients show significant gait disturbances. There may be no significant changes in facial expression and voice parameters. However, in some patients... Even with pronounced hypophonia or articulation disorder, gait and posture may remain normal. Similarly... In individuals showing decreased facial muscle movement, other motor symptoms appear early, around 35. 2 This may not be the case. This situation reduces the diagnostic sensitivity of systems based on a single biomarker. This limits and complicates clinical evaluation. In gait analysis, the current These systems are generally based on linear gait recordings along a straight line, and the individual It cannot simultaneously assess changes in direction, turns, and natural movements. In facial analysis, Assessments are made based on static facial images or a limited number of facial movements, 5 Changes in dynamic facial muscle activity over time cannot be quantitatively modeled. [Sound] In the analysis, free speech or unstructured audio recordings are used, and acoustic Comparability and standardization of parameters cannot be ensured in posture analysis. However, mostly static posture assessments are performed, while dynamic postural assessments during movement are also conducted. The numerical representation of changes and angular relationships between body regions is sufficient. 10 It has not been improved. Furthermore, the current techniques also utilize data obtained from different evaluation methods. a standardized data collection protocol and physical system that enables the simultaneous collection of data The infrastructure is not available. Therefore, the data was collected at different times and under different conditions. Analyses conducted on it lead to problems of intermodal compatibility and comparability. It opens. 15 All these limitations in current techniques lead to the development of Parkinson's disease in its early stages. a combination of scattered, low-intensity motor and vocal symptoms that vary among individuals, This results in the inability to analyze simultaneously and holistically. Consequently, Current techniques can manifest in different modalities such as gait, facial expression, voice, and posture. Because it cannot analyze variables and early symptoms together, a combined, standardized and objective 20 It is technically inadequate in producing a risk assessment. This invention addresses this... the technical problem involves any biological sample collection, expensive imaging equipment, or A specially designed L-shaped walkway that does not require complex laboratory infrastructure. Simultaneous and structured data collection workflow from four different modalities. By enabling standardized data acquisition, separate feature extraction and modeling for each modality. 25 through the development of modules and ultimately the probabilistic outputs obtained from all modalities a multimodal artificial intelligence system based on the principle of combining them with a late fusion mechanism It solves this by offering this approach. Thanks to this approach, the heterogeneous clinical nature of Parkinson's disease is addressed. Accordingly, each symptom group can be assessed independently, and symptoms can be evaluated within a single modality. Even in patients exhibiting these behaviors, analysis outputs can be generated through the relevant module, thus utilizing the existing 30 compared to single-module systems requiring expensive imaging equipment or involving invasive procedures Higher accuracy, higher generalizability, lower error rate, lower cost, Advantages such as being completely non-invasive and radiation-free can be achieved. Patent document number WO2025116115A1 describes an artificial intelligence system and how to treat Parkinson's disease in 35 years. The text describes a diagnostic method. The invention involves a joint model used to diagnose the target human being. 3 Obtaining joint point information of the body, capturing a target image and identifying the joint point. Correcting the data to correspond to the body size in the target image, multiple Corrected articulation point information in response to movement in the target image throughout the frame. by calculating motion vectors and a combination of these motion vectors This diagnosis is predicted along with the estimation of a Parkinson's disease diagnostic score. This system includes steps for assessing the presence of Parkinson's disease based on a score. It uses motion vectors based solely on skeletal joint points, and gait It does not include other clinical modalities such as facial expression, voice, and posture. In the present invention... In addition to gait, expression analysis based on facial muscle movements, structured speech Analysis of acoustic properties obtained from their functions and angular relationships between body regions 10 Four different modalities can be modeled simultaneously using numerical parameters. These modalities are evaluated, and the probabilistic outcomes are determined using a decision fusion mechanism. They are being combined. Patent document number US11013452B1 describes an AI-based Parkinson's disease diagnosis. The device and method are discussed. The invention involves magnetic resonance imaging of a patient's brain. Obtaining the first image by imaging with a device, Parkinson's Substantia nigra and nigrosome-1 region used as imaging biomarkers for the disease. The first image taken so that it can be observed is subjected to a quantitative sensitivity mapping algorithm. Based on this processing, a second 20-inch image containing the nigrosome-1 region was created from this first processed image. classification of the image and whether the detected nigrosome-1 region is normal. diagnosing whether a patient has Parkinson's disease through analysis This approach includes magnetic resonance imaging, which is expensive and not available in every center. It is based on the visual evaluation of a neuroanatomical biomarker using devices. The current invention, however, allows for the capture of video without the need for any expensive imaging equipment. It operates with basic equipment such as cameras and microphones, and from a structural change in the disease. primarily functional motor and vocal symptoms (gait, facial expression, voice, posture) is evaluating. Patent document number WO2025105987A1 describes a voice-based diagnosis of Parkinson's disease and 30 The monitoring method is discussed. The invention involves monitoring through the interface of a client electronic device. recording a subject's voice or uploading a pre-recorded audio file, this The recording is transmitted to a server via a web service, and the recording is processed using a neural network. processing and neural network analysis followed by the elimination of Parkinson's disease and / or parkinsonism symptoms in the subject The neural network response on the client device is 35, along with providing a response that includes a probability of its existence. the probability of the presence of parkinsonism and the classification of the record into two or more classes It includes the steps involved in obtaining it. This system relies on sound, which is a single modality, and 4 Parkinson's disease causes gait disturbance, reduced facial expression (hypomimia), and postural problems. It does not evaluate other important motor indicators such as instability in any way. In the present invention... In addition to voice, there are three further modalities: gait, facial expression, and posture. and thanks to the simultaneous analysis of these four modalities, it has been shown that a single modality is insufficient. Even in these situations, a reliable assessment can be made. 5 Patent document number US2020057078A1 describes the use of a device in the diagnosis of Parkinson's disease. This refers to the melanotransferrin protein or the nucleic acid that encodes it. The invention is a method for diagnosing Parkinson's disease in a subject, using the subject's saliva. or by evaluating the melanotransferrin level in a saliva sample and this level being 8.6 μg / ml 10 It is based on determining whether its value is below or above a certain level, such that A value below 8.6 μg / ml indicates Parkinson's disease. This method is non-invasive. Although it is not a biochemical test, it detects a single biomarker in saliva. It is based on measuring its concentration and has a direct link to the motor symptoms of the disease. There is no relation. The present invention, however, does not involve taking any body fluid sample, only 15 The results are obtained by analyzing video and audio recordings of physiological movements (walking, talking, facial expressions). It is an artificial intelligence system that provides information on the many motor and vocal function impairments of Parkinson's disease. It evaluates the multifaceted effects holistically. Patent document US11422137B2 states that in the diagnosis or prognosis of Alzheimer's disease, either 20 lactoferrin protein, or a combination thereof, is used in the diagnosis of Parkinson's disease. The text refers to a nucleic acid that codes for Alzheimer's disease. The invention is said to have helped diagnose Alzheimer's disease in a subject. It is a method aimed at measuring the level of lactoferrin in the subject's saliva or a saliva sample. assessment and whether this level is below or above 7.43 μg / ml It is based on the principle of determining that a value below 7.43 μg / ml indicates Alzheimer's 25 This demonstrates the disease. The invention also shows that the lactoferrin protein in a subject's saliva sample It also includes its use in the diagnosis of Parkinson's disease. The current invention utilizes saliva, for example. Unlike this approach, which is based on the analysis of biological samples, this is a completely non-invasive approach. It requires no laboratory infrastructure and allows the patient to perform natural motor and vocal activities. It offers a system that measures directly. 30 Patent document US12573032B2 describes how retinal images can be processed using machine learning. This text discusses systems and methods for predicting Parkinson's disease based on this analysis. The invention involves the acquisition of a retinal image by an image acquisition system, the acquired image This retinal image is analyzed by a 35-year-old trained to classify one or more retinal features contained within it. or processing using more machine learning models and one of these retinal images predicting whether the subject shows the onset or presence of Parkinson's disease This approach involves the steps to address structural changes in the retina of the eye. It relies on imaging and uses a special imaging device (optical coherence tomography, etc.) This requires specialized and expensive equipment, such as retinal imaging. Walking, facial expressions, and posture were recorded without hearing, only with standard video cameras. By analyzing the movements and sound signals recorded with standard microphones, Parkinson's disease 5 It determines the risk. Patent document CN121660967A describes a multimodal artificial intelligence-based Parkinson's disease treatment system. The invention refers to a motor symptom auxiliary diagnostic system. The invention features multi-mode imaging. It integrates the data, preprocessing, 3D convolutional neural network branch and VisionTransformer branch 10 Combining multimodal feature extraction, causal feature selection based on the IAMB algorithm, and multimodal feature extraction. Through modular fusion and multi-tasking learning modules, the four motor skills of Parkinson's disease It performs a combined classification and severity grading of the symptoms. This The system uses structural and functional imaging data (magnetic resonance imaging, etc.) It relies on and requires imaging equipment that is expensive and not available in every center. 15 The current invention allows for the use of only video and audio, without the need for expensive medical imaging equipment. It works based on recordings and uses four different types of data: gait, facial expression, voice, and posture. By combining these modalities, it enables the assessment of early-stage Parkinson's disease. In patent document number WO2021182645A1, the localization of accessory pathways of tachyarrhythmia is described. Diagnosis of idiopathic cardiomyopathy with a method and device for estimating the affected areas The text describes a method and device for assisting with electrocardiograms. The invention is an electrocardiogram (ECG). biopotential signal dataset and tachyarrhythmia accessory pathway localization region type information or an example where information about the type of idiopathic cardiomyopathy is entered as training data into an artificial intelligence model. training step and electrocardiogram biopotential signal dataset for a subject trained artificial 25 a step where accessory pathway localization region type information is calculated by entering it into the intelligence model This approach involves a single biopotential signal modality (electrocardiogram). It is based on and aimed at diagnosing a heart condition. The present invention, however, is based on electrophysiological It uses visual (video), auditory (sound), and kinematic (motion) data instead of signals, and Four different and complementary motor and vocal modalities are combined for the diagnosis of Parkinson's disease. 30 This allows for a much more comprehensive neurological assessment. Patent document KR102144718B1 describes nigrosome-1 classified using machine learning. an AI-based Parkinson's disease diagnostic device and method that uses the region It is mentioned that the invention involves imaging a patient's brain using a magnetic resonance imaging device for 35 days. Obtaining the first image related to the magnitude and phase of multiple echoes obtained by filming, Parkinson's Substantia nigra and nigrosome-1 region used as imaging biomarkers for the disease 6 The first image taken so that it can be observed is subjected to a quantitative sensitivity mapping algorithm. based on processing, this first processed image is analyzed through machine learning. Classification of a second image containing the nigrosome-1 region and the detected nigrosome-1 By analyzing whether the region is normal, the patient may be diagnosed with Parkinson's disease. This involves diagnosing whether or not it is present. The present invention describes a neuroanatomical biomarker 5 Instead of visual evaluation with expensive magnetic resonance imaging devices, low-cost Using inexpensive and readily available video cameras and microphones, the functional impairment of the disease is detected. assessing symptoms and using a holistic approach that combines four different modalities. It offers. Patent document CN106202952A describes a machine learning-based solution for Parkinson's disease. The diagnostic method is discussed. The invention improves upon traditional Parkinson's disease diagnostic methods. To address issues of low efficiency, high cost, and complex processes, primarily audio signals... the collection, feature extraction and selection from these audio signals, and then intelligent Creating an optimal support vector machine model using an optimization algorithm and 15 This model involves the steps of classifying and predicting the sounds to be tested. The method relies on sound, a single modality, and is used to treat gait abnormalities in Parkinson's disease. other significant motor disorders include decreased facial expression (hypomimia) and postural instability. It does not evaluate the symptoms in any way. In the present invention, however, in addition to sound, gait and facial features are also considered. There are three additional modalities, expression and posture, and these four modalities comprise 20 Thanks to its simultaneous analysis, it provides a reliable solution even in cases where a single modality is insufficient. An evaluation can be made. Patent document number US2023397866A1 describes pathologies of the cholinergic system, directly or indirectly. to diagnose, monitor or indirectly related diseases (including Parkinson's disease) 25 A test for stratification is being discussed. The invention involves the provided pupillary light reflex. This involves comparing the provided data with known normative data of the same type. pupillary light reflex data including at least one sample (at least one provided, measured with a known device) This includes data containing parameters (including) and at least one characteristic of the patient being examined. The probability of disease occurrence is determined by at least one neural network algorithm and / or part of the neural network. 30 using machine learning algorithms that include at least one mathematical function that is not present This is determined by the present invention, which is based on the measurement of a single physiological response (pupil reflex). In contrast to the previous approach, this study examines the four different clinical modalities of Parkinson's disease (gait, facial This involves simultaneously analyzing modalities (voice, posture, etc.) and generating probabilistic outputs from these modalities. a much more comprehensive evaluation system that combines decision fusion mechanisms 35 It offers. 7 Patent document number US2024410004A1 describes a neurodegenerative disease or disorder. a diagnosis that indicates a subject's likelihood of developing the disease and / or the progression of the disease A method for determining the score is described. The invention relates to a methylation state. a comparison with a reference pattern or a methylation status of additional markers It is based on comparison with a reference pattern. Methylation status, for example, a plasma 5 For example, it can be derived from extracellular DNA. The reference pattern is found in a library or repository. This can be realized on the device and, for example, from reference subjects using a machine learning technique. This can be achieved. This approach is based on genetic / epigenetic analysis and is non-invasive. Although it can be done with a non-existent blood test, it requires expensive genetic analysis infrastructure. The invention, however, does not require any genetic analysis; it can be done using only video and audio recordings. It is a system that directly assesses the motor and vocal symptoms of Parkinson's disease. Patent document number CN120392062A describes artificial intelligence, magnetic resonance imaging, and A system for the early auxiliary diagnosis of juvenile idiopathic arthritis by combining clinical information. and storage medium are mentioned. The invention involves deep learning-based tissue segmentation, 15 Multidimensional feature extraction and multimodal feature integration based on segmentation results. It performs disease classification using medical imaging (magnetic). Although it combines (resonance imaging) and clinical data in a multimodal way, it can treat joint conditions such as arthritis. It has focused on diseases and requires expensive imaging equipment. The current invention, however, Designed for Parkinson's disease, a neurological disorder, it has its own unique clinical testing environment (L-shaped 20 and uses video and audio-based data obtained through obstacle courses and structured tasks, and most Importantly, complementary motor modalities such as gait, facial expression, voice, and posture undergo delayed fusion. It differs fundamentally in terms of its integration. Patent document number KR102644343B1 describes 25 based on a machine learning model. A method and system for providing information for the diagnosis of Parkinson's disease. It is mentioned that the invention involves obtaining a brain image of a patient from an external device, and the image obtained... By performing segmentation on the brain image, multiple segments can be found within the brain image. Identifying anatomical regions of interest, and identifying multiple characteristics from these identified anatomical regions of interest. The value is calculated and 30 of these feature values ​​are selected using a machine learning model. This approach involves steps to determine the type of Parkinson's disease the patient has. neuroanatomical imaging data (magnetic resonance imaging, positron emission) (e.g., CT scans) relies on and requires expensive and specialized equipment. The present invention, however, without the need for expensive medical imaging methods such as brain imaging, just video and audio recordings to assess motor and vocal symptoms of Parkinson's disease, low-cost 35 It is a system. 8 Patent document US2009297446A1 describes a diagnostic formulation containing a tropane compound. It is stated that the invention contains at least 1.6 mCi / mL of radioactive material at least 51 hours after its creation. It provides a diagnostic formulation containing a tropane compound at a specific concentration. The formulation is optionally radiolabeled, useful in the diagnosis of Parkinson's disease. It contains a dopamine transporter ligand; an example of this is [123I]-2β-carbomethoxy-3β-(4-fluorophenyl)-5 N-(3-iodo-E-allyl)nortropan can be administered. This approach is suitable for nuclear medicine imaging (single photon). (emission computed tomography) relies on the use of radioactive materials and special It requires imaging equipment. The present invention, however, does not involve any radioactive material or a completely non-invasive and radiation-free method that does not require nuclear medicine infrastructure This approach determines the risk of Parkinson's disease using only video and audio recordings. 10 Patent document KR102122073B1 describes nigrosome-1 detected using machine learning. The invention describes a Parkinson's disease diagnostic device and method based on the region. multiple obtained by imaging a patient's brain with a magnetic resonance imaging device Obtaining the first image regarding echo magnitude and phase, image 15 of Parkinson's disease In order to observe the substantia nigra and nigrosome-1 region, which are used as biomarkers the processing of the first image obtained, and the processing of this first image through machine learning By analyzing the nigrosome-1 region, a second image containing this region is classified and identified. By analyzing whether the nigrosome-1 region is normal, the patient's Parkinson's disease can be diagnosed. This includes diagnosing whether or not the patient has the disease. The present invention relates to neuroanatomical 20 a biomarker visually detected with expensive magnetic resonance imaging devices Instead of evaluating them, low-cost and readily available video cameras and microphones are used. using four different modalities to assess the functional manifestations of the disease. It offers a holistic approach. Patent document number US2021079471A1 describes machine learning and high-dimensional technology. Idiopathic pulmonary fibrosis in transbronchial biopsies using transcriptional data Systems, methods, and classifiers for diagnosis are discussed. The invention includes examples. distinguishing between ordinary interstitial pneumonia and atypical interstitial pneumonia This approach allows for the genetic analysis of biopsy samples, which is an invasive procedure. It is based on. The present invention, however, uses a completely non-invasive method, removing any material from the body. Parkinson's disease can be diagnosed using only external video and audio recordings, without taking a tissue sample. It aims to determine the risk, and in this respect, it is much lower compared to invasive genetic tests. It offers a risk-free and patient-friendly approach. 35 Patent document CN114596306A describes a machine learning-based solution for Parkinson's disease. The diagnostic system is being discussed. The invention consists of an image preprocessing module and a fiber optic cable. 9 reconstruction module (using the diffusion tensor of each voxel to reconstruct the human brain in three dimensions) Probabilistic whole-brain fiber tracing that follows the most suitable fiber path between voxels in its spatial dimension. (using technology), a fiber bundle segmentation module, a feature extraction module, a training It includes a model module and a medical diagnostic module. This system uses diffusion tensor imaging. It relies on advanced magnetic resonance imaging techniques, such as those mentioned above, and is specialized and expensive. Imaging equipment and expert analysis are required. The current invention, however, focuses on the complex brain. Parkinson's disease can be diagnosed using only video and audio recordings, without the need for imaging techniques. a low-cost and easily applicable method for evaluating the motor and vocal symptoms of the disease It is a system. Patent document number WO2026030628A2 describes cell-type-specific spatial proteomics and machinery. Target, biomarker, and patient selection in central nervous system disorders using learning. Methods for its discovery are discussed. The invention involves preliminary steps from induced pluripotent stem cells. Generating neural cells from brain regions, cell production using antibody-enzyme conjugates Type-specific proteome profiling and spatial proteome profiling of sparse biological data 15 It includes the steps for applying statistical data amplification to clusters. SHAP-based feature Machine learning classifiers with varying degrees of importance, derived from mass spectrometry data. It identifies sequenced biomarkers. This approach is advanced in cell culture and proteomics. It is a complex laboratory method requiring infrastructure. The present invention, however, does not require any 20% of the patient's physiological processes do not require laboratory infrastructure or cell culture. It is a system based on video and audio recordings of their movements. Patent document number RU2682175C1 describes the differential diagnosis of post-traumatic and idiopathic epilepsy. A method for increasing the concentration of noregulin-1 in serum is described. The invention relates to this method. It is based on the determination that idiopathic epilepsy is in the range of 0-0.035 ng / mL, and above 25 In this case, a diagnosis of post-traumatic epilepsy is made. This method involves a single biochemical test. It is a biomarker-based laboratory test. The current invention, however, uses four different biomarkers instead of a single one. a method that uses hundreds of numerical parameters derived from modalities (gait, face, voice, posture) It is an artificial intelligence system that examines the multidimensional effects of Parkinson's disease on motor and vocal functions. It evaluates its effects holistically. 30 Patent document number KR102144719B1 describes the output generated by multiple learning models. an AI-based Parkinson's disease diagnostic device that uses multiple prediction results and The method is being discussed. The invention involves magnetic resonance imaging of a patient's brain. Obtaining the first image related to the magnitude and phase of multiple echoes captured by the device, 35 Substantia nigra and nigrosome-1 are used as imaging biomarkers for Parkinson's disease. Quantitative sensitivity mapping of the first image taken to observe the region. processing based on its algorithm extracts the nigrosome-1 region from this first processed image. Classification of a second image and determination of whether the detected nigrosome-1 region is normal. Diagnosis of whether a patient has Parkinson's disease is made by analyzing whether or not it is present. This includes the process of developing diagnostic tools. The diagnostic unit is capable of generating learning models for presence detection. The present invention enables the detection of a neuroanatomical biomarker using expensive magnetic resonance imaging devices. 5 instead of evaluating it visually, a low-cost and readily available video camera is used. using microphones to assess the functional symptoms of the disease and four different modalities It offers a holistic approach that unites all. Patent document CN121747900A describes 10 technologies based on multimodal brain image feature fusion. This refers to a Parkinson's disease diagnostic and predictive system for mild cognitive impairment. The invention is derived from structural magnetic resonance imaging images produced by FreeSurfer. Extraction of morphological features of the cortex and nucleus, PyRadiomics from relevant brain regions Extraction of first-order, shape and textural features using Lasso regression. Reduction of feature dimensionality and fusion of morphological, imaging, radiomic and clinical features 15 With this implementation, classification and prediction are carried out using machine learning algorithms. This system includes the steps involved in advanced magnetic resonance imaging analysis. It relies on expensive imaging hardware and requires complex software infrastructure. The current invention allows for the use of only video and audio, without the need for expensive medical imaging equipment. It works based on recordings and uses four different 20 features: gait, facial expression, voice, and posture. Assessing early-stage Parkinson's disease by combining motor and vocal modalities. It provides. Patent document US11534103B2 describes the use of deep learning to treat Parkinson's disease. 25 systems and methods for synthesizing magnetic resonance imaging images for diagnosis It is stated that the invention involves one or more human subjects using one or more processors. obtaining multiple magnetic resonance imaging images, this magnetic resonance preprocessing of imaging images, these magnetic resonance imaging images Implementing one or more convolutional neural networks to perform image analysis, To augment a dataset consisting of artificial scans for classification training, one or more 30 implementation of highly generative adversarial networks and magnetic resonance imaging images based on this, providing a classification related to the subject's Parkinson's disease diagnosis as output. It includes the following steps. The current invention addresses magnetic resonance imaging, which is expensive and not available in every center. Unlike this approach which requires imaging devices, any specialized imaging 35 without needing any additional equipment, just with basic equipment like a video camera and microphone. They are working and evaluating the functional symptoms of the disease. 11 Patent document CN116467647B describes a Parkinson's disease based on three-dimensional finger movement. The text discusses an artificial intelligence method and system for classifying diseases. The invention is based on three collected methods. Three-dimensional finger movement segmentation on three-dimensional finger movement data, three-dimensional This includes motion pattern feature extraction and automated diagnostic discrimination steps, particularly During the reciprocal movement of the thumb and index finger, simultaneously touch the closed and open points. 5 It solves the problem of inability to reach. This method only addresses the issue of being unable to reach a single body part (hand / finger). It analyzes a specific movement. The current invention is for the diagnosis of Parkinson's disease. four different assessment areas that are necessary and clinically significant (gait, facial expression, voice, This includes posture and is a holistic approach that is not limited to the analysis of a single finger movement. It offers an approach. 10 Patent document number CN120195408A describes a biomarker for diagnosing Parkinson's disease and The application of this is discussed. The invention utilizes machine learning and proteomics methods. by combining neuron-derived exosomes from a Parkinson's disease clinical cohort and A group of Parkinson's disease diagnostic biomarkers in exosomes derived from stellate glial cells 15 (One or more of NDUFC1, CHGA, CCL5 and ST3GAL6) were discovered and the AUC value It is stated that it reached 0.967. This method is used in advanced applications such as exosome isolation and proteomic analysis. It requires laboratory techniques. The present invention, however, does not require any laboratory infrastructure or Video and audio recording of the patient's physiological movements directly, without requiring biological sample processing. It is a system based on records. 20 Patent document number KR20250056304A describes how artificial intelligence can be used to treat a user's Parkinson's disease. an electronic device to determine whether it predicts the disease and how it works The method is described. The invention involves a processor that enables a user to make rapid eye movements Obtaining biological signal information extracted through polysomnography during sleep, these 25 Calculation of phenotypic information based on biological signaling information, biological signaling information and Determining the user's condition by providing phenotype information as input to a prediction model. and a predetermined threshold value between the user's biological signal information and phenotype information. If it exceeds this limit, the user will be predicted to have Parkinson's disease. This approach involves the steps of providing specialized and expensive sleep studies such as polysomnography. It requires work. The present invention, however, does not involve any specific night work or sleep monitoring. video and audio recordings that do not require any device and can be done quickly during daylight hours It is a system based on... Patent document number CN117476111A describes machine learning and plasma metabolism 35 The invention describes an early screening method for Parkinson's disease based on specific markers. Due to the difficulty of early diagnosis of Parkinson's disease and the reliance on cerebrospinal fluid biomarkers. 12 Due to the high risk and cost of the diagnosis, a machine learning algorithm (GBM- Analyzing patients' plasma metabolism data using RFE) and a novel prodromal The creation of a Parkinson's disease metabolite detection panel, along with a corresponding early detection, This involves the steps of constructing a screening diagnostic model. This method involves plasma samples. This requires the acquisition of data and advanced analytical techniques such as mass spectrometry. The present invention, however, 5 a completely non-invasive procedure that requires no blood samples or laboratory analysis It offers an approach. Patent document CN107045876A describes a sound-based diagnosis of Parkinson's disease severity. The method is discussed. The invention involves the use of sound signals and 10 in Parkinson's disease patients. summing up the corresponding Unified Parkinson's Disease Rating Scale values, Extracting features from audio signals can be used to create a domain adaptation algorithm. Using a ridge regression model, all obtained ridge regression models will be tested. Filtering the patient's existing data, and then performing ridge regression on the remaining filtering result. 15 models will be combined through a model fusion and tested using this combined model. steps for estimating the Unified Parkinson's Disease Rating Scale value for the patient This method relies on sound, a single modality, and is used by Parkinson's patients. When assessing the severity of the disease, other motor symptoms (gait, posture, facial expression) should be taken into account. It does not include that. The current invention, however, includes three additional features in addition to voice: gait, facial expression, and posture. It encompasses more modalities and combines probabilistic outputs from these four modalities into a decision fusion 20 By combining it with this mechanism, it offers a much more comprehensive assessment. Patent document MD1937Y describes non-infectious ocular arthritis in juvenile idiopathic arthritis in children. The text describes a method for diagnosing the activity of an inflammatory process. The invention applies to both... Collection of lacrimal fluid from the eye with Schirmer tapes and enzyme-linked immunosorbent assay 25 It is based on determining the concentration of the inflammatory biomarker protein S100A12. Different concentration ranges (inactive form up to 23.6 ng / mL, mild activity 23.7-28.4 ng / mL, (28.5-106.34 ng / mL is moderate activity, above 106.35 ng / mL is high activity), different types of inflammation. It diagnoses activity levels. This method is non-invasive but involves body fluid analysis. It is a biochemical test based on. The present invention, however, allows for a biochemical test performed in just 30 seconds without taking any sample of body fluids. The results are obtained by analyzing video and audio recordings of physiological movements (walking, talking, facial expressions). It is an artificial intelligence system that provides this. Patent document RU2764568C1 describes how video data can be analyzed using machine learning. The text describes a Parkinson's disease diagnostic method based on analysis. The invention involves a patient aged 35 and over. Collection and acquisition of video data involving one or more types of motor activity, The processing of the acquired video data, the processed video data for each type of motor activity 13 Obtaining a file containing key points of the patient's body from a single frame, Calculation of segments from the key points obtained, and based on the calculated segments Calculating relative velocity and acceleration, obtaining a data set from these calculations, and the resulting... segmentation of data and temporal and frequency analysis from the segmented dataset feature extraction, processing of extracted features with one or more classifiers, and output 5. It includes the steps for obtaining the data. This system is solely video-based motion. It is based on analytical methods and does not include other important clinical modalities such as voice and facial expression. The current invention, in addition to video-based gait and posture analysis, also includes structured speech. analysis of acoustic properties obtained from their functions and expression based on facial muscle movements It is a four-modality system that also includes analysis. 10 Patent document JP2022120755B1 describes a Parkinson's disease using neuromelanin imaging. The text discusses a device and method for providing information about a disease. The invention involves examining a patient's brain. an image receiving unit that captures an image taken by a magnetic resonance imaging device, Neuromelanin region 15, used as an imaging biomarker for Parkinson's disease. an image that pre-processes the magnetic resonance imaging image obtained so that it can be observed. The preprocessing unit analyzes the preprocessed magnetic resonance imaging image. classifying the first image containing the neuromelanin region and from the classified first image an image processing unit that detects the neuromelanin region and the detected neuromelanin region By analyzing whether it is normal or not, the doctor diagnoses whether the patient has Parkinson's disease. 20 It includes an image analysis unit. The present invention allows for the analysis of a neuroanatomical biomarker at an expensive cost. Instead of visual evaluation with magnetic resonance imaging devices, low Using inexpensive and readily available video cameras and microphones, the functional impairment of the disease is detected. assessing symptoms and using a holistic approach that combines four different modalities. It offers. 25 Patent document number RU2739321C1 describes the use of idiopathic atrial fibrillation in patients with this form of atrial fibrillation. This describes a method for diagnosing chronic active lymphocytic myocarditis. The invention concerns the patient's... levels of cardiac myosin-binding protein C and growth-stimulating factor 2 in the blood This involves determining specific threshold values ​​for these proteins (0.218 ng / mL and 30 ng / mL, respectively). A level above 22.06 ng / mL indicates the presence of chronic active myocarditis. This approach uses blood... It is a biomarker method based on analysis. The present invention is for the diagnosis of Parkinson's disease. Instead of biomarkers, a device that directly measures the most prominent motor and vocal symptoms of the disease, and By offering a functional evaluation system that makes it quantitative, the laboratory infrastructure It provides a low-cost and fast solution that does not require any additional effort. 35 14 Patent document number WO2025030062A2 describes clinical applications in patients with movement disorders. devices, systems, software, and methods for video-based monitoring of situations It is mentioned that the invention uses a machine learning algorithm to determine the severity of symptoms. It uses video-based objective symptom assessment for classification, and this methods can slow the progression of motor symptoms in patients with movement disorders such as Parkinson's disease 5 It can be used to diagnose and measure. This system is video-based only for motion detection. It is based on analysis, and other important methods for diagnosing Parkinson's disease such as voice analysis. It does not include modalities. The present invention, however, adds to video-based gait and posture analysis. Specifically, the analysis of acoustic features and facial muscles derived from structured speech tasks. It is a four-modality system that includes expression analysis based on movements. 10 Patent document number RU2657763C1 describes a method for diagnosing Parkinson's disease. The method is discussed. The invention belongs to the fields of neurology and clinical laboratory diagnostics, It can be used to diagnose Parkinson's disease. The method involves taking peripheral blood samples and measuring CD45+ levels. isolating a homogeneous fraction of the cells and lysing the isolated cell fraction 15 It includes measuring the concentration of oligomeric alpha-synuclein in CD45+ cell lysate, and A concentration above 2.7 pg / mL is diagnostic of Parkinson's disease. This approach, It is an invasive method requiring laboratory procedures such as blood sample collection and cell lysate analysis. The present invention, however, requires no blood samples or laboratory procedures whatsoever; it is completely natural. It offers a non-invasive approach. 20 Patent document number CN118415588B describes virtual reality and eye-tracking technology. The invention describes a Parkinson's disease diagnostic system and equipment based on an eye. motion trajectory acquisition module, an eye movement behavior feature extraction module, an eye movement function analysis module, an eye movement function evaluation module, an artificial 25 The system includes an intelligence module for Parkinson's disease diagnosis and a diagnostic result fusion analysis module. The eye movement trajectory acquisition module is collecting eye movement trajectories from a current subject; The AI ​​Parkinson's disease diagnostic module provides a preliminary diagnostic result output; the final diagnosis result is... The fusion analysis module then provides the final Parkinson's disease diagnosis result. This system, It requires virtual reality hardware and eye-tracking devices. The present invention, however, requires any 30 standard video only, requiring no special equipment (virtual reality glasses, eye tracker, etc.) It is a low-cost system that works with cameras and microphones. Patent document MD1938Y describes non-infectious ocular arthritis in juvenile idiopathic arthritis in children. A method for diagnosing the activity of the inflammatory process is mentioned (MD1937Y to 35). (Similarly). The invention, this time, concerns the determination of the chemokine IL-8 concentration in lacrimal fluid. It is based on the following: 60 pg / mL below inactive form, 61-125 pg / mL mild activity, 126-250 pg / mL moderate activity. and is identified as high activity above 250 pg / mL. The present invention describes this type of biochemical Unlike other tests, it does not require body fluids, provides immediate results, is repeatable, and involves the patient. Parkinson's disease without even needing any physical contact (like Schirmer tape). It offers a system that assesses the risk. Patent document number KR20220053382A describes a Parkinson's disease diagnostic device and its... The method is discussed. The invention involves patient-specific physical factors for multiple patients. clinical factors, retinal thickness factors obtained with optical coherence tomography, and Parkinson's disease. data that collects pathology levels over time, indicating the degree of disease progression. The 10-fold difference between the collection unit and the multiple factors collected, and the corresponding pathology level. a model that creates a predictive model using machine learning to learn about the relationship It includes a generation unit. The device also includes the patient's retinal thickness and physical and clinical data. Based on this, the system can diagnose Parkinson's disease. This system uses methods like optical coherence tomography. It requires a specialized imaging device. The current invention, however, is expensive, such as retinal imaging. and without the need for methods requiring special equipment, just with video and audio recordings 15 It is a low-cost system that works. Patent document CN121096610A specifies a general solution, not specifically for a condition like scoliosis. as a system in the field of medical image analysis and AI-assisted diagnosis It is mentioned. The invention is based on the human factor and the high 20 of the traditional Lenke classification. The semi-automatic nature and model excess of current artificial intelligence techniques are due to their lack of subjectivity. It offers a three-level task framework designed to overcome its inherent shortcomings. This framework, Spine segmentation and hierarchical attention mechanism using the Swin-Unet spine network. Precise lumbar pedicles are measured using the DeepLabv3+ model with automatic Cobb angle measurement. It includes segmentation and integrates multidimensional parameters to create complete Lenke tags 25 It produces a static image based on spinal column images, unlike the previous approach. Instead of performing an anatomical assessment, the focus is on dynamic motor and vocal tests specific to Parkinson's disease. analyzing the findings (gait, facial expression, voice, and posture) simultaneously and in time series It offers a multimodal system for processing data. Patent document number RU2712059C1 describes the severity of autonomic dysfunctions in Parkinson's disease. The text describes a method for diagnosing the degree of the disease. The invention involves analyzing a patient's blood serum. It is based on determining the amount of the neuropeptide galanin, and values ​​of 16 ng / mL and above. No autonomic dysfunction, mild autonomic dysfunction in the range of 10-16 ng / mL, 3-10 Moderate impairment in the ng / mL range and severe autonomic impairment at 2 ng / mL and below. 35 It is diagnosed. This method is a biomarker method based on blood analysis. The present invention, A functional assessment that directly measures the motor and vocal symptoms of Parkinson's disease. 16 By offering this system, we provide a low-cost and fast solution that does not require laboratory infrastructure. It provides. Patent document number RU2722666C1 describes the early (preclinical) diagnosis of Parkinson's disease. A method aimed at this is being discussed. The invention involves the production of alpha2-macroglobulin 5 in the tears of each eye. This involves determining the activity, and if this activity in one of the eyes is greater than 22 nmol / min*ml. higher and this value is more than 10 nmol / min*ml between matched eyes Parkinson's disease is diagnosed if it shows asymmetry. This method uses tear film. It is a method that requires sample collection and biochemical analysis. The present invention, however, allows for any a completely non-invasive procedure that requires no biological sample collection or laboratory analysis. It offers an approach. Studies have shown that efforts are being made to detect idiopathic Parkinson's disease in its early stages. Current techniques are largely based on subjective clinical observation scales or only a single... It appears to be based on digital systems that focus on biomarkers. In current systems, only 15 Image processing-based approaches that perform gait analysis, compared to those that analyze only audio recordings. acoustic models, deep learning-based solutions that evaluate facial expressions only, or only expensive imaging such as magnetic resonance imaging, which is not available in every center. Systems based on these devices are offered separately, and these systems have different motor and vocal capabilities. It does not include a decision-making mechanism that considers the parameters together. Clinical aspects of Parkinson's disease 20 The most fundamental technical challenge in diagnosis is the heterogeneous distribution of symptoms among individuals, Some patients exhibit significant gait disturbances, while others show changes in facial expression and voice parameters. Significant changes may not occur, and some patients may experience hypophonia or articulation disorders. While this is noticeable, walking and posture can remain normal. Additionally, some existing systems, such as saliva and blood, 25 biochemical tests based on the analysis of biological samples such as tears or plasma These approaches are becoming increasingly common and require invasive or semi-invasive procedures, which are expensive. It requires laboratory infrastructure. It is based on a single modality or a single biomarker. All of these systems provide a holistic view of the heterogeneous symptoms of Parkinson's disease. Their diagnostic sensitivity is limited because they lack the capacity for evaluation, resulting in low sensitivity. They operate with generalizability and a high error rate. Furthermore, there are 30 different current techniques. data obtained from evaluation methods in a simultaneous and standardized manner There is no data collection protocol or physical infrastructure in place to enable its collection, separate Analyses performed on data collected at different times and under different conditions are cross-modal analyses. This leads to compatibility and comparability issues. This invention eliminates all these technical shortcomings, making any biological sample collection process, which is expensive, 35 without the need for imaging devices or complex laboratory infrastructure, specifically 17 a clinic featuring a designed L-shaped walking track and a structured data collection workflow Four different clinical tests were conducted through a test environment: gait, facial expression, voice, and posture. It enables the acquisition of simultaneous and standardized data from the modality. The system provides for each It includes separate feature extraction and modeling modules for the modality, in the walking modality. Two-way GRU (Gated Recurrent Unit - 5) based on time series analysis of kinematic parameters Gated Repetitive Unit (RCU) based modeling, facial muscle movement units in the face modality Modeling derived features using an ensemble learning approach, acoustics in the sound modality. Machine learning-based classification of features and body regions in the posture modality. the principle of modeling angular relationships between them by expressing them with numerical parameters It applies this. The independently generated probabilistic classification outputs for these four modalities are 10 They are weighted and combined via a late fusion mechanism, resulting in a final Parkinson's risk score. The final assessment of an individual's risk of disease is made by comparing it to a threshold value. A classification decision is made. This allows for the identification of cases where a single modality is insufficient or where the symptoms are not adequate. Even in cases where it is disorganized, supporting findings from other modalities can help. fully 15 operating with high accuracy, higher generalizability and lower error rate Non-invasive, radiation-free, requires no biological samples, and has low cost. A costly early diagnosis assessment is offered. Ultimately, the problems mentioned above, which cannot be solved with the current technology, are the subject of discussion in the relevant technical field. This has made it necessary to make an innovation. THE PURPOSE OF THE INVENTION The present invention has been developed to eliminate the technical shortcomings mentioned above. With a multimodal artificial intelligence system and method for early diagnosis of idiopathic Parkinson's disease. It is related. The main aim of the invention is to treat idiopathic Parkinson's disease that occurs in the early stages and affects individuals over 25 years of age. objective, measurable, and repeatable heterogeneous motor and vocal symptoms that differ among them. The aim is to offer a technical solution that allows it to be evaluated in some way. In this way, the traditional Eliminating observer-dependent variability and inconsistency in scales based on subjective clinical observation. By removing this, a more reliable and standardized early diagnosis assessment is provided. Another aim of the invention is to improve the performance of existing single-modality digital diagnostic systems (walking only, only 30 (Systems that analyze voice or just facial expressions) have low sensitivity and a high error rate. The aim is to provide a fundamental solution to the problem of posture during work, including gait, facial expression, voice, and posture. Simultaneous and integrated analysis of data from four different clinical modalities. By doing this, supporting evidence can be obtained even in cases where a single biomarker is insufficient. By doing so, the classification performance is optimized. 35 18 Another aim of the invention is to provide a non-invasive, radiation-free solution for the early diagnosis of Parkinson's disease. The goal is to provide an assessment infrastructure that is non-inclusive and prioritizes patient comfort. Thanks to this system, which works based on video and audio recordings, patients can have any biological giving examples, wearing wearable sensors, or being exposed to ionizing radiation. This allows for evaluation without the need for further assessment, thus improving patient compliance and safety in clinical practice. 5 Another aim of the invention is to enable comparable and repeatable analysis of data from different modalities. a standardized clinical data collection protocol and physical infrastructure that enables the collection of data in this way The aim is to create a specially designed L-shaped walking path and a structured face, along with sound. Thanks to posture tasks, data from all modalities are simultaneously available within the same test flow. This is achieved by eliminating intermodal compatibility problems and thus leading to a more consistent decision. 10 The mechanism is provided. Another aim of the invention is to address the scattered and low-intensity manifestations of Parkinson's disease in its early stages. the possible symptoms, feature extraction and modeling designed separately for each modality. The aim is to enable detection through modules. Kinematics for walking. Bidirectional GRU-based modeling based on time series analysis of parameters, facial muscles 15 for the face Modeling features derived from motion units using ensemble learning, acoustics for sound. Machine learning-based classification of features and angular posture parameters for posture By expressing them numerically, each symptom group can be identified using the most appropriate method within its own context. is being evaluated. Another objective of the invention is to obtain probabilistic classification outputs from four different modalities. by combining weights via a late fusion mechanism, into a single modality. The goal is to produce a more balanced, reliable, and interpretable final decision outcome compared to others. Each System outputs are clinically evaluated by preserving modality-specific interim scores. It provides a verifiable decision-making framework in the processes, thus enabling physicians to make diagnostic decisions. A transparent and supportive contribution is offered to the process. 25 Another aim of the invention is to accommodate the heterogeneous clinical nature of Parkinson's disease, in different This allows for the individual evaluation of patients exhibiting combinations of symptoms. Even in patients exhibiting symptoms in a single modality, meaningful analysis can be performed via the relevant module. Outputs can be generated, and if there are no significant findings in other modalities, the system... a more comprehensive and reliable digital assessment while preserving the assessment capability 30 It is possible. Another aim of the invention is to create a practical, low-cost and easily applicable solution for a clinical setting. By offering a test infrastructure that can be established, it provides technology for the early diagnosis of Parkinson's disease. The goal is to increase accessibility. It only requires basic hardware components such as cameras and microphones. 19 Thanks to this system, an evaluation process that does not require special and expensive equipment can be achieved. This ensures easy applicability in different clinical settings and healthcare organizations. is obtained. The invention is particularly relevant in the field of neurological diagnosis, addressing the heterogeneous and early stages of idiopathic Parkinson's disease. objective, measurable and repeatable assessment of motor and vocal findings during period 5 an innovative system that aims to solve existing technical problems related to its evaluation and It offers a method. In current practices, disease assessment is largely subjective. clinical observation-based scales, digital systems focusing on only a single biomarker, expensive and imaging devices such as magnetic resonance imaging, which are not found in every clinical setting or invasive laboratory requiring analysis of biological samples such as blood, saliva, plasma 10 This is based on tests. This situation shows that Parkinson's disease varies among individuals. Low sensitivity, high error rate, limited generalizability due to clinical implications. This leads to technical problems such as cost, the need for invasive procedures, and the need for specialized equipment. This invention solves this technical problem by eliminating the need for expensive imaging equipment for any biological sample collection. or without the need for complex laboratory infrastructure, a specially designed L-shaped 15 A standard physical exercise involving facial, vocal, and posture tasks structured around a walking course. by obtaining simultaneous and standardized data from four different modalities through the mechanism It solves the problem. Based on the data obtained, the kinematic parameters for the gait modality are determined. A bidirectional GRU-based model, based on time series analysis, analyzes facial muscle movement for the facial modality. Modeling features derived from units using an ensemble learning approach, voice modality 20 machine learning-based classification of acoustic properties and body posture modality. The principle of modeling angular relationships between regions by expressing them with numerical parameters. It is applied. The invention involves independently generated probabilistic classifications belonging to these four modalities. the outputs are weighted and combined via a late fusion mechanism and a final Parkinson's It is based on comparing the risk score with a threshold value. Thus, the system uses a single 25 even when one modality is insufficient, supporting findings from other modalities Thanks to this, procedures that previously required only single-modality, expensive imaging equipment or were invasive are eliminated. More reliable, stable, interpretable, low-cost, and completely invasive compared to systems involving other methods. It can produce a decision outcome that is free of radiation and does not contain any harmful substances. The invention, along with the system, provides a multimodal artificial intelligence for the early diagnosis of idiopathic Parkinson's disease. It encompasses intelligence-based clinical decision support methods and includes the following steps: It includes: • The natural gait, change of direction, and turning movements of the individual being assessed, from the beginning and the finish must be completed on an L-shaped walking trail that offers a specific standard record-keeping pattern. 35 providing and maintaining the individual's postural posture through structured facial movements at a fixed angle and multiple angles. Recording visually via positioned video recording units. • During the individual's structured speech tasks, audio signals are recorded via a voice recording unit. acoustic recording. • Storing recorded raw video and audio data via a data collection and transmission subsystem. 5 and transfer to analysis modules for processing. • Walking characteristics can be determined from the walking footage obtained through the aforementioned video recording units. Using the subtraction module to identify anatomical key points and perform temporal and kinematic analysis. Converting walking parameters into numerical time series data. • Extracting facial features from facial images obtained through the aforementioned video recording units. 10 Using the module to detect facial muscle movement units and determine facial expression intensity and asymmetry. Creating digital face parameters containing. • Extracting sound features from sound signals obtained through the aforementioned sound recording unit. Using the module, acoustic properties including fundamental frequency, amplitude, and spectral distribution were numerically determined. Converting to variables. 15 • Posture characteristics can be determined from posture images obtained through the aforementioned video recording units. Detecting angular and positional relationships between body regions using an extraction module and To create numerical posture parameters related to postural disorders. • Normalize the time series data generated by the aforementioned walk feature extraction module. do this, then process it with a bidirectional GRU-based model via the walk modeling module 20 and generate the first probabilistic classification output. • Normalizing the numerical parameters generated by the aforementioned facial feature extraction module, then processing with a community learning-based model via the face modeling module and Generating the output of the second probabilistic classification. • Normalizing the acoustic variables produced by the aforementioned sound feature extraction module, 25 then processing with a machine learning-based model via the sound modeling module and Generating the third probabilistic classification output. • Normalize the angular parameters generated by the aforementioned posture feature extraction module. do this, then with a machine learning-based model via the posture modeling module. processing and generating the fourth probabilistic classification output. 30 21 • The four different probabilistic classification outputs generated are processed using a decision fusion module. combining them by weighting and assigning the resulting combined risk score to a threshold value. by comparing them, the final classification decision regarding an individual's Parkinson's disease risk status producing. The structure of the current invention and the best understanding of its advantages, including additional elements, are outlined in section 5. This should be evaluated together with the figures explained below. BRIEF DESCRIPTION OF THE FIGURES Figure 1: Multimodal AI-based system for early diagnosis of idiopathic Parkinson's disease. This is a schematic view of a clinical decision support system. REFERENCE NUMBERS 1 L-Shaped Walking Trail 2 Video Recording Units 3 Sound Recording Units 4 Data Collection and Transfer Subsystem 15 Walking Feature Extraction Module 6 Face Feature Extraction Modules 7 Sound Feature Extraction Modules 8 Posture Feature Extraction Modules 9 Walking Modeling Module 20 Face Modeling Module 11 Sound Modeling Modules 12 Posture Modeling Modules 13 Decision Fusion Modules DETAILED DESCRIPTION OF THE INVENTION This detailed explanation focuses solely on the innovation in the invention to provide a better understanding of the subject matter. This is conveyed without being limited to examples. Accordingly, in the following explanations and figures, Multimodal artificial intelligence system and method for early diagnosis of idiopathic Parkinson's disease. It is explained. 30 Figure 1 shows a multimodal AI-based system for the early diagnosis of idiopathic Parkinson's disease. This is a schematic view of a clinical decision support system. Accordingly, the system consists of the following components: It is coming from: 22 • Controlled natural gait, change of direction, and turning movements of the individual being evaluated. and enables recording in a manner that closely resembles natural movements, with a defined standard start and end point. an L-shaped walking trail offering a registration pattern (1), • The individual's structured facial movements and postural stance, and the aforementioned L-shaped gait Fixed-angle and multiple 5 cameras that enable visual recording of the walk on the trail (1). positioned video recording units (2), • Acoustically interpreting the voice signals of an individual during structured speech tasks. a sound recording unit (3) that enables recording, • Storing recorded raw video and audio data and sending it to analysis modules for processing. a data collection and transfer subsystem that enables transfer (4), 10 • Anatomical images of the gait obtained through the aforementioned video recording units (2) By identifying key points, temporal and kinematic walk parameters can be calculated numerically. a walk feature extraction module (5) that converts series data, • Facial muscle movements from facial images obtained through the mentioned video recording units (2) By identifying the units, numerical facial parameters such as expression intensity and facial asymmetry are determined. a face feature extraction module (6), which creates • Fundamental frequency, amplitude of the sound signals obtained through the aforementioned sound recording unit (3) and an audio feature extraction system that converts acoustic properties, such as spectral distribution, into numerical variables. module (7), • Body regions from posture images obtained through the mentioned video recording units (2) 20 numerical posture analysis of postural disorders by identifying angular and positional relationships between them. a posture feature extraction module (8) which forms its parameters, • Time series data generated by the mentioned walk feature extraction module (5) It operates with a bidirectional GRU-based model by normalizing and first probabilistic classification. a walk modeling module that produces its output (9), 25 • Normalize the numerical parameters generated by the aforementioned face feature extraction module (6). by operating with a community learning-based model and producing a second probabilistic classification output. a face modeling module that produces (10), • Normalize the acoustic variables produced by the aforementioned sound feature extraction module (7). by operating with a machine learning-based model and producing a third probabilistic classification output of 30 a voice modeling module that produces (11), 23 • Normalize the angular parameters generated by the aforementioned posture feature extraction module (8). by operating with a machine learning-based model and fourth probabilistic classification a posture modeling module (12) that produces its output and • The mentioned gait modeling module (9), face modeling module (10), voice modeling module Four different probabilistic classifications produced by the posture modeling module (12) (11) 5 by weighting and combining the outputs and assigning the resulting combined risk score to a threshold value by comparing them, it produces the final classification decision regarding an individual's Parkinson's disease risk status. a decision fusion module (13). The invention focuses on multimodal artificial intelligence for the early diagnosis of idiopathic Parkinson's disease. The clinical support system based on motor and vocal abilities of the individual being evaluated. Simultaneous processing and combination of multiple biomarkers obtained from their functions It is based on the principle that the system works by mimicking the individual's natural gait, change of direction, and turn. It is started by performing its movements on an L-shaped walking track (1); this track, By providing a standard recording pattern with defined start and end dates, the individual's movements are recorded in a controlled and natural manner. It allows the movements to be recorded in close proximity. The individual follows the aforementioned L-shaped walking path (1) 15 while performing walking tasks, they also use structured facial expressions and It also performs postural tasks; all this visual data is presented at a fixed angle and multiple angles. Simultaneously recorded via positioned video recording units (2). The individual also, Structured speaking tasks (reading from a fixed text, counting from one to twenty, weekly tasks) During the sorting of days, the sound signals are acoustically recorded via the sound recording unit (3) 20 It records all raw video and audio data. The data collection and transmission subsystem records all recorded data. (4) is stored and transferred to the relevant analysis modules for processing. Walking footage obtained through the aforementioned video recording units (2) shows the walking characteristics. The extraction module (5) processes the anatomical key through the images; this module (5) extracts the anatomical key from the images. By identifying the points, the right and left foot lift height and two and three-dimensional foot placement are determined. 25 angle, right and left knee bending angle, two and three-dimensional hip horizontal sliding angle, and two and three-dimensional Temporal and kinematic gait parameters such as heel horizontal glide angle are analyzed as numerical time series. converts it into data. Similarly, the face obtained through the aforementioned video recording units (2) The images are processed by the face feature extraction module (6); this module (6) extracts face images from the face features. By identifying facial muscle movement units, it can determine blinking frequency, expression intensity, facial asymmetry, and jaw movement. numerical facial parameters such as movement amplitude, lip movement parameters, and facial muscle activation times It creates the parameters. The sound signals obtained through the mentioned sound recording unit (3) are sound The feature extraction module (7) processes the digital signal processing methods. using fundamental frequency, amplitude, jitter, shimmer, speech rate, pause durations, spectral It converts acoustic properties such as dispersion and formant frequencies into numerical variables. The 35 mentioned... 24 Posture images obtained through video recording units (2) are used to extract posture features. This is processed by module (8); module (8) processes the angular and positional relationships between body regions. by determining two and three-dimensional trunk rotation angle, shoulder height difference, and two and three-dimensional neck numerical values ​​such as tilt angle, two- and three-dimensional body tilt angle, and two- and three-dimensional ear tilt angle. It establishes posture parameters. 5 Time series data generated by the mentioned walk feature extraction module (5) are normalized. After being processed, it is processed by the gait modeling module (9); this module (9) processes the data bidirectionally. It produces the first probabilistic classification output by evaluating it with a GRU-based model. The numerical parameters generated by the aforementioned face feature extraction module (6) are normalized. After being processed, the face modeling module (10) processes the data; this module (10) collects the data into a collection of 10 It produces a second probabilistic classification output by evaluating it with a learning-based model. The acoustic variables produced by the aforementioned sound feature extraction module (7) are normalized. After being processed, the data is processed by the sound modeling module (11); this module (11) processes the data into the machine. It produces a third probabilistic classification output by evaluating it with a learning-based model. Angular parameters generated by the aforementioned posture feature extraction module (8) are normalized to 15 After being processed, the data is processed by the posture modeling module (12); this module converts the data into machine modeling module data. It produces the fourth probabilistic classification output by evaluating it with a learning-based model. In the final stage, the four different probabilistic classification outputs generated are combined into a decision fusion module. (13) collects the scores for each modality by module (13), which is predetermined or It combines them by weighting them using weighting coefficients that can be learned by the system. 20 and by comparing the resulting combined risk score to a threshold value, the individual's Parkinson's disease is determined. It produces the final classification decision regarding the risk situation. Thanks to this holistic working principle... The system can accommodate other modality even when a single modality is insufficient or symptoms are scattered. A reliable, balanced, and interpretable decision outcome with supporting findings from the modalities. It offers. 25

Claims

REQUESTS 1. Multimodal AI-based clinical approach for early diagnosis of idiopathic Parkinson's disease. It is a decision support system, and its feature is... ● The individual being evaluated had the following abilities: natural gait, change of direction, and turning. enabling the recording of movements in a controlled manner that closely resembles natural movements. An L-shaped walking trail offering a standard record-keeping pattern with defined start and finish points. (1), ● The individual's structured facial movements and postural posture, and the aforementioned L-shaped 10 fixed angle and multiple recordings of the walk on the walking trail (1) positioned video recording units (2), ● Audio recording that captures an individual's vocal signals during structured speech tasks. registration unit (3), 15 ● Recorded raw video and audio data are analyzed for storage and processing. a data collection and transfer subsystem (4) that transfers data to its modules ● 20 walking images obtained through the aforementioned video recording units (2) Temporal and kinematic gait by identifying key anatomical points. A walk feature extraction process that converts parameters into numerical time series data. module (5), ● Face images obtained through the mentioned video recording units (2) 25 A face that generates numerical facial parameters by identifying muscle movement units. Feature extraction module (6), ● The acoustics of the sound signals obtained through the aforementioned sound recording unit (3) a sound feature extraction module (7) that converts the features into numerical variables, 30 ● From the posture images obtained through the aforementioned video recording units (2) Posture by identifying angular and positional relationships between body regions a posture characteristic that constitutes numerical posture parameters related to disorders subtraction module (8), 35 26 ● Time series generated by the mentioned walking feature extraction module (5) It operates with a bidirectional GRU-based model by normalizing the data and the first a walk modeling module that produces probabilistic classification output (9), ● The numerical parameters generated by the aforementioned face feature extraction module (6) are 5 It operates with a community learning-based model by normalizing and using a second probabilistic approach. a face modeling module that produces classification output (10), ● Acoustic variables produced by the aforementioned sound feature extraction module (7) normalizing and operating with a machine learning-based model and third 10 a voice modeling module that produces probabilistic classification output (11), ● Angular generated by the aforementioned posture feature extraction module (8) a machine learning-based model that operates by normalizing parameters and a posture modeling module 15 that produces the fourth probabilistic classification output (12), ● Combining and weighting four different probabilistic classification outputs, and By comparing the resulting combined risk score to a threshold value, the individual's Parkinson's disease can be diagnosed. a decision fusion 20 that produces the final classification decision regarding disease risk status module (13) It includes.

2. Multimodal AI-based system for early diagnosis of idiopathic Parkinson's disease. It is a clinical support method, and its characteristic feature is... ● The individual being assessed exhibited natural gait, changes of direction, and turns. L-shaped movements that offer a standard recording pattern with defined start and end times. to be carried out on the walking trail (1) and the individual’s structured face fixed-angle and multiple-position video recording of movements and postural stance recording through units (2), 30 ● The voice recording unit during the individual's structured speech tasks. (3) Acoustic recording of sound signals. ● Data collection and transmission subsystem for recorded raw video and audio data 35 (4) storage and transfer to analysis modules for processing, 27 ● Walk obtained through the aforementioned video recording units (2) anatomical key from images, using gait feature extraction module (5) Determining the points and the temporal and kinematic gait parameters. conversion to numerical time series data, 5 ● Face obtained through the mentioned video recording units (2) facial muscle movement from images is extracted using the facial feature extraction module (6). identification of units and numerical data including facial expression intensity and facial asymmetry. Creating facial parameters, 10 ● From the sound signals obtained through the aforementioned sound recording unit (3), sound Fundamental frequency, amplitude and spectral distribution were determined using the feature extraction module (7). converting acoustic properties into numerical variables, ● Posture obtained through the aforementioned video recording units (2) Using the posture feature extraction module (8) to extract body regions from images Identifying the angular and positional relationships between them and correcting postural disorders. creation of numerical posture parameters related to, ● Time series generated by the mentioned walking feature extraction module (5) normalization of data and bidirectional walking modeling through the walking modeling module (9) processing with a GRU-based model and the first probabilistic classification output production, ● Digital output generated by the aforementioned face feature extraction module (6) normalization of parameters and community through face modeling module (10) processing with a learning-based model and second probabilistic classification producing the output, ● Acoustics produced by the aforementioned sound feature extraction module (7) normalization of variables and machine through sound modeling module (11) processing with a learning-based model and third probabilistic classification producing the output, 35 ● Angular generated by the aforementioned posture feature extraction module (8) normalization of parameters through the posture modeling module (12) 28 processing with a machine learning-based model and fourth probabilistic generating classification output, ● The four different probabilistic classification outputs generated are combined into a decision fusion module. (13) by weighting and combining and the resulting combined 5 An individual's risk of Parkinson's disease is assessed by comparing their risk score to a threshold value. producing the final classification decision regarding its status It includes the steps of the process.

Images