Multi-modal cerebellar function evaluation neural physical examination method and system

Abstract

The invention relates to the field of biomedical engineering, and provides a multi-mode cerebellum function evaluation neurological physical examination method and system, and the method comprises the steps: guiding a patient to execute an examination task, and synchronously collecting myoelectricity and skeletal animation signals; continuously collecting signals according to a fixed time interval and a fixed time window; according to the time sequence, analyzing and processing the signal corresponding to each time window in sequence; the analysis processing comprises the steps of splicing the myoelectricity and skeletal animation signals of the current time window, and inputting the model to obtain an evaluation result corresponding to the current time window, and the evaluation result depends on an evaluation result corresponding to a preorder time window. The problems that the cerebellum function evaluation result is not comprehensive and the efficiency is low are solved.

Description

Technical Field

[0001] This invention relates to the field of biomedical engineering, and more particularly to a method and system for multimodal cerebellar functional assessment and neurological examination. Background Technology

[0002] Current clinical assessments of cerebellar function primarily rely on traditional methods, all of which have significant limitations and inefficient processes. For example, manual visual assessment involves doctors instructing patients to perform specific actions, and cerebellar function is judged by visual observation. However, this method cannot quantify deviations in movement trajectories, leading to highly subjective results. Single-sensor detection, using devices such as accelerometers and electromyography sensors to collect data, easily loses spatiotemporal characteristics related to ataxia, making it difficult to comprehensively reflect cerebellar functional status. Static balance benches can only capture steady-state data and lack the ability to analyze transitional movements, such as the take-off and landing phase of a single-leg hop. Furthermore, the need for patients to perform designated actions based on manual demonstrations during assessment can lead to problems such as delayed initiation and inconsistent movement amplitude, requiring repeated actions and resulting in low clinical examination efficiency. Therefore, existing cerebellar function assessment methods require further optimization. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a multimodal cerebellar function assessment neurological examination method and system to address the above-mentioned deficiencies of the prior art, aiming to solve the problems of incomplete and inefficient cerebellar function assessment results in the prior art.

[0004] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0005] Firstly, a multimodal cerebellar functional assessment neurological examination method is proposed, including the following steps:

[0006] Explain the key points of the examination task to the patient, and then guide the patient to perform the examination task; the examination task includes finger-to-nose test, standing balance test, single-leg hop, and short-distance walking test;

[0007] Electrode pads are attached to the patient's limbs to collect electromyographic signals as the patient performs the examination task; the patient's three-dimensional posture is acquired through an inertial motion capture system to collect skeletal animation signals as the patient performs the examination task; during the patient's examination task, signals are continuously collected at fixed time intervals t1, each collection period corresponds to a time window of duration t2, and each time window corresponds to a set of electromyographic signals and skeletal animation signals; the collection continues until the examination task is completed or the preset time limit is reached.

[0008] The electromyography and skeletal animation signals corresponding to each time window are analyzed and processed sequentially according to the time sequence; the signal data corresponding to the previous time window is analyzed and processed before the signal data corresponding to the next time window is analyzed and processed.

[0009] The analysis and processing include: preprocessing a set of electromyographic signals and skeletal animation signals corresponding to the current time window and then splicing them to obtain spliced ​​data; inputting the spliced ​​data into a preset model and outputting evaluation results, which include task completion markers, task completion scores, and abnormal event markers.

[0010] Furthermore, in the analysis and processing, the steps of inputting the spliced ​​data into a preset model and outputting evaluation results include: inputting the spliced ​​data into a convolutional neural network model to perform feature fusion and obtain feature vectors; inputting the feature vectors into a neural network model based on a long short-term memory network to obtain the latent feature representations corresponding to the feature vectors; and using a network structure based on a multilayer perceptron to map the latent feature representations to the evaluation results; when processing the feature vectors of the current time window, the neural network model based on a long short-term memory network integrates the evaluation results of previous time windows.

[0011] Furthermore, the analysis and processing also includes calculating the confidence level corresponding to the current time window. When the confidence level is greater than the preset confidence level threshold, the inspection task is determined to be completed. The confidence level is calculated as CS_t = α * task quality confidence level + β * timing consistency confidence level + γ * signal quality confidence level. In the formula, α, β, and γ are weighting coefficients, and α + β + γ = 1.

[0012] Furthermore, when the confidence level is greater than a preset confidence threshold, in addition to terminating the currently ongoing inspection task, inspection tasks associated with the currently ongoing inspection task are also skipped.

[0013] Furthermore, in the analysis and processing, the preprocessing steps for the electromyographic signals include filtering and downsampling to obtain a matrix of dimension C×(f*t2), where C is the number of electromyographic signal channels, f is the frequency of the skeletal animation signal, and t2 is the acquisition time window.

[0014] Furthermore, in the analysis and processing, the preprocessing step of the skeletal animation signal includes: extracting key points from each frame of the skeletal animation signal, with each key point represented by a set of two-dimensional coordinates to form a coordinate matrix of dimension P×2×(f*t2), where P is the number of key points, f is the frequency of the skeletal animation signal, and t2 is the duration of the time window; establishing an adjacency matrix of dimension P×P based on the connection relationship between key points, where P is the number of key points, and the matrix elements are defined as the distance between key points with a connection relationship, while the elements corresponding to key points without a connection relationship are set to 0; inputting the coordinate matrix and the adjacency matrix into a graph convolutional neural network model and mapping them to a matrix of dimension C×(f*t2), where C is the number of electromyographic signal channels, f is the frequency of the skeletal animation signal, and t2 is the acquisition time window.

[0015] Furthermore, the spliced ​​data is a three-dimensional matrix with dimensions 2×C×(f*t2), where C is the number of electromyographic signal channels, f is the frequency of skeletal animation signals, and t2 is the acquisition time window.

[0016] Furthermore, when performing standing balance tests, single-leg hops, and short-distance walking tests, plantar pressure data of the patient is collected through a plantar pressure acquisition system; the plantar pressure data includes plantar pressure distribution, pressure center trajectory, impulse, and gait spatiotemporal parameters.

[0017] Furthermore, during the analysis and processing, the plantar pressure data is spliced ​​with electromyography (EMG) signals and skeletal animation signals to form a three-dimensional matrix with dimensions 3×C×(f*t2), where C is the number of EMG signal channels, f is the frequency of skeletal animation signals, and t2 is the acquisition time window.

[0018] Secondly, a multimodal cerebellar functional assessment neurological examination system is proposed, including:

[0019] The task guidance module is used to explain the key points of the examination task to the patient and then guide the patient to perform the examination task; the examination tasks include finger-to-nose test, standing balance test, single-leg hop, and short-distance walking test;

[0020] The multimodal signal acquisition module is used to attach electrode pads to the patient's limbs to acquire electromyographic signals when the patient performs the examination task; it also acquires the patient's three-dimensional posture through an inertial motion capture system to acquire skeletal animation signals when the patient performs the examination task; during the patient's examination task, signals are continuously acquired at fixed time intervals t1, with each acquisition period corresponding to a time window of duration t2, and each time window corresponding to a set of electromyographic signals and skeletal animation signals; acquisition continues until the examination task is completed or the preset time limit is reached;

[0021] The signal processing module is used to analyze and process the electromyographic signals and skeletal animation signals corresponding to each time window in a time sequence; after the signal data corresponding to the previous time window has been analyzed and processed, the signal data corresponding to the next time window will be analyzed and processed.

[0022] The analysis and processing include: preprocessing a set of electromyography (EMG) signals and skeletal animation signals corresponding to the current time window and then splicing them to obtain spliced ​​data; inputting the spliced ​​data into a convolutional neural network model for feature fusion to obtain a feature vector; inputting the feature vector into a neural network model based on a long short-term memory network to obtain the latent feature representation corresponding to the feature vector; using a network structure based on a multilayer perceptron to map the latent feature representation to the evaluation result, which includes task completion label, task completion score, and abnormal event label; when processing the feature vector of the current time window, the neural network model based on a long short-term memory network integrates the evaluation results of previous time windows.

[0023] Furthermore, the signal processing module is also used to synchronously map the patient's actual actions to the task guidance module, ensuring that the key points of the examination task actions explained by the task guidance module to the patient are consistent with the patient's actual actions.

[0024] The beneficial effects of this invention are:

[0025] 1. After analyzing and processing the signal data corresponding to the previous time window, the signal data corresponding to the next time window is analyzed and processed. Based on the continuous evaluation of this time window, the evaluation results corresponding to the previous time window are integrated using an LSTM model, so that the evaluation results corresponding to the current time window depend on the completion status of the previous actions. This achieves the effects of strengthening the temporal correlation of the evaluation results, correcting dynamic biases, and improving evaluation accuracy. Moreover, when the confidence level reaches a preset threshold, the current examination task is automatically terminated, skipping related redundant examination tasks, which significantly improves the efficiency and relevance of clinical examinations.

[0026] 2. Establish a spatiotemporal correlation model between electromyographic signals (muscle activation timing) and three-dimensional posture (joint movement trajectory), overcome the limitations of single modality, solve the problem of multi-dimensional quantification, realize the full-dimensional quantitative assessment of cerebellar function, and make the cerebellar function assessment results more comprehensive and reliable.

[0027] 3. Develop a real-time interactive mechanism of "execution-feedback-correction". The signal processing module is used to synchronously map the patient's actual actions to the task guidance module, ensuring that the key points of the examination task actions explained by the task guidance module to the patient are consistent with the patient's actual actions. This allows the patient's actions to be corrected in a timely manner, further improving examination efficiency, enhancing the standardization of examinations, and obtaining more reliable evaluation results.

[0028] 4. After processing the skeletal animation signal using a graph convolutional neural network (GCN) model, it is then spliced ​​with electromyographic signals using a convolutional neural network (CNN) model. This lightweight GCN-CNN model design can intelligently identify subtle ataxia features, such as intention tremor and postural compensation, breaking through the subjective limitations of traditional visual assessment. Attached Figure Description

[0029] The invention will now be further described with reference to the accompanying drawings.

[0030] Figure 1 This is a framework diagram of the multimodal cerebellar functional assessment neurological examination method and system according to an embodiment of the present invention. Detailed Implementation

[0031] like Figure 1 As shown in the figure, this invention proposes a multimodal cerebellar functional assessment neurological examination method, including the following steps:

[0032] S1. Explain the key points of the examination task to the patient and then guide the patient to perform the examination task; the examination task includes finger-to-nose test, standing balance test, single-leg hop, and short-distance walking test;

[0033] Specifically, patients need to be guided to stand in a designated position to obtain better movement space and signal acquisition effect. A virtual coaching system should be used to play audiovisual prompts to explain the examination tasks to be performed, and the key points of the movements should be explained to the patient through demonstration. For the finger-to-nose test, the patient should be seated or standing with their eyes looking straight ahead and keeping their limbs relaxed. First, use the non-dominant hand to complete the action: raise the arm, accurately touch the tip of the nose with the fingertip, and then slowly return the arm to the original position, repeating several times. Repeat the above action with the dominant hand, avoiding excessive trunk swaying during the process. For the standing balance test, the patient should stand barefoot with feet together, arms hanging naturally at their sides, eyes open, and maintain a stable stance. For the single-leg hop test, the patient should stand with hands on hips for trunk stability, raise the non-dominant leg, and hop repeatedly along the marked line on the ground using the dominant leg. Repeat the above action with the non-dominant leg, avoiding trunk twisting or loss of balance during the process. For the short-distance walking test, the patient should walk at a normal pace along the marked straight line from the starting point to the end point, and then return to the starting point along the same route, maintaining a natural gait throughout. In addition to the examination tasks proposed in this embodiment, examination tasks can be added or removed as needed, such as the Romberg test, the heel-to-shin test, etc.

[0034] Before acquiring signals, spatial positioning calibration is required. This includes ensuring the patient maintains a standard anatomical posture, such as standing upright with arms hanging naturally at their sides, palms facing forward, and feet together. Using the spatial positions of the 17 IMU nodes at this point as a reference, a human skeletal coordinate system is established, and the three-dimensional posture data is uniformly mapped to this coordinate system. Furthermore, a synchronization signal is triggered to align the time axis of the electromyography (EMG) signal acquisition device with that of the inertial motion capture system, ensuring precise temporal matching between the EMG signals and the skeletal animation signals at the same moment.

[0035] S2, electrode pads are attached to the patient's limbs to collect electromyographic (EMG) signals during the examination. These EMG signals are surface EMG signals with 4 signal channels. The patient's three-dimensional posture is acquired through an inertial motion capture system, and skeletal animation signals during the examination are collected. Specifically, the inertial motion capture system uses 17 inertial measurement units (IMUs) with integrated accelerometers, gyroscopes, and magnetometers at key skeletal locations, including the head, neck, shoulders, elbows, wrists, hips, knees, and ankles, to collect data on posture, angular velocity, and acceleration of each part in real time during the examination. Combining skeletal kinematics principles and algorithm models, the system accurately calculates the three-dimensional motion posture of the human body, ultimately generating a skeletal animation that is synchronized with the human body's movements in real time and features smooth and natural joint movements.

[0036] In one implementation, when performing standing balance tests, single-leg hops, and short-distance walking tests, plantar pressure data is collected from the patient using a plantar pressure acquisition system. This system includes a piezoresistive pressure sensor array, a signal processing and data acquisition module, and data analysis software to acquire and analyze plantar pressure distribution, pressure center trajectory, impulse, and gait spatiotemporal parameters. Specifically, plantar pressure distribution can quantify the peak pressure and distribution symmetry in various areas such as the heel, arch, and forefoot. Patients with ataxia may exhibit abnormally asymmetrical pressure distribution or concentration in atypical areas. The pressure center trajectory is the gold standard for assessing balance function; it reflects the path of the body's center of gravity projection on the sole of the foot. Cerebellar dysfunction may lead to increased amplitude, speed, or irregularity of the trajectory swaying. Impulse, the integral of pressure over time, is particularly important for assessing the total load borne by the foot during the support phase and helps identify potential injury risk points. Gait spatiotemporal parameters can accurately calculate stride length, stride speed, cadence, and the duration of the support and swing phases. In dynamic testing, these parameters objectively reflect the rhythmicity and stability of walking. The four types of features extracted from the plantar pressure data are time-varying sequences. The plantar distribution is a two-dimensional matrix heatmap sequence, in the form of (length, width, time * sampling rate); the pressure center trajectory is a two-dimensional coordinate sequence, in the form of (2(x,y coordinates), time * sampling rate); and the impulse (1-dimensional) and gait spatiotemporal parameters (5-dimensional) are concatenated to form a time-series feature sequence, in the form of (6, time * sampling rate).

[0037] During the patient's examination, signals are continuously collected at fixed time intervals t1. Each collection period corresponds to a time window of duration t2, and each time window corresponds to a set of electromyographic signals and skeletal animation signals. The collection continues until the examination is completed or the preset time limit is reached. In one embodiment, in addition to a set of electromyographic signals and skeletal animation signals, each time window also corresponds to a set of plantar pressure data.

[0038] S3, according to the time sequence, analyzes and processes the electromyographic signals and skeletal animation signals corresponding to each time window in sequence; after the signal data corresponding to the previous time window is analyzed and processed, the signal data corresponding to the next time window is analyzed and processed.

[0039] The analysis and processing include: preprocessing and then splicing a set of electromyographic signals and skeletal animation signals corresponding to the current time window to obtain spliced ​​data; inputting the spliced ​​data into a trained convolutional neural network (CNN) model for feature fusion to obtain a feature vector of length N; inputting the feature vector into a trained long short-term memory (LSTM) based neural network model to obtain the latent feature representation corresponding to the feature vector of the current time window; mapping the latent feature representation into a 3-dimensional score using a multilayer perceptron (MLP) based network structure to obtain the scoring result corresponding to the current time window. The evaluation result includes task completion label, task completion score, and abnormal event label. The LSTM-based neural network model ensures that when processing the features of each current time window, it includes processing information from all previous time windows, integrating and transmitting the evaluation results of previous time windows, thereby better identifying the dependence of the action on previous states during the completion process. This evaluation result should be submitted to the clinical decision-making interface for clinicians to use in decision-making.

[0040] In one implementation, the data corresponding to the current time window includes a set of electromyographic signals and skeletal animation signals, as well as a set of plantar pressure data.

[0041] Specifically, video signals from normal individuals and patients performing examination tasks need to be collected simultaneously. Neurologists then review the videos and score and label them based on dimensions such as task completion, task completion rating, and abnormal event labeling. The electromyography (EMG) signals, skeletal animation signals, plantar pressure data, and the neurologists' scores and labels are compiled into a training set. Based on the aforementioned model structure, this training set is used for training, and 10-fold cross-validation is employed to verify the model's performance.

[0042] Task Completion Labeling: This refers to the labeling of task completion by professional neurologists based on the patient's actual performance during a specific examination task. The labeling result is between 0 and 1, rounded to one decimal place. Labels marked as 1 are placed at the end of the video segment, and the percentage of completion during the examination task is calculated based on the time taken for the patient to complete the action, i.e., t3 / T, where t3 is any point in time during the patient's performance of the examination task, and T is the time taken to complete the task. For labels not marked as 1, indicating that the patient did not complete the examination task, a label is formed at the point when the doctor determines that the patient cannot complete the action; subsequent time points until the end of the video segment have a completion label of 0. The calculation of completion before the manually labeled point is similar to the above. Task Completion Score: This refers to the quantitative evaluation and scoring of the quality of the patient's completion of the examination task by professional neurologists. The labeling result is between 0 and 1, rounded to one decimal place. Abnormal Event Labeling: Professional neurologists record and label any events or phenomena that deviate from normal conditions during the patient's performance of the examination task. For example, significant tremors, increased muscle tone, and other abnormal events are labeled, along with the time period of the abnormal event.

[0043] In one embodiment, the analysis process further includes calculating the confidence level corresponding to the current time window. When the confidence level is greater than a preset confidence threshold, the inspection task is determined to be completed. The confidence level is calculated as CS_t = α*(task quality confidence level) + β*(timing consistency confidence level) + γ*(signal quality confidence level). In the formula, α, β, and γ are weighting coefficients, and α + β + γ = 1.

[0044] This confidence level is derived from the consistency of the multimodal signal analysis results within the time window and the overall signal quality. Specifically: the task quality confidence level is the task completion score in the evaluation results output by the MLP model, with a score between 0 and 1, reflecting the model's evaluation of the quality of the current action completion. The temporal consistency confidence level calculates the difference between the task completion score corresponding to the current time window and the average task completion score corresponding to the previous N time windows (e.g., 3 time windows). It is used to assess the continuity and stability of the action; the smoother and more continuous the action, the smaller the difference, and the higher this score. The specific calculation formula is as follows: In the formula, C is the time-series consistency confidence level; S0 is the task completion score of the current time window; and N is the number of preceding reference time windows; S iThe task completion score for the i-th preceding time window is calculated (i=1,2,...,N). The signal quality confidence score is calculated based on the signal-to-noise ratio (SNR) of the electromyographic signal and the integrity of the skeletal animation signal (e.g., whether keypoints are missing) within the current time window. For example, in this embodiment, the signal quality confidence score = 0.5 * (SNR_normalized) + 0.5 * (Data_Integrity), where SNR_normalized = 20 × log0 10 (Signal root mean square value / Noise root mean square value), the signal root mean square value is the intensity quantification index of the electromyographic signal during effective activity, calculated as the root mean square of the signal during this period; the noise root mean square value is the background noise intensity of the muscle at rest, taken as the signal root mean square during periods without electromyographic activity; Data_Integrity = number of valid frames / total number of frames, the number of valid data frames is the number of frames that meet all of the following verification conditions: no missing skeletal joint data; compliant signal range, such as joint angles / acceleration within the physiologically reasonable range; signal stability meets the standard, with no logical conflicts or abnormal noise frames in the joint data of the same skeletal chain within a single frame. The specific values ​​of the weight coefficients α, β, and γ can be optimized through the validation set during the model training phase, for example, initially set to α=0.5, β=0.3, γ=0.2 to balance the contribution of different factors. When CS_t is higher than the preset reliability threshold (set to 0.75 in this embodiment), the system can determine that the patient has fully completed the current examination task, thereby automatically terminating the task and improving the evaluation efficiency.

[0045] In one implementation, when the confidence level exceeds a preset confidence threshold, in addition to terminating the currently ongoing examination task, related examination tasks are skipped. In this embodiment, the standing balance test and single-leg hops are defined as related examination tasks. If the patient's confidence level in performing a single-leg hop reaches a preset threshold, the standing balance test can be skipped. Clinically, single-leg hops are generally considered a more advanced and complex movement pattern than the standing balance test, placing higher demands on the subject's physical function. Successful single-leg hops require not only excellent static balance ability (which is the main assessment of the standing balance test), but also strong lower limb muscle strength, joint stability, core control, and sensory integration ability to maintain body stability in dynamic situations. If a patient can complete a single-leg hop with high quality, this strongly demonstrates their ability to perform static standing balance. In terms of movement difficulty, single-leg hops are an "advanced" test of standing balance. More movement association settings can be manually added by clinicians.

[0046] In one embodiment, the preprocessing step of the electromyographic signal during the analysis and processing includes filtering and downsampling to obtain a matrix of dimension C×(f*t2), where C is the number of electromyographic signal channels, f is the frequency of the skeletal animation signal, and t2 is the acquisition time window.

[0047] In one embodiment, the preprocessing step of the skeletal animation signal during the analysis and processing includes: extracting key points from each frame of the skeletal animation signal, with each key point represented by a set of two-dimensional coordinates to form a coordinate matrix of dimension P×2×(f*t2), where P is the number of key points, f is the frequency of the skeletal animation signal, and t2 is the duration of the time window; establishing an adjacency matrix of dimension P×P based on the connection relationship between key points, where P is the number of key points, the matrix elements are defined as the distance between key points with a connection relationship, and the elements corresponding to key points without a connection relationship are set to 0; inputting the coordinate matrix and the adjacency matrix into a Graph Convolutional Neural Network (GCN) model for skeletal modeling, mapping it to a matrix of dimension C×(f*t2), where C is the number of electromyographic signal channels, f is the frequency of the skeletal animation signal, and t2 is the acquisition time window.

[0048] In one implementation, the spliced ​​data is a three-dimensional matrix with dimensions 2×C×(f*t2), where C is the number of electromyographic signal channels, f is the frequency of skeletal animation signals, and t2 is the acquisition time window.

[0049] In one embodiment, during the analysis and processing, the data corresponding to the current time window includes a set of plantar pressure data in addition to a set of electromyographic signals and skeletal animation signals. The plantar pressure data includes plantar pressure distribution, pressure center trajectory, impulse, and gait spatiotemporal parameters. The preprocessing steps for the plantar pressure data include: the plantar pressure distribution is a two-dimensional matrix heatmap sequence, which is convolved into a matrix of dimension (A, f*t2) through a convolutional neural network model; the pressure center trajectory, impulse, and gait spatiotemporal parameters are upsampled and organized into a matrix of dimension (8, f*t2); where A+8=C, and C is the number of electromyographic signal channels; the above data is concatenated with the electromyographic signals and skeletal animation signals, and further organized into a 3×C×(f*t2) three-dimensional matrix concatenated data.

[0050] This invention also provides a multimodal cerebellar function assessment neurological examination system, comprising:

[0051] The task guidance module is used to explain the key points of the examination task to the patient and then guide the patient to perform the examination task; the examination tasks include finger-to-nose test, standing balance test, single-leg hop, and short-distance walking test;

[0052] The multimodal signal acquisition module is used to attach electrode pads to the patient's limbs to acquire electromyographic signals when the patient performs the examination task; it also acquires the patient's three-dimensional posture through an inertial motion capture system to acquire skeletal animation signals when the patient performs the examination task; during the patient's examination task, signals are continuously acquired at fixed time intervals t1, with each acquisition period corresponding to a time window of duration t2, and each time window corresponding to a set of electromyographic signals and skeletal animation signals; acquisition continues until the examination task is completed or the preset time limit is reached;

[0053] The signal processing module is used to analyze and process the electromyographic signals and skeletal animation signals corresponding to each time window in a time sequence; after the signal data corresponding to the previous time window has been analyzed and processed, the signal data corresponding to the next time window will be analyzed and processed.

[0054] The analysis and processing include: preprocessing a set of electromyography (EMG) signals and skeletal animation signals corresponding to the current time window and then splicing them to obtain spliced ​​data; inputting the spliced ​​data into a trained convolutional neural network (CNN) model for feature fusion to obtain a feature vector of length N; inputting the feature vector into a trained long short-term memory (LSTM) based neural network model to obtain the latent feature representation corresponding to the feature vector of the current time window; and using a multilayer perceptron (MLP) based network structure to map this latent feature representation into a 3-dimensional score to obtain the scoring result corresponding to the current time window. The evaluation result includes task completion label, task completion score, and abnormal event label. The LSTM-based neural network model ensures that when processing the features of each current time window, it includes processing information from all previous time windows, integrating and transmitting the evaluation results of previous time windows, thereby better identifying the dependence of the action on previous states during the completion process. This evaluation result should be submitted to the clinical decision-making interface for clinicians to use in decision-making.

[0055] In one implementation, the signal processing module is also used to synchronously map the patient's actual movements to the task guidance module, ensuring that the key points of the examination task movements explained by the task guidance module to the patient are consistent with the patient's actual movements. This function can be achieved through inertial motion compensation skeletal animation, which is existing technology and will not be described further.

[0056] It should be fully understood that the user information involved in this application (including but not limited to user physiological information, user personal information, etc.) is information and data authorized by the user or fully authorized by all parties. The use of user information shall comply with privacy policies and practices that are generally considered to meet or exceed industry or government requirements for maintaining user privacy. The collection, use and processing of related data shall comply with relevant laws, regulations and standards, and provide corresponding operation access points for users to choose to authorize or refuse.

[0057] The above descriptions of the embodiments are merely illustrative of the concept of this application and should not be construed as limiting the scope of protection of this application. It should be understood that the application of this invention is not limited to the examples described above. Those skilled in the art can make improvements or modifications based on the above descriptions, and all such improvements and modifications should fall within the scope of protection of the appended claims.

Claims

1. A multimodal cerebellar functional assessment neurological examination method, characterized in that, Includes the following steps: Explain the key points of the examination task to the patient, and then guide the patient to perform the examination task; the examination task includes finger-to-nose test, standing balance test, single-leg hop, and short-distance walking test; Electrode pads are attached to the patient's limbs to collect electromyographic signals as the patient performs the examination task; the patient's three-dimensional posture is acquired through an inertial motion capture system to collect skeletal animation signals as the patient performs the examination task; during the patient's examination task, signals are continuously collected at fixed time intervals t1, each collection period corresponds to a time window of duration t2, and each time window corresponds to a set of electromyographic signals and skeletal animation signals; the collection continues until the examination task is completed or the preset time limit is reached. The electromyography and skeletal animation signals corresponding to each time window are analyzed and processed sequentially according to the time sequence; the signal data corresponding to the previous time window is analyzed and processed before the signal data corresponding to the next time window is analyzed and processed. The analysis and processing include: preprocessing a set of electromyographic signals and skeletal animation signals corresponding to the current time window and then splicing them to obtain spliced ​​data; inputting the spliced ​​data into a preset model and outputting evaluation results, which include task completion markers, task completion scores, and abnormal event markers.

2. The multimodal cerebellar functional assessment neurological examination method as described in claim 1, characterized in that: The steps for inputting spliced ​​data into a preset model and outputting evaluation results during the analysis and processing include: inputting the spliced ​​data into a convolutional neural network model to perform feature fusion and obtain feature vectors; inputting the feature vectors into a neural network model based on a long short-term memory network to obtain the latent feature representations corresponding to the feature vectors; and using a network structure based on a multilayer perceptron to map the latent feature representations to the evaluation results. When processing the feature vectors of the current time window, the neural network model based on a long short-term memory network integrates the evaluation results of previous time windows.

3. The multimodal cerebellar functional assessment neurological examination method as described in claim 1, characterized in that: The analysis and processing also includes calculating the confidence level corresponding to the current time window. When the confidence level is greater than the preset confidence level threshold, the inspection task is determined to be completed. The confidence level is calculated as follows: CS_t = α * task quality confidence level + β * timing consistency confidence level + γ * signal quality confidence level. In the formula, α, β, and γ are weighting coefficients, and α + β + γ = 1.

4. The multimodal cerebellar functional assessment neurological examination method as described in claim 3, characterized in that: When the confidence level is greater than the preset confidence threshold, in addition to terminating the currently ongoing inspection task, the inspection tasks associated with the currently ongoing inspection task are also skipped.

5. The multimodal cerebellar functional assessment neurological examination method as described in claim 1, characterized in that: In the analysis and processing, the preprocessing steps for the electromyographic signals include filtering and downsampling to obtain a matrix of dimension C×(f*t2), where C is the number of electromyographic signal channels, f is the frequency of the skeletal animation signal, and t2 is the acquisition time window.

6. The multimodal cerebellar functional assessment neurological examination method as described in claim 1, characterized in that: The analysis and processing process includes the following steps for preprocessing the skeletal animation signal: extracting key points from each frame of the skeletal animation signal, with each key point represented by a set of two-dimensional coordinates to form a coordinate matrix of dimension P×2×(f*t2), where P is the number of key points, f is the frequency of the skeletal animation signal, and t2 is the duration of the time window. Based on the connection relationship between key points, an adjacency matrix of dimension P×P is established, where P is the number of key points, the matrix element is defined as the distance between key points with a connection relationship, and the element corresponding to the key point without a connection relationship is 0; the coordinate matrix and the adjacency matrix are input into a graph convolutional neural network model and mapped to a matrix of dimension C×(f*t2), where C is the number of electromyographic signal channels, f is the frequency of skeletal animation signal, and t2 is the acquisition time window.

7. The multimodal cerebellar functional assessment neurological examination method as described in claim 1, characterized in that: The spliced ​​data is a three-dimensional matrix with dimensions 2×C×(f*t2), where C is the number of electromyographic signal channels, f is the frequency of skeletal animation signals, and t2 is the acquisition time window.

8. The multimodal cerebellar functional assessment neurological examination method as described in claim 1, characterized in that: When performing standing balance tests, single-leg hops, and short-distance walking tests, plantar pressure data of the patient is also collected through a plantar pressure acquisition system. The plantar pressure data includes plantar pressure distribution, pressure center trajectory, impulse, and gait spatiotemporal parameters.

9. A multimodal cerebellar functional assessment neurological examination system, characterized in that, include: The task guidance module is used to explain the key points of the examination task to the patient and then guide the patient to perform the examination task; the examination tasks include finger-to-nose test, standing balance test, single-leg hop, and short-distance walking test; The multimodal signal acquisition module is used to attach electrode pads to the patient's limbs to acquire electromyographic signals when the patient performs the examination task; it also acquires the patient's three-dimensional posture through an inertial motion capture system to acquire skeletal animation signals when the patient performs the examination task; during the patient's examination task, signals are continuously acquired at fixed time intervals t1, with each acquisition period corresponding to a time window of duration t2, and each time window corresponding to a set of electromyographic signals and skeletal animation signals; acquisition continues until the examination task is completed or the preset time limit is reached; The signal processing module is used to analyze and process the electromyographic signals and skeletal animation signals corresponding to each time window in a time sequence; after the signal data corresponding to the previous time window has been analyzed and processed, the signal data corresponding to the next time window will be analyzed and processed. The analysis and processing include: preprocessing a set of electromyography (EMG) signals and skeletal animation signals corresponding to the current time window and then splicing them to obtain spliced ​​data; inputting the spliced ​​data into a convolutional neural network model for feature fusion to obtain a feature vector; inputting the feature vector into a neural network model based on a long short-term memory network to obtain the latent feature representation corresponding to the feature vector; using a network structure based on a multilayer perceptron to map the latent feature representation to the evaluation result, which includes task completion label, task completion score, and abnormal event label; when processing the feature vector of the current time window, the neural network model based on a long short-term memory network integrates the evaluation results of previous time windows.

10. A multimodal cerebellar function assessment neurological examination system as described in claim 9, characterized in that: The signal processing module is also used to synchronously map the patient's actual movements to the task guidance module, ensuring that the key points of the examination task explained to the patient by the task guidance module are consistent with the patient's actual movements.

Images