Freezing gait (FOG) assessment system and method for Parkinson's disease patient

Abstract

The invention belongs to the technical field of intelligent medical treatment and computer vision, and discloses a system and method for FOG evaluation of a Parkinson's disease patient, and the system comprises a video collection module; an attitude estimation module; a feature analysis module; an FOG detection module; a result is visualized and connected with a cloud database; according to the method, wearable equipment is not needed, data can be collected only through a common RGB camera, and the method is more suitable for large-scale screening or home environment use and convenient to apply to remote medical treatment and remote areas. A deep learning network architecture of STGCN is used, and a spatial structure (body topology) and a time dynamic state (action change) can be automatically learned from a key point sequence; manual feature construction (such as wavelet transform, statistical analysis and the like) is avoided, and the method is more suitable for complex action recognition; and the method has expandability and can be migrated to other action detection tasks.

Description

Technical Field

[0001] This invention belongs to the fields of intelligent medical and computer vision technology, and in particular relates to a system and method for assessing frozen gait (FOG) in Parkinson's disease patients. Background Technology

[0002] Freezing of gait (FOG) is a common disabling symptom in the middle and late stages of Parkinson's disease (PD). Clinically, it manifests as sudden, brief, intermittent interruptions in gait during walking, where the patient, despite the intention to walk, is unable to produce effective strides. Statistics show that approximately 70% of PD patients experience FOG symptoms in the later stages of the disease, significantly increasing the risk of falls and severely impacting their ability to live independently and their quality of life. Currently, clinical assessment of freezing of gait primarily relies on two traditional methods: one is the use of standardized questionnaires, such as the Freezing of Gait Questionnaire (FOGQ). While simple to use, this method depends entirely on the patient's subjective description, and the assessment results are easily influenced by the patient's cognitive state and recall biases; the other method involves frame-by-frame analysis of video recordings by clinical experts. While relatively objective, this method is extremely time-consuming and labor-intensive, and significant subjective differences may exist between different assessors. In recent years, with the development of wearable sensor technology, automated FOG (Follicular Unit of Gynecology) detection systems based on Inertial Measurement Units (IMUs) have gradually emerged. These systems collect motion data by installing accelerometers and gyroscopes on multiple parts of the patient's body, and then use algorithms to analyze the data to identify FOG events. Although this technology represents a significant improvement over traditional methods, it still has obvious limitations: First, the system requires patients to wear multiple sensor nodes during assessment, which places an additional burden on patients with limited mobility (PD); second, sensors need to be calibrated regularly to ensure data accuracy, increasing the complexity of use; and third, compatibility issues exist between sensor devices from different manufacturers, limiting the widespread application of the system. In the field of computer vision, deep learning-based pose estimation technology has made breakthrough progress in recent years, especially with the emergence of lightweight models such as MediaPipe BlazePose, which makes it possible to achieve high-precision human pose tracking using ordinary cameras. This technology completely eliminates the dependence on wearable devices or markers, requiring only ordinary cameras to extract the three-dimensional coordinate information of key human points in real time, providing a completely new technical path for FOG assessment. A few studies have attempted to apply computer vision technology to FOG detection, but these studies either require complex multi-camera systems or rely on expensive motion capture equipment, making them difficult to promote and apply in ordinary clinical or home settings.

[0003] In existing technologies, a recent study published by PMC proposed a graph-based frozen gait detection method. This method maps human keypoint sequences to a temporal pose graph and classifies them based on the Fréchet mean and variance features of the graph Laplacian matrix. While this method can capture overall pose relationships from videos, it relies solely on global graph structure statistics, making it difficult to reflect short-term, sudden frozen jitter. Furthermore, it is extremely sensitive to keypoint noise and occlusion; if the video lighting or background is complex, the keypoint extraction error increases significantly, leading to a decrease in detection accuracy.

[0004] Furthermore, this method primarily processes pose maps in batches in offline environments, without considering the latency and computational limitations of embedded or cloud-based collaborative computing, and also lacks seamless integration with clinical visualization platforms. In actual industrial deployment, it not only struggles to guarantee real-time performance but also fails to provide therapists with the historical event query and report generation functions they require, thus hindering the application and promotion of this technology in rehabilitation monitoring and telemedicine scenarios. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a system and method for FOG assessment in Parkinson's disease patients.

[0006] This invention is implemented as follows: a system for FOG assessment in Parkinson's disease patients, comprising:

[0007] Video acquisition module, pose estimation module, feature analysis module, FOG detection module, results visualization and cloud database;

[0008] The video acquisition module, connected to the pose estimation module, is used to acquire continuous images of the patient's natural gait using a single camera;

[0009] The pose estimation module, connected to the feature analysis module, is used to extract the coordinates of human key points from the video in real time.

[0010] The feature analysis module, connected to the FOG detection module, is used to construct motion features related to FOG from the extracted keypoint sequence;

[0011] The FOG detection module, connected to the result visualization and cloud database, is used to identify and quantify the frequency, duration and timing of FOG events through preset models or deep learning algorithms.

[0012] Results visualization and cloud database connection, along with the FOG detection module, are used to generate highly readable assessment reports for therapists to assist in clinical decision-making. At the same time, therapists and patients can access their personal databases to view historical assessment records, thereby gaining a comprehensive understanding of rehabilitation progress and treatment effectiveness.

[0013] Another object of the present invention is to provide a method for FOG assessment in patients with Parkinson's disease, comprising:

[0014] Step 1: Equipment Installation and Data Acquisition

[0015] In the data acquisition phase, a standard RGB camera is used to capture video data of the patient completing the Timed Up and Go (TUG) test in the sagittal plane; the acquired video data will be input into the system for processing using the pose estimation module;

[0016] Step 2: Data Calculation

[0017] Multi-level spatiotemporal feature analysis of the raw posture data: First, the kinematic parameters of individual joints are calculated, including the range of motion angles of each joint in three dimensions, changes in angular velocity, and relative positional relationships between joints; second, the overall gait characteristics are quantified, including the dynamic changes of spatiotemporal parameters such as stride length, stride width, and stride duration, as well as the statistical analysis of overall motion indicators such as stride frequency and stride speed.

[0018] Step 3: Classification and Recognition

[0019] The system adopts a neural network architecture based on spatiotemporal graph convolution, inputs the extracted multi-dimensional features into the trained classification model, outputs the probability prediction of FOG events, and makes frame-level judgments, which makes it possible to identify the duration of FOG occurrence.

[0020] Step 4: Storage, Management and Transportation

[0021] After the measurement is completed, the data obtained from this measurement, as well as the patient's physical condition information, number, and other personal information, can be stored together with the FOG assessment results. The rehabilitation therapist can log in to the data storage, management, and transmission system to remotely obtain the patient's assessment data and related information, evaluate the frozen gait of Parkinson's patients, compile a personalized rehabilitation assessment report, and send the assessment report to the patient remotely through the data storage, management, and transmission system for the patient to further strengthen their exercises in a targeted manner.

[0022] Another object of the present invention is to provide a computer device including a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method for FOG assessment of Parkinson's disease patients.

[0023] Another object of the present invention is to provide a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the method for FOG assessment of Parkinson's disease patients.

[0024] Another object of the present invention is to provide an information data processing terminal for implementing the system for FOG assessment of Parkinson's disease patients.

[0025] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0026] First, this invention addresses the technical problems of existing FOG evaluation methods, such as strong subjectivity, reliance on expensive equipment, low evaluation efficiency, and poor real-time performance. It proposes an automated FOG evaluation system and method based on ordinary cameras and deep learning, which successfully achieves real-time identification and quantitative evaluation of FOG events, and has significant innovation and practical value.

[0027] The FOG assessment system proposed in this invention, based on a monocular camera and a lightweight pose estimation algorithm, significantly reduces system complexity and usage costs while maintaining high detection accuracy, thus possessing better clinical applicability and promotional value.

[0028] The system of this invention collects video data of the patient's walking process based on a monocular camera, uses a lightweight human pose estimation algorithm (such as MediaPipe BlazePose) to extract the three-dimensional coordinates of the patient's key skeletal points in real time, and combines the temporal feature analysis module to identify FOG events and output evaluation results.

[0029] This invention proposes an automated FOG (Folling Organisation) evaluation system based on computer vision. The system acquires video data using a monocular camera and combines pose estimation algorithms and deep learning models to achieve non-contact, high-precision FOG event recognition. Addressing the issues of stage interference and boundary ambiguity caused by the natural transitions, continuity, and overlap of sub-stages in TUG (Turn-Off) testing, a deep learning algorithm with temporal modeling capabilities is designed, effectively improving the accuracy of FOG evaluation. Simultaneously, a database system integrating data storage, transmission, display, management, and analysis has been developed, overcoming the shortcomings of existing systems in recording and managing evaluation results. This invention provides a convenient and efficient solution for the objective evaluation of FOG and its subsequent data processing.

[0030] Step 1: The lightweight pose estimation algorithm, MediaPipe BlazePose, is employed, enabling real-time skeletal keypoint extraction on ordinary cameras and low-computing-power devices. Compared to traditional IMU sensors requiring wearable devices and methods like OpenPose, which are computationally complex and demanding, BlazePose offers greater portability and versatility. This makes FOG monitoring possible in clinical and home environments.

[0031] Steps 2 & 3: Addressing the challenges of phase segmentation and FoG recognition arising from the natural flow of actions and lack of obvious pauses (i.e., the end of one action immediately marks the beginning of the next, lacking clear cut-off points or landmark events) in TUG testing, the system designs two collaborative modules: multi-level spatiotemporal feature calculation (Step 2) and temporal modeling and phase recognition based on Spatio-Temporal Graph Convolution Network (STGCN) (Step 3), jointly solving the following three key problems:

[0032] 1. Difficulty in dividing stages due to the continuity of actions

[0033] The transitions between TUG action phases (such as standing up, walking, and turning) are seamless and have blurred boundaries, making it difficult to accurately identify the timing of transitions. The system first extracts key spatiotemporal features through multi-level kinematic analysis and establishes a phase transition model. Based on this, the STGCN architecture is introduced to model the time-series data. Combined with post-processing strategies (such as phase sequence constraints and smoothing rules), the accuracy of phase division is significantly improved.

[0034] 2. FoG features are not easily observed or identified directly.

[0035] FoG (FoG) events are often accompanied by subtle angular velocity anomalies, abrupt gait changes, pauses, and abnormal deceleration during turns, making them difficult to detect using traditional methods. This system extracts fine-grained features such as joint angle changes, angular velocities, and relative positions, and leverages STGCN's joint modeling capabilities in spatial nodes and temporal dynamics to more sensitively identify subtle motion characteristics preceding FoG, thereby improving the sensitivity and early warning capabilities of FoG detection.

[0036] 3. Lack of feature modeling for high-risk action segments

[0037] Clinical studies have shown that FoG (Forward Gestational Gastrointestinal) often occurs during transitional movements such as "standing up-starting" or "turning around." Therefore, the system divides TUG (Transitional Gestational Gastrointestinal ...

[0038] Step 4: The data storage, management, and transmission system of this invention enables physical therapists to remotely access and assess patient evaluation data, improving usability in remote, clinical, and home environments, increasing work efficiency, and reducing expenses for patients and social rehabilitation services. Data storage, management, and transmission facilitate the management of patient data, preventing data loss and other disruptions to rehabilitation progress, and enabling long-term tracking and personalized rehabilitation plan development.

[0039] Compared to similar invention patents:

[0040] Compared to "A method, apparatus, and storage medium for frozen gait detection based on random forest algorithm" and "An apparatus and method for preventing and relieving frozen gait in Parkinson's disease patients," this invention requires no wearable devices; data can be collected using only a regular RGB camera. It is more suitable for large-scale screening or home environments, and facilitates applications in telemedicine and remote areas. Using the STGCN deep learning network architecture, it can automatically learn spatial structure (body topology) and temporal dynamics (action changes) from keypoint sequences; avoiding manual feature construction (such as wavelet transform, statistical analysis, etc.), it is more suitable for complex action recognition; and it is more scalable, transferable to other action detection tasks.

[0041] Compared to the "method of establishing a frozen gait recognition model based on machine vision," the MediaPipe used in this invention is lighter and more suitable for running on mobile phones or low-end devices; it has a faster extraction speed and more stable operation; and it is suitable for large-scale deployment and remote monitoring. The STGCN model can simultaneously model spatial skeleton structure and temporal dynamic features; it is better at detecting complex and subtle motion changes, such as pre-FoG micro-motion trends; and it can be used for frame-by-frame FoG stage segmentation, resulting in more accurate recognition. The constructed database better meets clinical needs, laying the foundation for clinical auxiliary diagnosis and remote rehabilitation platforms.

[0042] Secondly, this invention's system utilizes a standard RGB camera and deep learning algorithms to assess frozen gait (FOG) in Parkinson's disease patients, offering advantages such as low equipment cost, ease of operation, and remote deployment. This system can be widely applied in settings such as nursing homes, rehabilitation centers, and hospitals, and is expected to significantly reduce the reliance on professional personnel and expensive equipment in traditional assessments, lower clinical assessment costs, and improve the timeliness and personalization of rehabilitation interventions. With the continued growth in the number of Parkinson's patients worldwide, this system has considerable market demand and commercial prospects, and is expected to form a closed-loop business model of "assessment services + cloud system + remote rehabilitation," generating sustainable revenue.

[0043] FOG (Foreign Object Gaits) is one of the most disabling symptoms of Parkinson's disease. Its uncertainty and transient nature make it difficult to accurately identify and quantify with traditional assessment methods, which has long plagued clinicians and patient management. This invention is the first to achieve automatic identification of FOG events using video + AI, and can output key indicators such as duration, frequency, and timing. It overcomes the pain points of previous assessment methods, such as lack of objective quantification and inability to continuously monitor, providing a reliable basis for the development of personalized rehabilitation plans and solving the long-standing clinical technical challenges of "difficult FOG identification and data analysis".

[0044] For a long time, FOG assessment has mainly relied on wearable sensors or subjective observation by clinicians. This approach has led to the inherent cognitive bias that "FOG cannot be assessed without wearable devices" and "expert judgment is essential." This invention breaks through traditional thinking by using a monocular camera combined with posture estimation and a graph neural network model for the first time to achieve non-contact, automated, and low-cost FOG identification and quantification. It breaks the industry's technical bias that "FOG cannot be objectively identified through visual means," and verifies that video-based gait feature extraction can also achieve a certain level of assessment accuracy, thus expanding the technical approach for assessing Parkinson's disease movement disorders. Attached Figure Description

[0045] Figure 1 This is a system structure diagram for FOG assessment of Parkinson's disease patients provided in an embodiment of the present invention.

[0046] Figure 2 This is a flowchart of a method for FOG assessment in Parkinson's disease patients provided in an embodiment of the present invention.

[0047] Figure 3 This is a schematic diagram of device connection provided in an embodiment of the present invention.

[0048] Figure 4 This is a schematic diagram of the implementation process provided in the embodiments of the present invention.

[0049] Figure 5 The comparison of frame count error (MAE) in each sub-stage of TUG for different models is shown.

[0050] Figure 1 The system comprises: 1. Video acquisition module; 2. Pose estimation module; 3. Feature analysis module; 4. FOG detection module; and 5. Result visualization and cloud database. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0052] In traditional clinical assessments, the identification of frozen gait (FOG) in Parkinson's disease patients relies on manual observation and subjective scoring, which makes continuous monitoring and quantitative analysis difficult. Furthermore, limitations imposed by the assessment environment and observer experience hinder the guarantee of sufficient reliability and consistency in industrial applications. To overcome these bottlenecks, this system, based on monocular video, utilizes an end-to-end video acquisition and processing pipeline to ensure the entire assessment process can be embedded into industrial scenarios such as rehabilitation robots and intelligent monitoring platforms, achieving cross-environment and cross-device versatility.

[0053] During system startup, the video acquisition module uses a high dynamic range (HDR) camera to capture the patient's natural gait at 30fps, reserving interfaces for image distortion and illumination adaptive correction. The pose estimation module, based on a lightweight convolutional neural network (MobileNetV3+), combined with spatiotemporal consistency constraints, performs regression estimation on key human joints (hip, knee, ankle, etc.) in each frame of the image, and uses nonlinear low-pass filtering to remove jitter noise, ensuring the temporal accuracy of subsequent feature analysis.

[0054] In the process of constructing motion features, the feature analysis module integrates temporal normalization processing and multi-resolution Fourier transform to extract multi-dimensional indicators such as stride period, joint angle change rate, and bipedal support phase ratio from the joint point trajectory. These are then combined with entropy weighting to generate a unified feature vector. This process abandons traditional single statistical measures and adopts a combination of wavelet packet decomposition and time-frequency analysis to improve sensitivity to short-term frozen states. Furthermore, redundant information is eliminated through mutual information evaluation, enhancing the overall discriminative power of the features.

[0055] The FOG detection module deploys a hybrid architecture that integrates a deep residual network (ResNet-1D) and a bidirectional long short-term memory network (BiLSTM) on top of the feature vectors. On one hand, it uses residual branches to capture instantaneous feature mutations; on the other hand, it mines the dynamic correlation before and after freezing through temporal memory, ultimately outputting the freezing probability curve. During network training, a composite loss function combining cross-entropy and focal loss is introduced to address the class imbalance problem, and leave-one-out cross-validation ensures the model's robustness on multi-center data.

[0056] The results visualization and cloud database module utilizes a microservice architecture based on the front-end React framework and the back-end Kubernetes orchestration. It presents test results as a combination of heatmaps and time-series curves, and supports dual export in JSON format and the DICOM standard. Therapists can retrieve patients' past freeze events in real time through an access-controlled web interface and generate scale reports based on normalized Z-scores to assist clinical decision-making. Simultaneously, data is encrypted using TLS 1.3 and AES-256 during transmission and storage, achieving HIPAA-compliant security.

[0057] The embedded deployment of the entire process in medical devices and rehabilitation equipment has undergone collaborative testing between edge computing units and the cloud, ensuring real-time performance and stability under network jitter and computing power constraints. This system not only provides a non-invasive and quantifiable technical path for standardized FOG assessment but also lays a solid foundation for subsequent remote monitoring and personalized rehabilitation program development.

[0058] like Figure 1 As shown in the figure, an embodiment of the present invention provides a system for assessing frozen gait (FOG) in Parkinson's disease patients, comprising:

[0059] Video acquisition module 1, pose estimation module 2, feature analysis module 3, FOG detection module 4, result visualization and cloud database 5;

[0060] The video acquisition module 1, connected to the pose estimation module 2, is used to acquire continuous images of the patient's natural gait using a single camera;

[0061] Pose estimation module 2, connected to feature analysis module 3, is used to extract the coordinates of human key points from the video in real time;

[0062] Feature analysis module 3, connected to FOG detection module 4, is used to construct motion features related to FOG from the extracted keypoint sequence;

[0063] The FOG detection module 4, connected to the result visualization and cloud database 5, is used to identify and quantify the frequency, duration and timing of FOG events through preset models or deep learning algorithms.

[0064] Results visualization and cloud database 5, connected to FOG detection module 4, are used to generate highly readable assessment reports for therapists to assist in clinical decision-making; at the same time, it supports therapists and patients to access personal databases to view historical assessment records, thereby gaining a comprehensive understanding of rehabilitation progress and treatment effects.

[0065] The system first completes data acquisition and pose reconstruction in real time on the local terminal: the video acquisition module 1 continuously captures RGB image streams during the patient's natural walking process at a frame rate of 25–30 fps, and synchronizes timestamps; the pose estimation module 2 uses a lightweight convolutional network combined with temporal optimization (such as HRNet-Lite+TCN post-processing) to output 2D / 3D human keypoint coordinate sequences in real time at the edge side, ensuring millisecond-level latency and joint recognition accuracy. After coordinate normalization and noise filtering, the skeleton data of each frame is packaged into a structured time series and enters the subsequent feature analysis process.

[0066] Feature analysis module 3 performs kinematic modeling on the keypoint sequence within a sliding window, extracting frozen gait high-correlation indicators, including stride length coefficient of variation, instantaneous gait frequency fluctuation, bipedal phase ratio, ankle vertical displacement amplitude, and the classic FOG index (toe-Achilles tendon angle frequency domain energy ratio). These multidimensional features are then fused and input into FOG detection module 4: this module loads a pre-trained dual-channel temporal network (CNN-BiLSTM or TCN-Transformer architecture), one branch processes low-dimensional handcrafted features, and the other branch processes the original skeleton sequence. Attention fusion is used to determine whether FOG occurs within the window, and the system outputs the event start / end frame, duration, and confidence score. The system also counts the FOG frequency per unit distance or unit time to achieve quantitative evaluation.

[0067] Upon detecting a FOG event or completing an assessment, Module 4 pushes the results along with the original features to the results visualization and cloud database 5. This module automatically generates interactive reports for therapists: FOG segments are marked on a timeline, the moment of freeze is recreated with thermal skeletal animation, and gait parameters, event statistics, and risk levels are listed. All data is encrypted and stored in the cloud by patient ID, supporting cross-device access, longitudinal trend comparison, and large-scale group analysis. Therapists can remotely view historical records, develop training plans, or adjust medication dosages; patients can also understand their rehabilitation progress through a mobile application, achieving an integrated closed loop of clinical decision-making, rehabilitation guidance, and remote monitoring.

[0068] like Figure 2As shown in the figure, an embodiment of the present invention provides a method for assessing frozen gait in Parkinson's disease patients, comprising:

[0069] S101: Equipment Installation and Data Acquisition

[0070] During the data acquisition phase, a standard RGB camera was used to capture video data of the patient completing the TUG test in the sagittal plane. The acquired video data was then input into the system for processing using a pose estimation module. This module employs an optimized MediaPipe BlazePose algorithm, a lightweight convolutional neural network model designed specifically for mobile devices, which can achieve real-time human pose estimation while maintaining high accuracy. The algorithm identifies the three-dimensional coordinate information of 33 key anatomical landmarks in the video frame by frame, including important locations such as the hip, knee, ankle, shoulder, and elbow joints, forming a complete kinematic data sequence.

[0071] S102: Data Calculation

[0072] Multi-level spatiotemporal feature analysis is performed on the raw posture data: First, the kinematic parameters of individual joints are calculated, including the range of motion angles, angular velocity changes, and relative positional relationships between joints in three dimensions; second, the overall gait characteristics are quantified, including the dynamic changes of spatiotemporal parameters such as stride length, stride width, and stride duration, as well as the statistical analysis of overall motion indicators such as stride frequency and stride speed; particularly important is the specific analysis of gait transitions and turning movements, because clinical studies have shown that these movements are most likely to induce FOG events; the system divides the TUG test into seven characteristic phases (sitting, standing, forward walking, first turn, returning walking, second turn, and sitting down), calculates the characteristic parameters of each phase, and establishes a transition feature model between phases;

[0073] S103: Classification and Recognition

[0074] The system employs a neural network architecture based on STGCN, inputting extracted multi-dimensional features into a pre-trained classification model to output probability predictions of FOG events, performing frame-level judgments and enabling the identification of the duration of FOG occurrences. The model's training data comes from standard TUG test videos of 30 clinically diagnosed PD patients (including those with and without FOG symptoms). All video data was independently labeled by at least two movement disorder specialists to ensure the reliability of the supervised learning dataset labels. To verify the system's clinical applicability, a multi-level validation scheme was designed: first, the model's classification performance metrics (such as accuracy, sensitivity, and specificity) were evaluated using a hold-out method; second, the system output results were correlated with standard FOGQ scores and gold standard video analysis results; and finally, a clinical applicability questionnaire was used to assess the acceptance of the system by healthcare professionals and patients. The entire system adopts a modular design, allowing for flexible configuration according to different application scenarios. It can be used as an independent clinical assessment tool or integrated into a remote medical monitoring platform for long-term FOG monitoring in the home environment.

[0075] S104: Storage, Management and Transportation

[0076] After the measurement is completed, the data obtained from this measurement, as well as the patient's physical condition information, number, and other personal information, can be stored together with the FOG assessment results. The rehabilitation therapist can log in to the data storage, management, and transmission system to remotely obtain the patient's assessment data and related information, evaluate the frozen gait of Parkinson's patients, compile a personalized rehabilitation assessment report, and send the assessment report to the patient remotely through the data storage, management, and transmission system for the patient to further strengthen their exercises in a targeted manner.

[0077] In the computer device of this embodiment, the processor (which may be a CPU with an integrated GPU / VPU or a standalone NPU acceleration unit) is connected to high-speed DDR memory, non-volatile memory, and various I / O interfaces (camera MIPI / USB, wireless Wi-Fi / BLE, Gigabit Ethernet, etc.) via a system bus. When the FOG evaluation application in the computer-readable storage medium is loaded, the startup routine first deploys the depth pose estimation model and the timing detection model in memory, and initializes the signal buffer, task scheduler, and encrypted communication stack, laying the foundation for the continuous acquisition-inference-analysis process.

[0078] During program execution, camera video streams are directly written to the frame buffer via DMA; the processor invokes a multi-threaded pipeline to feed the RGB image into a lightweight pose estimation network running on the NPU / GPU, outputting keypoint coordinates in real time. The main thread uses a high-precision hardware timer to inject a uniform timestamp into each frame and maintains the skeleton sequence in a shared memory circular buffer to ensure that subsequent modules obtain a synchronized and frame-free data stream.

[0079] Feature analysis and FOG detection are performed collaboratively by hardware and software: the processor executes sliding window statistics (low-dimensional indicators such as step size, step frequency, and dual-support phase), while simultaneously feeding the original skeleton sequence into the TCN-Transformer temporal network branch; the two feature streams are combined at the fusion layer to obtain the FOG confidence, start and end frames, and duration. The detection results are written to a relational data table and pushed to a local or cloud database via an encrypted REST / gRPC interface; if the device is offline, the results are cached in local key-value storage and asynchronously synchronized after the network is restored to ensure data integrity.

[0080] Information data processing terminals (such as tablets, desktop workstations, or smart wearable devices) embed the same computer program, which can choose between local full-process inference or hybrid cloud-edge collaborative mode depending on the hardware computing power: in resource-constrained scenarios, only skeleton extraction is performed and features are uploaded to the cloud, where the cloud GPU cluster completes the detection and visualization rendering; in hospital or laboratory environments, all calculations can be completed locally and interactive reports can be generated instantly. The terminal UI is implemented based on a cross-platform framework, supporting 3D skeleton playback, trend charts, and event heatmaps, and providing doctors with access to parameter threshold adjustment and model OTA updates, forming a closed-loop working principle system integrating "program—hardware—cloud".

[0081] Figure 3 This is a diagram showing the device connections.

[0082] Figure 4 This is a schematic diagram of the implementation process.

[0083] The system for assessing frozen gait (FOG) in Parkinson's disease patients described in this invention is suitable for scenarios such as neurorehabilitation, movement disorder clinics, community rehabilitation centers, geriatric departments, and telemedicine platforms, and can be embedded into the following products to achieve specific applications:

[0084] Intelligent Rehabilitation Assessment System: This invention is embedded in a camera-based motor function assessment terminal, and in conjunction with TUG testing, it enables automatic identification and assessment report generation of FOG events, which can be used for screening and follow-up management of Parkinson's patients.

[0085] Remote Home Rehabilitation Platform: This invention can be deployed as a lightweight assessment tool on home terminals or mobile apps. Patients can complete exercise assessments at home using a regular camera and upload the data to the cloud. Rehabilitation therapists can remotely view the results and develop training plans.

[0086] It can be extended to other auxiliary diagnostic systems for neurological diseases, such as multiple system atrophy and progressive supranuclear palsy, which are disease assessment tools that exhibit frozen gait, and has broad clinical application value.

[0087] To verify the effectiveness of our system in recognizing motor phases and frozen gait (FOG) events in Parkinson's disease patients during the Timed Stand-Up-Walk (TUG) test, we constructed a phase recognition model using a deep learning approach based on a spatiotemporal graph convolutional network (ST-GCN). This model takes the keypoint coordinate sequence extracted by MediaPipe as input and automatically divides the entire TUG process into several key phases, including: standing up, walking forward, turning around, returning, and sitting down. A multi-layer STGCN network architecture is employed, combining temporal convolution and graph convolution for feature extraction. Each frame's corresponding phase is manually labeled by experts, and cross-validation (e.g., Leave-One-Subject-Out) is used to evaluate generalization performance. Results show that STGCN demonstrates superior frame-level recognition accuracy across TUG sub-phases, significantly outperforming traditional BiLSTM, Dilated TCN, and EDGCN models, proving its advantages in spatiotemporal feature modeling. Figure 5 The comparison of frame count error (MAE) in each sub-stage of TUG for different models is shown.

[0088] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A system for assessing frozen gait in Parkinson's disease patients, characterized in that, It includes a video acquisition module, a pose estimation module, a feature analysis module, a FOG detection module, as well as result visualization and a cloud database; The video acquisition module is connected to the pose estimation module to acquire continuous images of the patient's natural gait. The pose estimation module is connected to the feature analysis module and is used to extract the coordinates of human key points from continuous images in real time. The feature analysis module connects to the FOG detection module and is used to generate motion features based on key point coordinates. The FOG detection module connects the results visualization to a cloud database to identify and quantify the frequency, duration, and timing of frozen gait events. Results visualization and a cloud database are used to store the event data and generate evaluation reports.

2. The system according to claim 1, characterized in that, The attitude estimation module employs a convolutional neural network incorporating spatiotemporal consistency constraints to improve the accuracy and robustness of keypoint localization.

3. The system according to claim 1, characterized in that, The feature analysis module constructs multidimensional feature vectors based on wavelet packet decomposition and entropy weighting to improve the detection sensitivity of short-term frozen gait states.

4. A method for assessing frozen gait in Parkinson's disease patients, characterized in that, Includes the following steps: Step 1: Use a monocular camera to capture video of the patient's natural gait; Step 2 involves performing human keypoint estimation on the acquired video to obtain a keypoint sequence; Step 3 calculates stride period, joint angular velocity, and bipedal support duration based on key point sequence to obtain motion characteristics; Step 4 inputs the motion features into the trained deep learning model, outputs the probability distribution of frozen gait events, and quantifies the frequency and duration of the events; Step 5 involves linking the quantitative results with the patient's identity information, storing them in a cloud database, and generating a visual assessment report.

5. The method according to claim 4, characterized in that, The deep learning model is trained using leave-one-out cross-validation to evaluate its generalization performance on multi-center datasets.

6. A computer-readable storage medium, characterized in that, The medium stores instructions that, when executed by a processor, cause the processor to perform steps 1 to 5 of the method described in claim 4.

7. A human posture extraction device, characterized in that, It includes an image preprocessing unit, a joint regression unit, and a temporal filtering unit; The image preprocessing unit is used to perform illumination equalization and distortion correction on the input video frames; The joint regression unit is used to extract the coordinates of key points in the human body based on a lightweight neural network; The timing filtering unit is used to perform nonlinear low-pass filtering on the coordinate sequence of key points on the human body; The units are connected via a high-speed bus to enable end-to-end real-time processing.

8. The apparatus according to claim 7, characterized in that, The data transmission delay of the high-speed bus does not exceed 1 millisecond.

9. An information data processing terminal, characterized in that, The terminal includes a processor, a memory, and a display screen. The memory is pre-installed with the application program of the system of claim 1. When the processor runs the application program, it performs image acquisition, pose estimation, feature analysis, frozen gait detection, and report generation, and outputs the evaluation results on the display screen.

10. The information data processing terminal according to claim 9, characterized in that, The terminal uses an encryption protocol that complies with medical data security standards to transmit and store the evaluation results, and implements hierarchical control over data queries according to access permissions.

Images