Alzheimer's disease early warning system and method based on gait video

Abstract

The invention discloses an Alzheimer's disease early warning system and method based on a gait video, and the system comprises a data collection and labeling module which is used for carrying out the standardized collection and labeling of the gait video based on a unified gait video collection device; the data preprocessing module is used for preprocessing the gait video data through a gait video segmentation technology to obtain a standard gait video; the human body key point detection module is used for carrying out human body key point detection based on the processed video and extracting space, time and variability features of the gait video based on the human body key points; and the feature classification and early warning module is used for carrying out classification based on the extracted features and a video classification network so as to carry out Alzheimer's disease detection and early warning. The gait video data is acquired based on the unified gait video acquisition device, and the alzheimer's disease early warning is realized by using the gait video segmentation and human body key point detection technology, so that the alzheimer's disease early warning accuracy is improved, and the cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to an Alzheimer's disease early warning system and method based on gait video. Background Art

[0002] Alzheimer's disease is the most common type of dementia. Currently, there is no cure for Alzheimer's disease, the therapeutic effects of existing drugs are limited, and the development of new drugs has always been difficult. Since the beginning of this century, major pharmaceutical companies have invested more than $200 billion in the development of Alzheimer's drugs, but with little success. This shows that there are still great deficiencies in the understanding of Alzheimer's disease in the past, and it is necessary to challenge the treatment of Alzheimer's disease from a new perspective and direction. Safe and effective non-drug interventions, such as non-invasive neuromodulation technology and acupuncture therapy, play an important role in delaying the progression of the disease and improving prognosis, and have become the focus of academic and clinical attention.

[0003] Non-invasive neuromodulation is considered to be the future of treatment for Alzheimer's disease and other functional brain diseases. The occurrence of such complex brain diseases is related to the dysfunction of specific neural circuits in the brain. Neuromodulation technology can achieve regulation from molecular cells to neural circuits to overall behavioral functions by stimulating corresponding targets, thereby alleviating or improving clinical symptoms. Transcranial magnetic stimulation and transcranial electrical stimulation are currently the main non-invasive control methods, but the spatial resolution of stimulation of these two methods is low and they cannot act on deep brain areas. Ultrasound, as a new type of physical control tool, provides a new treatment method and solution for Alzheimer's disease. Ultrasound has the advantages of being non-invasive, deep penetrating, and highly targeted. Combined with real-time guidance and monitoring of high-field magnetic resonance images, it can focus on the relevant functional nuclei and circuits in the skull for precise and safe stimulation, achieving higher precision and personalized treatment.

[0004] However, current Alzheimer's disease detection methods often rely on expensive testing equipment, special testing environments, and professional medical personnel, which often gives Alzheimer's disease detection the disadvantages of high time cost, high inspection expenses, and difficulty in census. With the development of image perception technology and machine learning, a gait analysis method based on computer big data has been proposed. Computer-based gait diagnosis uses machine learning models to analyze a large number of gait videos for disease classification, thereby achieving early warning and detection of Alzheimer's disease. Computer gait diagnosis has the advantages of being non-destructive, painless, and non-invasive, and can automatically, objectively, and conveniently complete the goals of Alzheimer's disease warning and diagnosis.

[0005] The existing Alzheimer's disease early warning methods based on gait recognition, on the one hand, are not objective and difficult to quantify based on symptoms, and the early symptoms of Alzheimer's disease may confuse it with other similar diseases. In addition, the Alzheimer's disease early warning based on symptoms relies on the experience and clinical observation of relevant medical personnel, so it is costly, time-consuming, and not accurate and objective enough; on the other hand, methods such as neurological function assessment, cranial MRI, and tracer positron emission tomography (PET) scanning of beta-amyloid plaques or dopamine transporters have a good effect on Alzheimer's disease early warning. However, neurological function assessment and cranial MRI rely on professional equipment and medical teams, making it difficult to conduct efficient and low-cost Alzheimer's disease screening and early warning, and are not suitable for Alzheimer's disease detection in remote or underdeveloped areas.

[0006] Therefore, the prior art needs to be improved. Summary of the invention

[0007] The technical problem to be solved by the present invention is that, in view of the defects of the prior art, the present invention provides an Alzheimer's disease early warning system and method based on gait video to solve the problems that the existing Alzheimer's disease early warning method based on gait recognition is not accurate and objective and has high cost.

[0008] The technical solution adopted by the present invention to solve the technical problem is as follows: In a first aspect, the present invention provides an Alzheimer's disease early warning system based on gait video, comprising: A data collection and annotation module is used to perform standardized collection and annotation processing of gait videos based on a unified gait video collection device to obtain gait video data; A data preprocessing module, used to preprocess the gait video data by using a gait video segmentation technology to obtain a video that only retains the subject's standard gait collection process actions; A human key point detection module is used to detect human key points based on the processed video, and to extract spatial features, temporal features, and variability features of the gait video based on the human key points; The feature classification and early warning module is used to perform classification based on the spatial features, the temporal features, the variability features and the video classification network to perform Alzheimer's disease detection and early warning.

[0009] In one implementation, the data collection and annotation module includes: A unified gait acquisition unit, used for performing standardized acquisition of gait videos according to preset gait movements based on the unified gait video acquisition device; A quality evaluation unit is used to evaluate the quality of the collected gait video data according to the clarity of the gait video, the standard degree of the patient's action and the number of invalid video frames, so as to obtain gait video data that meets the conditions; The data annotation unit is used to perform Alzheimer's disease analysis on each subject to obtain pure gait video data annotation.

[0010] In one implementation, the data preprocessing module includes: The action frame selection unit is used to delete frames in the gait video data that are not related to the subject's actions, and to delete videos with preset video quality and preset label purity, so as to obtain the video that only retains the subject's actions in the standard gait collection process.

[0011] In one implementation, the human body key point detection module includes: A key point extraction unit, used for performing human key point detection based on the processed video using a human key point detection algorithm to extract arm key points and leg key points; The multimodal feature extraction unit is used to extract the spatial features, temporal features and variability features of the gait video according to the extracted arm key points and leg key points.

[0012] In one implementation, the feature classification and early warning module includes: A video classification unit is used to directly splice the extracted multiple gait features on the final fully connected layer features of the contrast graph convolutional network based on the spatial features, the temporal features and the variability features to perform video classification and obtain a video classification result of Alzheimer's disease; wherein the positive class is a video of an Alzheimer's positive sample, and the negative class is a video of an Alzheimer's negative sample; The early warning unit is used to output the Alzheimer's disease test results and provide early warning for Alzheimer's disease.

[0013] In a second aspect, the present invention provides an Alzheimer's disease early warning method based on gait video, comprising: Based on a unified gait video acquisition device, gait video is collected and annotated in a standardized manner to obtain gait video data; Preprocessing the gait video data by using gait video segmentation technology to obtain a video that only retains the subject's standard gait acquisition process actions; Detect human key points based on the processed video, and extract spatial features, temporal features, and variability features of the gait video based on the human key points; Classification is performed based on the spatial features, the temporal features, the variability features and the video classification network to perform Alzheimer's disease detection and early warning.

[0014] In one implementation, the standardized acquisition and annotation processing of gait video based on a unified gait video acquisition device to obtain gait video data includes: Based on the unified gait video acquisition device, standardized gait video acquisition is performed according to preset gait movements; According to the gait video clarity, the standard degree of the patient's action and the number of invalid video frames, the quality of the collected gait video data is evaluated to obtain gait video data that meets the conditions; Alzheimer's disease analysis was performed on each subject to obtain clean gait video data annotations.

[0015] In one implementation, the gait video data is preprocessed by using a gait video segmentation technology to obtain a video that only retains the subject's standard gait collection process action, including: Frames irrelevant to the subject's actions in the gait video data are deleted, and videos with preset video quality and preset label purity are deleted to obtain the video that only retains the subject's standard gait collection process actions.

[0016] In one implementation, the detecting of human key points based on the processed video and extracting spatial features, temporal features and variability features of the gait video based on the human key points include: Based on the processed video, the human key point detection algorithm is used to detect the human key points and extract the arm key points and leg key points; Based on the extracted arm key points and leg key points, the spatial features, temporal features and variability features of the gait video are extracted.

[0017] In one implementation, the classification based on the spatial feature, the temporal feature, the variability feature and the video classification network to perform Alzheimer's disease detection and early warning includes: Based on the spatial features, the temporal features, and the variability features, the extracted multiple gait features are directly spliced ​​on the final fully connected layer features of the contrast graph convolutional network to perform video classification, thereby obtaining a video classification result of Alzheimer's disease; wherein the positive class is a video of an Alzheimer's positive sample, and the negative class is a video of an Alzheimer's negative sample; Output Alzheimer's disease test results and provide Alzheimer's disease warning.

[0018] The present invention adopts the above technical solution to achieve the following effects: The present invention performs standardized acquisition and annotation processing of gait videos based on a unified gait video acquisition device, and pre-processes gait video data through gait video segmentation technology, so as to obtain a video that only retains the standard gait acquisition process of the subject; and performs human key point detection based on the processed video, and extracts the spatial features, temporal features and variability features of the gait video based on the human key points; and can classify according to the extracted features and the video classification network to achieve Alzheimer's disease detection and early warning. The present invention obtains high-quality, standardized gait video data, and uses gait video segmentation and human key point detection technology to achieve Alzheimer's disease early warning with a machine learning classification model, thereby improving the accuracy of Alzheimer's disease early warning and reducing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0020] Figure 1 Schematic diagram of the Alzheimer's disease early warning system based on gait video in the present invention.

[0021] Figure 2 It is a flow chart of the Alzheimer's disease early warning method based on gait video in the present invention.

[0022] Figure 3 It is a schematic diagram of key point detection of the human body in the present invention.

[0023] Figure 4 It is a functional principle diagram of a terminal in one implementation of the present invention.

[0024] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0026] Exemplary Devices At present, the clinical detection of Alzheimer's disease is mainly based on symptom manifestations, neurological function assessment and cranial MRI, as well as tracer positron emission tomography (PET) scanning of beta-amyloid plaques or dopamine transporters, but lacks relatively fast and accurate early screening methods, which is especially unsuitable for early identification of Alzheimer's patients in community and other grassroots hospitals. In recent years, many scholars have conducted a lot of research on the early warning and detection of Alzheimer's disease, and these studies have achieved corresponding results in understanding, early warning, detection, and treatment of Alzheimer's disease. Therefore, the Alzheimer's disease early warning method based on gait recognition has great potential. However, the existing methods have the following problems: 1) Alzheimer's disease warning based on symptoms is not objective and difficult to quantify, and the early symptoms of Alzheimer's disease may cause it to be confused with other similar diseases. In addition, Alzheimer's disease warning based on symptoms relies on the experience and clinical observation of relevant medical personnel, so it is costly, time-consuming, and not accurate and objective enough.

[0027] 2) Neurological function assessment, cranial MRI, positron emission tomography (PET) scanning of tracers such as amyloid beta plaques or dopamine transporters, etc. have good warning effects on Alzheimer's disease. However, neurological function assessment and cranial MRI rely on professional equipment and medical teams, making it difficult to conduct efficient and low-cost screening and early warning of Alzheimer's disease. They are also not suitable for Alzheimer's disease detection in remote or underdeveloped areas.

[0028] 3) Existing AI-based medical methods lack objective research on Alzheimer's disease, and lack dedicated collection and detection equipment for gait video information, and the level of automation is not high. When collecting samples from patients, some people are unable to use the equipment according to the operating instructions, which will cause some samples to be defective. In addition, the relevant gait video information perception standards are imperfect, making it impossible to collect high-quality gait video data for Alzheimer's disease warning.

[0029] 4) The existing computer video processing algorithms and the acquisition and preprocessing methods of computer-based dynamic gait videos are not perfect. Therefore, the gait dynamic and static feature extraction methods, feature learning methods, and model training methods for Alzheimer's disease are not perfect, and it is urgent to study automated, dynamic and static feature-containing, efficient, and objective Alzheimer's disease early warning methods based on gait video processing.

[0030] In response to the above technical problems, an embodiment of the present invention provides an Alzheimer's disease warning system based on gait video. The system can perform standardized collection and annotation processing of gait videos based on a unified gait video acquisition device, and pre-process the gait video data through gait video segmentation technology to obtain a video that only retains the subject's standard gait acquisition process; and perform human key point detection based on the processed video, and extract the spatial features, temporal features, and variability features of the gait video based on the human key points; and classify according to the extracted features and the video classification network to achieve Alzheimer's disease detection and warning. Therefore, the embodiment of the present invention obtains high-quality, standardized gait video data through a unified gait video acquisition device, and uses gait video segmentation and human key point detection technology to achieve Alzheimer's disease warning with a machine learning classification model, thereby improving the accuracy of Alzheimer's disease warning and reducing costs. like Figure 1 As shown, the present invention provides an Alzheimer's disease early warning system based on gait video, comprising: The data collection and annotation module 10 is used to perform standardized collection and annotation processing of gait videos based on a unified gait video collection device to obtain gait video data; A data preprocessing module 20, for preprocessing the gait video data by using a gait video segmentation technology to obtain a video that only retains the subject's standard gait acquisition process action; A human key point detection module 30 is used to detect human key points based on the processed video, and extract spatial features, temporal features and variability features of the gait video based on the human key points; The feature classification and early warning module 40 is used to perform classification based on the spatial feature, the temporal feature, the variability feature and the video classification network to perform Alzheimer's disease detection and early warning.

[0031] Specifically, in an implementation of this embodiment, the data collection and annotation module 10 includes: A unified gait acquisition unit, used for performing standardized acquisition of gait videos according to preset gait movements based on the unified gait video acquisition device; A quality evaluation unit is used to evaluate the quality of the collected gait video data according to the clarity of the gait video, the standard degree of the patient's action and the number of invalid video frames, so as to obtain gait video data that meets the conditions; The data annotation unit is used to perform Alzheimer's disease analysis on each subject to obtain pure gait video data annotation.

[0032] Specifically, in an implementation of this embodiment, the data preprocessing module 20 includes: The action frame selection unit is used to delete frames in the gait video data that are not related to the subject's actions, and to delete videos with preset video quality and preset label purity, so as to obtain the video that only retains the subject's actions in the standard gait collection process.

[0033] Specifically, in an implementation of this embodiment, the human key point detection module 30 includes: A key point extraction unit, used for performing human key point detection based on the processed video using a human key point detection algorithm to extract arm key points and leg key points; The multimodal feature extraction unit is used to extract the spatial features, temporal features and variability features of the gait video according to the extracted arm key points and leg key points.

[0034] Specifically, in an implementation of this embodiment, the feature classification and warning module 40 includes: A video classification unit is used to directly splice the extracted multiple gait features on the final fully connected layer features of the contrast graph convolutional network based on the spatial features, the temporal features and the variability features to perform video classification and obtain a video classification result of Alzheimer's disease; wherein the positive class is a video of an Alzheimer's positive sample, and the negative class is a video of an Alzheimer's negative sample; The early warning unit is used to output the Alzheimer's disease test results and provide early warning for Alzheimer's disease.

[0035] This embodiment achieves the following technical effects through the above technical solution: This embodiment performs standardized gait video acquisition and annotation processing based on a unified gait video acquisition device, and pre-processes the gait video data through gait video segmentation technology, so as to obtain a video that only retains the subject's standard gait acquisition process; and performs human key point detection based on the processed video, and extracts the spatial features, temporal features and variability features of the gait video based on the human key points; and can classify according to the extracted features and the video classification network to achieve Alzheimer's disease detection and early warning. This embodiment obtains high-quality, standardized gait video data, and uses gait video segmentation and human key point detection technology to achieve Alzheimer's disease early warning with a machine learning classification model, thereby improving the accuracy of Alzheimer's disease early warning and reducing costs.

[0036] Exemplary Methods Based on the Alzheimer's disease early warning system based on gait video provided in the above embodiments, an embodiment of the present invention provides an Alzheimer's disease early warning method based on gait video.

[0037] like Figure 2As shown, the Alzheimer's disease early warning method based on gait video includes the following steps: Step S100: performing standardized collection and annotation processing of gait videos based on a unified gait video collection device to obtain gait video data.

[0038] In this embodiment, the Alzheimer's disease early warning method based on gait video is mainly divided into four steps: 1) Standardized gait video collection to obtain high-quality, unified, and standardized gait video data; 2) Gait video segmentation and human key point detection; 3) Gait feature extraction to obtain spatial, temporal and variability features; 4) Machine learning classification model design and Alzheimer's identification.

[0039] The Alzheimer's disease early warning method based on gait video proposed in this embodiment can solve the problems existing in the existing Alzheimer's disease early warning methods, and can also provide a reference framework for other disease early warnings based on gait videos.

[0040] Specifically, in an implementation of this embodiment, step S100 includes the following steps: Step S101, based on the unified gait video acquisition device, perform standardized acquisition of gait video according to preset gait movements; Step S102, evaluating the quality of the collected gait video data according to the gait video clarity, the standard degree of the patient's action, and the number of invalid video frames, to obtain gait video data that meets the conditions; Step S103 , performing Alzheimer's disease analysis on each subject to obtain pure gait video data annotation.

[0041] In this embodiment, by standardizing the processing method of gait video data collection and annotation, high-quality, unified, and standardized gait video data can be obtained; in this embodiment, the following Alzheimer's video collection process is formulated: For each subject, the preset gait action is: first walk forward normally for three meters; then sit on a chair facing the patient in a clockwise direction (the seat height can be 48 cm); finally, the patient stands up and walks three meters back to the starting position.

[0042] In addition to formulating the Alzheimer's video acquisition process, this embodiment also requires quality evaluation of the acquired gait video data; the specific content of the quality evaluation includes: quality evaluation of the gait video clarity, the standard degree of patient action, the number of invalid video frames, etc. After the quality evaluation of the acquired gait video data, the gait video data that meets the conditions is selected as sample data; wherein the gait video data that meets the conditions is: gait video data with high clarity, high standard degree of patient action, and a small number of invalid video frames. By performing quality evaluation on the acquired gait video data, not only can the automation level of the acquisition module be improved, but also the reliability of subsequent data analysis can be guaranteed.

[0043] After the quality evaluation of the collected gait video data, in order to ensure the reliability of the data, in this embodiment, data labeling processing is also required in a tertiary hospital: that is, a strict Alzheimer's disease analysis is performed on each subject to obtain pure gait video data labeling; wherein, the pure gait video data specifically includes: data corresponding to the degree of Alzheimer's disease of positive patients and the health status of negative participants.

[0044] like Figure 2 As shown, an embodiment of the present invention provides an Alzheimer's disease early warning method based on gait video, comprising the following steps: Step S200, pre-processing the gait video data by using gait video segmentation technology to obtain a video that only retains the subject's standard gait collection process actions.

[0045] In this embodiment, after the standardized gait video data is collected and labeled, the gait video is segmented and frames in the gait video data that are not related to the subject's actions are deleted to obtain sample data that is more suitable for model classification, that is, a video that only retains the subject's standard gait collection process actions.

[0046] Specifically, in an implementation of this embodiment, step S200 includes the following steps: Step S201, deleting frames irrelevant to the subject's actions in the gait video data, and deleting videos with preset video quality and preset label purity, to obtain the video that only retains the subject's standard gait collection process actions.

[0047] In this embodiment, the process of preprocessing the gait video data is as follows: In this embodiment, the collected gait video is preprocessed to delete frames irrelevant to the subject's actions (ie, frames irrelevant to the preset gait movements) and to delete videos with low video quality and low label purity.

[0048] As an example, for the gait video data, an optical flow method (e.g., Lucas-Kanade method) can be used to calculate the optical flow between frames to identify the movement of the subject. For example, a threshold can be set to delete frames without significant movement.

[0049] Then, a simple action recognition algorithm (e.g., a 3D CNN algorithm) is applied to recognize the subject’s walking action by training the model and retaining the frames related to walking.

[0050] After that, the image clarity of each frame is calculated (such as using the Laplacian operator), blurry frames are deleted, and the noise level of the frame can be evaluated by the signal-to-noise ratio (SNR) or other indicators to remove low-quality frames; to ensure that the label of each frame in the video is consistent with the actual action. The accuracy of the label can be evaluated by automated review or manual inspection, as well as to ensure that each subject's behavior in the video has a complete label. If some frames are missing labels or the labels are inaccurate, these frames can be deleted.

[0051] Finally, frames that do not meet the requirements are cut out according to the above standards to form a clean and coherent gait video, ensuring that the resolution and frame rate of all videos are consistent for subsequent processing.

[0052] like Figure 2 As shown, an embodiment of the present invention provides an Alzheimer's disease early warning method based on gait video, comprising the following steps: Step S300, performing human key point detection based on the processed video, and extracting spatial features, temporal features, and variability features of the gait video based on the human key points.

[0053] In this embodiment, after the gait video data is preprocessed, a human key point detection algorithm (for example, the AlphaPose algorithm) is used to detect the human key points. The human key point extraction effect in this embodiment is as follows: Figure 3 Based on these key points of the human body, especially the key points of the arms and legs, the movement state of the human body can be estimated, and the gait features can be extracted accordingly; wherein the gait features include the spatial features, temporal features and variability features of the gait video.

[0054] Specifically, in an implementation of this embodiment, step S300 includes the following steps: Step S301, based on the processed video, using a human key point detection algorithm to perform human key point detection, extracting arm key points and leg key points; Step S302: extracting the spatial features, temporal features and variability features of the gait video based on the extracted arm key points and leg key points.

[0055] In this embodiment, three types of gait features (spatial features, temporal features, and variability features) are extracted based on the key points of the human body. These three types of gait features total 15 gait features. Among them, the variability feature can be described by standard deviation, coefficient of variation (coefficient of variation = standard deviation / mean × 100%), or by nonlinear indicators (for example, detrended fluctuation analysis, local dynamic stability, and entropy). Specifically, these three types of gait features mainly include: 1. Spatial characteristics: stride length characteristics, that is, the longitudinal straight-line distance from the landing of one heel to the landing of the other heel when walking.

[0056] 2. Spatial characteristics: stride characteristics, that is, the longitudinal straight-line distance from the first landing to the second landing of the same-side heel when walking.

[0057] 3. Spatial characteristics: step width characteristics, that is, the lateral distance between the midpoint of one heel and the midpoint of the other heel when walking.

[0058] 4. Time characteristics: Cadence characteristics, that is, the number of steps taken in a time unit, commonly used unit: steps / min.

[0059] 5. Time characteristics: gait speed characteristics, which are calculated by dividing the stride by the stride time.

[0060] 6. Time characteristics: step duration characteristics, that is, the time from one heel landing to the other heel landing when walking.

[0061] 7. Time characteristics: Stride time characteristics, that is, the time from the first heel touches the ground to the next heel touches the ground when walking.

[0062] 8. Time characteristics: Single support time characteristics, that is, the time from the heel of one lower limb touching the ground to the toe of the same side leaving the ground.

[0063] 9. Time characteristics: Double support time characteristics, that is, the time when both feet support the body weight at the same time.

[0064] 10. Time characteristics: Standing time characteristics, that is, the duration from the lower limb of one side touching the ground to leaving the ground.

[0065] 11. Time characteristics: Swing time characteristics, that is, the duration from one side of the lower limb leaving the ground to re-contacting the ground.

[0066] 12. Variability feature: The step length variability feature refers to the change of step length in multiple walking cycles.

[0067] 13. Variability feature: Stride variability feature, that is, the change of stride length in multiple walking cycles.

[0068] 15. Variability feature: The step width variability feature refers to the change of step width in multiple walking cycles.

[0069] 15. Variability characteristics: The variability characteristics of step time, that is, the changes in step time in multiple walking cycles.

[0070] Among the above 15 gait characteristics, the stride time variability feature is the change of stride time in multiple walking cycles.

[0071] like Figure 2 As shown, an embodiment of the present invention provides an Alzheimer's disease early warning method based on gait video, comprising the following steps: Step S400, classification is performed based on the spatial features, the temporal features, the variability features and the video classification network to perform Alzheimer's disease detection and early warning.

[0072] In this embodiment, based on the extracted 15-dimensional gait features, a model is designed to classify healthy / Alzheimer's gait videos to achieve Alzheimer's disease detection and early warning.

[0073] Specifically, in an implementation of this embodiment, step S400 includes the following steps: Step S401, based on the spatial features, the temporal features and the variability features, a contrastive graph convolutional network is used to classify the video; wherein the positive class is a video of an Alzheimer's positive sample, and the negative class is a video of an Alzheimer's negative sample; In one implementation of this embodiment, the contrastive graph convolutional network is used to perform video classification based on the spatial features, the temporal features, and the variability features, including: directly splicing the extracted multiple gait features on the final fully connected layer features to perform video classification, and using a loss function for processing the category imbalance problem as the classification loss function to obtain a video classification result for Alzheimer's disease.

[0074] Step S402, outputting the Alzheimer's disease detection result and issuing an Alzheimer's disease early warning.

[0075] In this embodiment, a contrastive graph convolutional network is used as the video classification network (ie, the backbone network), wherein the positive class is a video of an Alzheimer's positive sample, and the negative class is a video of an Alzheimer's negative sample.

[0076] At the same time, the 15 gait features extracted are directly spliced ​​on the final fully connected layer features for video classification, and Focal Loss (the loss function for dealing with the class imbalance problem) is used as the classification loss function to alleviate the sample imbalance problem (i.e. the number of healthy people is often larger than that of Alzheimer's positive people, so the number of samples is far greater than that of Alzheimer's positive samples). The benefits of doing this are: 1. The influence of irrelevant information such as background, subject's clothes, and subject's skin color on Alzheimer's video classification can be excluded; 2. Ability to fuse gait features with gait classification network to obtain better classification results; 3. By using interpretable features for classification, the interpretability in gait video classification and Alzheimer's disease identification can be improved.

[0077] It is worth mentioning that this embodiment can be improved or replaced by the following methods: 1) Combining gait videos with other biometric data (e.g., static images, acoustic signals) to build a multi-dimensional data collection framework. This fusion approach may enhance the accuracy and robustness of the analysis while providing a broader scope of protection for the patent.

[0078] 2) Use advanced image processing and pattern recognition techniques to extract more refined or different types of gait features. This may include automatic feature learning based on deep learning, or innovative methods combined with traditional gait analysis to improve the distinguishability of features and the sensitivity of analysis.

[0079] 3) Design customized loss functions to more effectively handle class imbalance problems or optimize the performance of the model on specific types of data. This may include introducing new loss functions or adjusting the parameters of existing functions to improve the generalization ability and analysis accuracy of the model.

[0080] 4) Designing gait analysis technology as a general solution independent of specific diseases (such as Alzheimer’s disease). This design can expand the application scope of the technology, making it suitable for a wider range of clinical and research scenarios, while providing a more flexible protection strategy for patents.

[0081] This embodiment achieves the following technical effects through the above technical solution: This embodiment performs standardized gait video acquisition and annotation processing based on a unified gait video acquisition device, and pre-processes the gait video data through gait video segmentation technology, so as to obtain a video that only retains the subject's standard gait acquisition process; and performs human key point detection based on the processed video, and extracts the spatial features, temporal features and variability features of the gait video based on the human key points; and can classify according to the extracted features and the video classification network to achieve Alzheimer's disease detection and early warning. This embodiment obtains high-quality, standardized gait video data, and uses gait video segmentation and human key point detection technology to achieve Alzheimer's disease early warning with a machine learning classification model, thereby improving the accuracy of Alzheimer's disease early warning and reducing costs.

[0082] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be as follows: Figure 4 shown.

[0083] The terminal includes: a processor, a memory, an interface, a display screen and a communication module connected through a system bus; wherein the processor of the terminal is used to provide computing and control capabilities; the memory of the terminal includes a storage medium and an internal memory; the storage medium stores an operating system and a computer program; the internal memory provides an environment for the operation of the operating system and the computer program in the storage medium; the interface is used to connect to external devices; the display screen is used to display corresponding information; and the communication module is used to communicate with a cloud server or other devices.

[0084] When the computer program is executed by a processor, it is used to implement the operation of the Alzheimer's disease early warning method based on gait video.

[0085] It can be understood by those skilled in the art that Figure 4 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0086] In one embodiment, a terminal is provided, which includes: a processor and a memory, wherein the memory stores an Alzheimer's disease early warning program based on gait video, and the Alzheimer's disease early warning program based on gait video is used to implement the operation of the above-mentioned Alzheimer's disease early warning method based on gait video when executed by the processor.

[0087] In one embodiment, a storage medium is provided, wherein the storage medium stores an Alzheimer's disease early warning program based on gait video, and when the Alzheimer's disease early warning program based on gait video is executed by a processor, it is used to implement the operation of the above-mentioned Alzheimer's disease early warning method based on gait video.

[0088] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and volatile memory.

[0089] In summary, the present invention provides an Alzheimer's disease early warning system and method based on gait video, including: a data acquisition and annotation module, which is used to perform standardized acquisition and annotation processing of gait videos based on a unified gait video acquisition device; a data preprocessing module, which is used to preprocess gait video data through gait video segmentation technology to obtain a standard gait video; a human key point detection module, which is used to detect human key points based on the processed video, and extract the spatial, temporal and variability characteristics of the gait video based on the human key points; a feature classification and early warning module, which is used to classify based on the extracted features and the video classification network to perform Alzheimer's disease detection and early warning. The present invention obtains gait video data based on a unified gait video acquisition device, and uses gait video segmentation and human key point detection technology to achieve Alzheimer's disease early warning, improves the accuracy of Alzheimer's disease early warning, and reduces costs.

[0090] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. An Alzheimer's disease early warning system based on gait video, characterized in that: include: A data collection and annotation module is used to perform standardized collection and annotation processing of gait videos based on a unified gait video collection device to obtain gait video data; A data preprocessing module, used to preprocess the gait video data by using a gait video segmentation technology to obtain a video that only retains the subject's standard gait collection process actions; A human key point detection module is used to detect human key points based on the processed video, and to extract spatial features, temporal features, and variability features of the gait video based on the human key points; The feature classification and early warning module is used to perform classification based on the spatial features, the temporal features, the variability features and the video classification network to perform Alzheimer's disease detection and early warning.

2. The Alzheimer's disease early warning system based on gait video according to claim 1, characterized in that: The data collection and annotation module includes: A unified gait acquisition unit, used for performing standardized acquisition of gait videos according to preset gait movements based on the unified gait video acquisition device; A quality evaluation unit is used to evaluate the quality of the collected gait video data according to the clarity of the gait video, the standard degree of the patient's action and the number of invalid video frames, so as to obtain gait video data that meets the conditions; The data annotation unit is used to perform Alzheimer's disease analysis on each subject to obtain pure gait video data annotation.

3. The Alzheimer's disease early warning system based on gait video according to claim 1, characterized in that: The data preprocessing module comprises: The action frame selection unit is used to delete frames in the gait video data that are not related to the subject's actions, and to delete videos with preset video quality and preset label purity, so as to obtain the video that only retains the subject's actions in the standard gait collection process.

4. The Alzheimer's disease early warning system based on gait video according to claim 1, characterized in that: The human body key point detection module comprises: A key point extraction unit, used for performing human key point detection based on the processed video using a human key point detection algorithm to extract arm key points and leg key points; The multimodal feature extraction unit is used to extract the spatial features, temporal features and variability features of the gait video according to the extracted arm key points and leg key points.

5. The Alzheimer's disease early warning system based on gait video according to claim 1, characterized in that: The feature classification and early warning module includes: A video classification unit is used to directly splice the extracted multiple gait features on the final fully connected layer features of the contrast graph convolutional network based on the spatial features, the temporal features and the variability features to perform video classification and obtain a video classification result of Alzheimer's disease; wherein the positive class is a video of an Alzheimer's positive sample, and the negative class is a video of an Alzheimer's negative sample; The early warning unit is used to output the Alzheimer's disease test results and provide early warning for Alzheimer's disease.

6. An Alzheimer's disease early warning method according to any one of claims 1 to 5, characterized in that: include: Based on a unified gait video acquisition device, gait video is collected and annotated in a standardized manner to obtain gait video data; Preprocessing the gait video data by using gait video segmentation technology to obtain a video that only retains the subject's standard gait acquisition process actions; Detect human key points based on the processed video, and extract spatial features, temporal features, and variability features of the gait video based on the human key points; Classification is performed based on the spatial features, the temporal features, the variability features and the video classification network to perform Alzheimer's disease detection and early warning.

7. The Alzheimer's disease early warning method based on gait video according to claim 6, characterized in that: The standardized acquisition and annotation processing of gait video based on the unified gait video acquisition device to obtain gait video data includes: Based on the unified gait video acquisition device, standardized gait video acquisition is performed according to preset gait movements; According to the gait video clarity, the standard degree of the patient's action and the number of invalid video frames, the quality of the collected gait video data is evaluated to obtain gait video data that meets the conditions; Alzheimer's disease analysis was performed on each subject to obtain clean gait video data annotations.

8. The Alzheimer's disease early warning method based on gait video according to claim 6, characterized in that: The gait video data is preprocessed by the gait video segmentation technology to obtain a video that only retains the subject's standard gait collection process action, including: Frames irrelevant to the subject's actions in the gait video data are deleted, and videos with preset video quality and preset label purity are deleted to obtain the video that only retains the subject's standard gait collection process actions.

9. The Alzheimer's disease early warning method based on gait video according to claim 6, characterized in that: The method of detecting human key points based on the processed video and extracting spatial features, temporal features and variability features of the gait video based on the human key points includes: Based on the processed video, the human key point detection algorithm is used to detect the human key points and extract the arm key points and leg key points; Based on the extracted arm key points and leg key points, the spatial features, temporal features and variability features of the gait video are extracted.

10. The Alzheimer's disease early warning method based on gait video according to claim 6, characterized in that: The classification based on the spatial feature, the temporal feature, the variability feature and the video classification network to detect and warn of Alzheimer's disease includes: Based on the spatial features, the temporal features, and the variability features, the extracted multiple gait features are directly spliced ​​on the final fully connected layer features of the contrast graph convolutional network to perform video classification, thereby obtaining a video classification result of Alzheimer's disease; wherein the positive class is a video of an Alzheimer's positive sample, and the negative class is a video of an Alzheimer's negative sample; Output Alzheimer's disease test results and provide Alzheimer's disease warning.

Images