Cerebral stroke prediction method based on gait video

Abstract

The invention relates to a stroke prediction method based on gait videos. The method comprises the following steps: collecting video data of healthy people and stroke patients during walking; decomposing the video into frame images, extracting figure posture articulation points in each frame image, and outputting coordinates of each articulation point; dividing the obtained attitude sequence data according to gaits, taking the coordinate of the right ankle height in each frame of image, and defining the coordinate between every two peak values as one step; inputting the extracted attitude point sequence data with the length of 20 in each step into an MSResNet1D model, carrying out learning training, and then outputting two values by the model; and extracting attitude point data of each step in the new video, inputting the data into the trained MSResNet1D model, and outputting a prediction result by the model. The method has the beneficial effects that the early diagnosis rate of cerebral apoplexy is improved, the medical cost is reduced, and the consistency and accuracy of diagnosis are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of stroke detection, and in particular relates to a stroke prediction method based on gait video. Background Art

[0002] Stroke is a common sudden neurological disease, characterized by sudden rupture or blockage of cerebral blood vessels. This pathological change can quickly lead to insufficient blood supply to local or widespread areas of the brain, which in turn causes ischemia, hypoxia, and even necrosis of brain tissue. According to the World Health Organization (WHO), stroke is the second leading cause of death worldwide and the leading cause of disability in adults.

[0003] Time is a crucial factor in the treatment of stroke. Studies have shown that within the first few hours after a stroke, the patient's treatment outcome is closely related to their recovery potential. Therefore, early diagnosis and timely intervention play a decisive role in improving the patient's prognosis. However, traditional stroke diagnosis methods rely on doctors' clinical experience and imaging examinations, such as CT scans and MRI. Although they have unparalleled advantages in accuracy, their high cost and reliance on professional operating skills make these technologies difficult to popularize in areas with limited resources. This not only limits the early diagnosis of stroke patients, but also exacerbates the imbalance of global medical resources.

[0004] In addition, there is a certain time delay in the use of CT and MRI equipment. From the onset of symptoms to the examination and then to the doctor's interpretation of the results, the entire process may take several hours or even longer. This time delay may mean the loss of the best treatment window for stroke patients, thus affecting the treatment effect and recovery prospects.

[0005] At the same time, during the diagnosis process, due to the subjectivity of expert interpretation and the influence of human factors, there may be certain errors and inconsistencies in the diagnostic results. Since the imaging manifestations of stroke are complex and changeable, even experienced radiologists may make deviations when interpreting images. Such errors and inconsistencies increase the risk of misdiagnosis and missed diagnosis to a certain extent, and have an adverse effect on the treatment and rehabilitation of patients. Summary of the invention

[0006] In order to make up for the deficiencies of the prior art, the present invention provides a gait video-based stroke prediction method that has rapid and accurate diagnosis, a wide range of applications, and reduces dependence on expensive equipment and professional operating skills.

[0007] The present invention is achieved through the following technical solutions: A method for predicting stroke based on gait video, characterized by comprising the following steps: (1) Collect video data of healthy people and stroke patients walking; (2) Decompose the video into frame images, extract the character posture joint points in each frame image, and output the coordinates of each joint point; (3) Divide the acquired posture sequence data according to gait, take the coordinates of the right ankle height in each frame image, and define the coordinates between every two peaks as one step; (4) Input the posture point sequence data of each step with a length of 20 extracted in step (3) into the MSResNet1D model for learning and training. Then the model outputs two values, representing the probabilities of stroke patients and healthy people respectively; (5) Extract the posture point data of each step in the new video and input these data into the trained MSResNet1D model. The model outputs the prediction result, that is, whether the walker in the new video suffers from a stroke.

[0008] Preferably, in step (1), the video acquisition device is kept 3 meters away from the person being acquired, and the acquisition device is placed on the side of the person being acquired.

[0009] Preferably, in step (2), the video stream is converted into a static image sequence.

[0010] Preferably, in step (3), the posture point sequence of each step is interpolated so that its length is unified to 20 data points.

[0011] Preferably, in step (5), the probability of belonging to a stroke patient or a healthy person is predicted at each step, and finally the average of the results of each step is taken to obtain the probability that the subject is a stroke patient or a healthy person.

[0012] The beneficial effects of the present invention are: 1. It can be performed without professional medical equipment, making early screening of stroke more convenient. Its advantages are that it is non-invasive, easy to operate and can respond quickly, which not only helps to improve the early diagnosis rate of stroke, but also reduces medical costs. 2. Due to its automated characteristics, it reduces dependence on interpretation by professional medical personnel, reduces subjective errors in the diagnostic process, and improves the consistency and accuracy of diagnosis. 3. It is especially valuable for areas with limited resources and helps to popularize stroke prevention and treatment. With the continuous advancement and optimization of technology, it is expected to become an important tool for stroke screening and make important contributions to the prevention and treatment of stroke worldwide. DETAILED DESCRIPTION

[0013] First, it is necessary to build a gait video database containing stroke patients and healthy people. These videos will serve as the basis for algorithm training and testing to ensure that the model can detect the difference between normal gait and abnormal gait. During the video acquisition process, the acquisition device should be kept about 3 meters away from the person being acquired. At the same time, the acquisition device should be placed on the side of the person being acquired to obtain a clear gait image. This setting helps to capture the complete gait cycle while avoiding image distortion caused by being too close or too far away.

[0014] After the video is captured, OpenCV is used to decompose the video into frame images. This process can convert the video stream into a static image sequence, which is convenient for subsequent posture analysis. Next, we use the open source framework Alphapose to extract the character posture joint points in each frame, including the positions of joints and body parts. These posture point coordinates will provide the detailed information required for gait analysis. The Alphapose framework is based on deep learning technology, especially convolutional neural networks (CNN), to achieve accurate recognition of human posture. Its core working principles are as follows: 1. First, Alphapose pre-processes the input image, including scaling, cropping, and normalization to ensure that the image is suitable for subsequent neural network processing; 2. Then, the Alphapose pre-trained convolutional neural network (CNN) outputs a set of key points, which correspond to various joints of the human body, such as head, shoulders, elbows, wrists, hips, knees, and ankles. The coordinate information of these key points provides the basis for subsequent posture analysis; 3. Then, after the key points are located, Alphapose applies post-processing techniques to optimize the results, such as reducing noise through smoothing, or using a pose graph to connect key points in consecutive frames to obtain smoother posture estimates; 4. Finally, Alphapose outputs the coordinates of each joint point, which can be used for further analysis, such as gait analysis, motion capture, or human interaction.

[0015] In this example, the character's joint posture points processed by the Alphapose framework include 24 points, each of which represents: 0. Root node, 1. Left hip joint, 2. Left knee joint, 3. Left ankle joint, 4. Right hip joint, 5. Right knee joint, 6. Right ankle joint, 7. Middle spine joint, 8. Upper spine joint, 9. Neck spine joint, 10. Left shoulder joint, 11. Left elbow joint, 12. Left wrist joint, 13. Right shoulder joint, 14. Right elbow joint, 15. Right wrist joint, 16. Neck joint, 17. Head joint, 18. Left hip joint, 19. Right hip joint, 20. Left ankle joint, 21. Right ankle joint, 22. Left hand joint, 23. Right hand joint.

[0016] In the analysis and processing stage, we first need to divide the joint point sequence into steps and select the z coordinate of the 20th or 21st node as the gait data of the subject when walking. This selection is based on the general length of the gait cycle and ensures the continuity and integrity of the data.

[0017] In order to improve the data quality, we further processed the gait data {a_n} in the stride_video_process.py file, mainly using normalization, smoothing filtering and Kalman filtering.

[0018] Normalization is to scale the data to a specific range, usually between 0 and 1. When estimating posture, normalization can help us eliminate the impact of different individual size differences on posture estimation. We can use the norm function in the code to perform a series of linear transformations to convert the coordinates of the human joint points into a unified reference frame, so that the model can give consistent posture estimates for people of different heights and body shapes.

[0019] Smoothing filtering is mainly used to reduce noise and outliers in data. In posture data, smoothing filtering can help us get a smoother and more coherent posture trajectory. The smooth_pose function in the code uses the sliding average method to implement smoothing filtering. It uses a window of length L=fps*0.25. The filtered value a1_n is the mean of {a_n} with length L centered at n. It calculates the average value of the joint point coordinates within a certain time window instead of the coordinates of a single time point, thereby reducing data fluctuations. This step helps reduce noise and makes gait features clearer.

[0020] Kalman filter is a recursive filter. When estimating posture, Kalman filter can be used to predict and correct posture data to improve the accuracy of estimation. The kalman_pose function in the code implements a simplified version of Kalman filter. Through looping, Kalman filter continuously predicts and updates new posture data to generate a series of smooth and continuous posture estimates. The prediction step is based on the state of the previous moment and the known dynamic model, and the update step combines the current observation data to correct the predicted value. These estimates are used in subsequent gait analysis, such as gait cycle detection and gait data segmentation, which improves the accuracy and continuity of posture estimation and provides high-quality input data for gait analysis.

[0021] After obtaining the filtered data, we divide the gait data into steps and select the data between every two maximum values ​​as a step, which usually corresponds to a complete gait cycle. To ensure the consistency of the data, we interpolate the posture point sequence of each step to make it a sequence with a time length of 20. This processing makes each gait sample consistent in time length, which is convenient for subsequent model training.

[0022] Next, we use these processed gait data to train the MSResNet1D model. MSResNet1D is a convolutional neural network designed for one-dimensional feature sequences. It improves on ResNet and introduces a multi-scale feature fusion mechanism to enhance the representation and generalization capabilities of the model. This design enables the model to have high performance when processing time series and has low computational complexity, which is very suitable for processing posture point data during walking. The input to the MSResNet1D model is a posture point sequence of length 20, representing the features of each step, and then the model outputs two values, representing the probabilities of stroke patients and healthy people. During training, we use the Adam optimizer, the learning rate is set to 10^-5, the loss function is the cross entropy loss function, and the samples are classified into two categories. This setting helps the model learn to distinguish the differences in gait between stroke patients and healthy people. The model uses the PyTorch framework to construct a multi-scale one-dimensional residual network (MSResNet), which can capture the multi-scale features in gait data, because gait abnormalities may be manifested at different time scales. In this way, the model can more accurately identify stroke patients and normal people.

[0023] After training the MSResNet1D model, we enter the inference phase. For a new gait video, we first use alphapose to extract the posture joints in each frame and divide them according to the previously defined gait cycle. Then, the posture point sequence of each step is input into the trained MSResNet1D model to predict the probability of each step belonging to a stroke patient or a healthy person. Finally, the results of each step are averaged to obtain the probability of the subject being a stroke patient or a healthy person.

[0024] The stroke prediction method based on gait video of the present invention uses artificial intelligence-based image analysis technology and deep learning algorithms to train a large amount of medical image data, which can automatically identify and analyze the image features of stroke, thereby achieving rapid and accurate diagnosis. It reduces the dependence on expensive equipment and professional operating skills, making early diagnosis of stroke possible in resource-poor areas.

[0025] The algorithm uses computer vision and machine learning technology to process and analyze videos of patients walking, making the analysis of patients' gait characteristics more efficient and accurate. In this process, the algorithm first collects high-quality images of the patient's walking video, and then extracts key gait characteristic parameters such as stride length, stride frequency, and gait symmetry through computer vision technology. These features are then input into the model and compared with the gait database of normal people. Through the comparison results, it is possible to quickly and accurately determine whether the patient is at risk of stroke.

Claims

1. A stroke prediction method based on gait video, characterized by: The following steps are involved: (1) Collect video data of healthy people and stroke patients walking; (2) Decompose the video into frame images, extract the character posture joint points in each frame image, and output the coordinates of each joint point; (3) Divide the acquired posture sequence data according to gait, take the coordinates of the right ankle height in each frame image, and define the coordinates between every two peaks as one step; (4) Input the posture point sequence data of each step with a length of 20 extracted in step (3) into the MSResNet1D model for learning and training. Then the model outputs two values, representing the probabilities of stroke patients and healthy people respectively; (5) Extract the posture point data of each step in the new video and input these data into the trained MSResNet1D model. The model outputs the prediction result, that is, whether the walker in the new video suffers from a stroke.

2. The method for predicting stroke based on gait video according to claim 1, characterized in that: In step (1), the video acquisition device is kept 3 meters away from the person being acquired, and the acquisition device is placed on the side of the person being acquired.

3. The method for predicting stroke based on gait video according to claim 1, characterized in that: In step (2), the video stream is converted into a sequence of static images.

4. The method for predicting stroke based on gait video according to claim 1, characterized in that: In step (3), the posture point sequence of each step is interpolated to make its length uniform to 20 data points.

5. The method for predicting stroke based on gait video according to claim 1, characterized in that: In step (5), the probability of belonging to a stroke patient or a healthy person is predicted at each step, and finally the average of the results of each step is taken to obtain the probability that the subject is a stroke patient or a healthy person.