Rat gait analysis and automatic scoring method based on deep learning
Through deep learning and image processing technology, combined with deep neural network, the automatic scoring of rat gait behavior is solved, and the problems of high subjectivity, low efficiency and insufficient automation in traditional methods are achieved, and high-precision and automated rat steps are achieved. state analysis, suitable for high-throughput experiments.
Patent Information
- Application Number
- CN202411226946.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-03
- Publication Date
- 2024-12-27
AI Technical Summary
Traditional rat gait analysis has problems such as high subjectivity, low efficiency and insufficient accuracy of automation systems, making it difficult to deal with large-scale data and complex behavior patterns.
Using deep learning-based methods, the gait behavior of rats is recorded through video, the gait characteristics are extracted using key point detection technology, and the gait characteristics are classified in combination with deep neural networks to achieve automatic scoring, reduce manual intervention and improve the objectivity of scoring and consistency.
It significantly improves the reliability and repeatability of rat gait analysis, can process large-scale data, reduce manual scoring time, and provides more efficient and accurate analysis tools suitable for high-throughput experiments.
Smart Images

Figure CN119206857A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of rat gait analysis and scoring, and in particular to a rat gait analysis and automatic scoring method based on deep learning. The present invention is used to automatically and accurately evaluate the gait behavior of rats, eliminate the subjectivity of manual scoring, and improve the efficiency and consistency of research. The method of the present invention records the gait behavior of rats through video, extracts gait features using key point detection technology, and classifies these features in combination with a deep neural network, thereby achieving high-precision automatic scoring, significantly improving the reliability and repeatability of behavior analysis. Background Art
[0002] In the prior art, traditional rat gait analysis mainly relies on manual scoring and qualitative observation. Although these methods can provide accurate results in some cases, there are obvious subjectivity and consistency differences, resulting in low reproducibility of the results. In addition, the manual scoring process is cumbersome and time-consuming, and it is difficult to process large-scale data, which limits its application in high-throughput experiments. At the same time, although some automated gait analysis systems have improved their efficiency with the help of sensors or basic image processing technology, they can usually only capture limited gait features, lack an in-depth understanding of complex behavioral patterns, and rely on predefined feature sets and simple classification algorithms, which are difficult to deal with complex or abnormal gaits. Summary of the invention
[0003] The purpose of the present invention is to make up for the defects of high subjectivity, low efficiency and insufficient precision of the automated system in the traditional rat gait analysis of the prior art, and to provide a rat gait analysis and automatic scoring method based on deep learning. By introducing deep learning and image processing technology, the present invention can automatically and accurately analyze the gait of rats, and significantly improve the objectivity, consistency and repeatability of the scoring. The system can not only effectively capture complex gait features, but also process large-scale data sets, greatly reduce manual intervention and scoring time, thereby providing a more efficient and reliable analysis tool for neuroscience research.
[0004] The present invention is achieved through the following technical solutions: A rat gait analysis and automatic scoring method based on deep learning specifically comprises the following steps: (1) Collect the rat's gait behavior, perform preprocessing, and detect the key point areas of the video; (2) Preprocess the detected key point area to generate a composite image; (3) Perform deep learning on the composite image and extract and classify the features; (4) Automatically generate gait scores based on the classification results; (5) Optimize and correct the results through denoising.
[0005] The step (1) of collecting the rat's gait behavior and preprocessing is performed to detect the key point area of the video, as follows: First, the rat's gait behavior video is captured by a camera. Then, image processing software is used to detect key points in the video to accurately identify and mark the key parts of the rat's gait.
[0006] The detected key point area is preprocessed as described in step (2) to generate a composite image, as follows: The detected key point areas were cropped and standardized, including cropping the rat's gait on the ladder and the gait in the mirror from the previous frame, the current frame, and the next frame, respectively, for a total of six image blocks. Subsequently, these image blocks were spliced to generate a composite image for analysis, and the cropped images were adjusted to a uniform resolution.
[0007] Step (3) of deep learning the composite image and extracting and classifying the features is as follows: The processed composite image is input into a pre-trained deep neural network for feature extraction and classification. The model automatically classifies the image into five different gait behavior categories through a deep residual learning framework: normal movement, slight slip, deep slip, complete mistake, and foot lifting. The classification results provide a quantitative scoring basis for the gait behavior of each frame.
[0008] According to the classification results described in step (4), a gait score is automatically generated, as follows: According to the classification results, a gait score is automatically generated and output in the form of a numerical value, indicating the specific behavior category of the rat in the gait.
[0009] Based on the classification results, the gait score is automatically generated. The process is as follows: each frame of the video is analyzed and the corresponding behavior category value is output. In order to accurately reflect the complete gait process, the same predicted labels in consecutive frames are merged into one step, and the behavior category value corresponding to the step is output as the score. The score is generated based on the prediction results of multiple consecutive frames, indicating the specific behavior category of the rat in this gait, ensuring the accuracy and consistency of the score.
[0010] The optimization and correction of the results by denoising in step (5) are as follows: In the prediction results of each frame, the system selects the classification results that remain consistent in multiple consecutive frames, eliminating the noise interference caused by single-frame errors, thereby optimizing and correcting the final score.
[0011] The working principle of the present invention is based on deep learning and image processing technology, and automatic scoring is achieved by analyzing the video of rat gait behavior. Specifically, the system first captures the video of the rat's gait through a camera device, and uses image processing technology to detect key points and crop the image. Then, the gait images of the previous and next frames are spliced with the gait images in the mirror to form a composite image, which is input into a pre-trained deep neural network for feature extraction and classification. The system generates a gait score based on the classification results, and improves the accuracy of the score through denoising.
[0012] The advantages of the present invention are: 1. High precision: The present invention uses a deep learning model to perform feature extraction and classification, thereby achieving high-precision analysis of rat gait and avoiding the subjectivity and inconsistency of manual scoring.
[0013] 2. Automation: The entire process of the present invention is fully automated, from video capture to scoring output, which greatly reduces manual intervention and improves work efficiency.
[0014] 3. High efficiency: The present invention can process large-scale data sets and is suitable for high-throughput experiments, which significantly speeds up the research process.
[0015] 4. Reliability: The present invention ensures the stability and reliability of the scoring results through multi-frame denoising processing, and is suitable for the analysis of complex or abnormal gaits.
[0016] 5. The present invention effectively solves the problems of strong subjectivity, low efficiency, and insufficient automation in traditional gait analysis methods. By applying deep learning technology, the present invention achieves efficient, accurate, and automated gait analysis of rats, significantly improves the quality and speed of research work, and provides a reliable analysis tool for neuroscience and behavioral research. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 The overall framework diagram for automatic scoring of rat gait; Figure 2 This is the visualization and dimensionality reduction analysis diagram of the deconvolution features; Figure 3 A comprehensive analysis of rat gait assessment. DETAILED DESCRIPTION
[0018] like Figure 1 As shown, a rat gait analysis and automatic scoring method based on deep learning specifically includes the following steps: 1. Video acquisition and preprocessing First, a video of the rat's gait behavior is captured by a camera. Next, image processing software (such as DeepLabCut) is used to detect key points in the video, accurately identifying and annotating key parts of the rat's gait, such as the position of the wrist and fingers. These key points provide an accurate coordinate basis for subsequent image processing steps.
[0019] 2. Image processing The detected key point regions were cropped and normalized. The specific operation involved cropping the rat’s gait on the ladder and the gait in the mirror from the previous frame, the current frame, and the next frame, for a total of six image blocks. These image blocks were then stitched together to generate a composite image for analysis. During this process, the cropped images were adjusted to a uniform resolution to ensure consistency of the data in subsequent analysis.
[0020] 3. Deep Learning Model Classification The processed composite image is input into a pre-trained deep neural network (such as ResNet-50) for feature extraction and classification. The model automatically classifies the image into five different gait behavior categories through a deep residual learning framework: Correct Action, Slight Slip, Deep Slip, Total Miss, and Lift the Foot. These classification results provide a quantitative scoring basis for the gait behavior of each frame.
[0021] 4. Automatic scoring and output Based on the classification results, the system automatically generates a gait score. The score is output in the form of a numerical value, indicating the specific behavior category of the rat in that gait, for researchers to further analyze and interpret. This automatic scoring method eliminates the subjectivity in manual scoring and improves the objectivity and consistency of the scoring.
[0022] 5. Result optimization and correction To further improve the accuracy of the results, this method also includes a denoising process. Specifically, in the prediction results of each frame, the system will screen out the classification results that remain consistent in multiple consecutive frames, excluding noise interference caused by single-frame errors, thereby ensuring the reliability and stability of the final score. This denoising method effectively filters out random noise and enhances the accuracy and consistency of the scoring results.
[0023] like Figure 3Figure 1. Comprehensive analysis of rat gait assessment: (A) A video frame shows an image of a rat walking on a ladder, accompanied by a graph of its gait trajectory. (B) A timeline compares the real data of the gait with the AI's intermediate and final predictions, showing the improvement of the analysis over time. (C) The confusion matrix details the classification accuracy of different gait errors, demonstrating the ability of the AI to accurately identify specific gait anomalies. (D) The bar graph shows the Dice similarity coefficient values, which are used to evaluate the consistency of the AI predictions and expert annotations in each experiment. (E) The comparison of ladder scores shows the difference between the AI and expert assessments, highlighting the reliability of the AI system in gait analysis. (F) The scatter plot shows the percentage difference between the expert measurement and the AI prediction on different gait analysis parameters. (G) The bar graph further shows the average difference between the expert score and the AI prediction, confirming the accuracy of the AI in gait analysis. Together, these parts highlight the effectiveness and accuracy of the AI in assessing rat gait, showing that its results are comparable to expert assessments.
[0024] like Figure 2 As shown, the rat gait analysis and automatic scoring method of the present invention are analyzed for interpretability through deconvolution and dimensionality reduction processes. Specifically, the input gait image is first processed using a convolutional neural network (ConvNet) and a deconvolutional neural network (DeconvNet) to achieve feature visualization. Through the deconvolution process, the most significant features in the image for the classification task can be found, and these significant features are presented in image blocks 1, 2, and 4. This step helps understand the specific areas or patterns that the neural network focuses on when analyzing gait. Subsequently, in the dimensionality reduction step, these significant features are mapped back to the position of the original image to obtain their corresponding spatial coordinates. Through a specific spatial mapping formula, these significant features are reduced in dimension to 3D space for visualization, intuitively showing the differences and distributions between different gait categories, thereby further explaining the decision-making process of the model. This process as a whole helps understand how the model recognizes and classifies different gait behaviors of rats, and provides a deeper explanation and verification for gait analysis and automatic scoring methods.
Claims
1. A rat gait analysis and automatic scoring method based on deep learning, characterized in that: The specific steps include: (1) Collect the rat's gait behavior, perform preprocessing, and detect the key point areas of the video; (2) Preprocess the detected key point area to generate a composite image; (3) Perform deep learning on the composite image and extract and classify the features; (4) Automatically generate gait scores based on the classification results; (5) Optimize and correct the results through denoising.
2. A rat gait analysis and automatic scoring method based on deep learning according to claim 1, characterized in that: The step (1) of collecting the rat's gait behavior and preprocessing is performed to detect the key point area of the video, as follows: First, the rat's gait behavior video is captured by a camera. Then, image processing software is used to detect key points in the video to accurately identify and mark the key parts of the rat's gait.
3. A rat gait analysis and automatic scoring method based on deep learning according to claim 1, characterized in that: The detected key point area is preprocessed as described in step (2) to generate a composite image, as follows: The detected key point areas were cropped and standardized, including cropping the rat's gait on the ladder and the gait in the mirror from the previous frame, the current frame, and the next frame, respectively, for a total of six image blocks. Subsequently, these image blocks were spliced to generate a composite image for analysis, and the cropped images were adjusted to a uniform resolution.
4. A rat gait analysis and automatic scoring method based on deep learning according to claim 1, characterized in that: Step (3) of deep learning the composite image and extracting and classifying the features is as follows: The processed composite image is input into a pre-trained deep neural network for feature extraction and classification. The model automatically classifies the image into five different gait behavior categories through a deep residual learning framework: normal movement, slight slip, deep slip, complete mistake, and foot lifting. The classification results provide a quantitative scoring basis for the gait behavior of each frame.
5. A rat gait analysis and automatic scoring method based on deep learning according to claim 1, characterized in that: According to the classification results described in step (4), a gait score is automatically generated, as follows: According to the classification results, a gait score is automatically generated, and the score is output in the form of a numerical value, indicating the specific behavior category of the rat in the gait; Based on the classification results, the gait score is automatically generated. The process is as follows: each frame of the video is analyzed and the corresponding behavior category value is output. The same predicted labels in consecutive frames are merged into one step, and the behavior category value corresponding to the step is output as the score. The score is generated based on the prediction results of multiple consecutive frames, indicating the specific behavior category of the rat in this gait.
6. A rat gait analysis and automatic scoring method based on deep learning according to claim 1, characterized in that: The optimization and correction of the results by denoising in step (5) are as follows: In the prediction results of each frame, the system selects the classification results that remain consistent in multiple consecutive frames, eliminating the noise interference caused by single-frame errors, thereby optimizing and correcting the final score.
Citation Information
Patent Citations
ConvLSTM network-based Mouse open field experiment behavior analysis method
CN113569675A
Fine-grained behavior recognition model construction method and system
CN113920583A
Real-time analysis method for spontaneous gaits of mice in long and narrow runway
CN115512439A
Method for establishing Parkinson's disease gait impairment assessment model, use method, equipment and medium
CN116543455A
Pig gait scoring method and system based on improved 3D convolutional neural network
CN117037024A
Cited By
Health assessment method and system for rheumatoid arthritis synovial injury DBA / 1 mouse
CN120259221A
