Real-time Gait Abnormality Detection System Based on Dynamic Optical Flow Field Analysis

Abstract

The present invention relates to the technical field of human motion analysis, and specifically to a real-time gait anomaly detection system based on dynamic optical flow field analysis. The system comprises modules such as data acquisition, topological feature construction, spatiotemporal feature fusion, manifold distance anomaly detection, and system optimization. The data acquisition module acquires gait RGB-D video and extracts three-dimensional spatial data; the topological feature construction module constructs a dynamic optical flow field model and uses multi-level downsampling to extract key frame features; the spatiotemporal feature fusion module extracts joint motion features from three dimensions, calculates the rate of change of motion direction and the proportion of support phase time, and generates gait timing features; the manifold distance anomaly detection module constructs an abnormal gait recognition model based on neural networks and support vector machines, uses modal partitioning to discriminate and classify abnormal gaits, and outputs the results; and the optimization module dynamically adjusts GPU computing resources according to the operating status to ensure real-time performance and effectively capture subtle gait anomalies.

Description

Technical Field

[0001] The present invention relates to the technical field of human motion analysis, and particularly to a real-time gait abnormality detection system based on dynamic optical flow field analysis, which is applicable to fields such as medical rehabilitation assessment, elderly fall prevention, and early screening of neurological diseases. Background Art

[0002] Gait analysis is an important means to evaluate human motor function and has wide applications in medical rehabilitation, elderly care, and neurological disease diagnosis. Traditional gait analysis methods mainly rely on wearable sensors or marker point tracking systems. These methods not only interfere with the natural gait of the subjects but also require professional operation, making it difficult to achieve long-term continuous monitoring.

[0003] Existing vision-based gait analysis systems mostly adopt two-dimensional image processing techniques, lacking depth information, which leads to a reduction in recognition accuracy in complex backgrounds. In addition, most existing systems focus on gait parameters in a single plane (such as the sagittal plane), such as step length and step frequency, while ignoring multi-dimensional joint movement characteristics, making it difficult to comprehensively describe gait abnormality states.

[0004] On the other hand, traditional gait abnormality detection algorithms often use fixed thresholds for discrimination, with poor adaptability and difficulty in dealing with individual differences and environmental changes. At the same time, insufficient real-time processing capabilities also limit the application of these systems in actual scenarios, especially in cases where computing resources are limited.

[0005] Therefore, there is an urgent need for a detection system that can analyze gait abnormalities non-contact, multi-dimensionally, and in real-time to meet the needs of clinical rehabilitation assessment and daily health monitoring. Summary of the Invention

[0006] The purpose of the present invention is to provide a real-time gait abnormality detection system based on dynamic optical flow field analysis, which can obtain human gait information non-contact, extract joint movement characteristics from multiple dimensions, and achieve high-precision gait abnormality detection through topological feature space construction and manifold distance measurement.

[0007] The present invention proposes a real-time gait abnormality detection system based on dynamic optical flow field analysis, including:

[0008] A data acquisition module for acquiring a gait RGB-D video sequence and extracting three-dimensional spatial data containing depth information;

[0009] A topological feature construction module connected to the data acquisition module for receiving the gait RGB-D video sequence, constructing a dynamic optical flow field model, and extracting key frame features through multi-level downsampling;

[0010] A spatio-temporal feature fusion module, connected to the topological feature construction module, is used to extract joint motion features from three dimensions of the sagittal plane, frontal plane, and cross-section, calculate the change rate of the motion direction and the proportion of the stance phase time, and fuse the feature change rates to generate gait time-series features;

[0011] A manifold distance anomaly detection module, connected to the spatio-temporal feature fusion module, is used to receive the gait time-series features, construct an abnormal gait recognition model based on a neural network algorithm and a support vector machine classifier, discriminate and classify abnormal gaits through modal division, and output a discrimination result;

[0012] A system optimization module, connected to the manifold distance anomaly detection module, is used to dynamically adjust the GPU computing resources according to the system operation state to ensure the real-time performance of the system.

[0013] Preferably, the data acquisition module includes:

[0014] A near-infrared depth camera, installed on the ceiling, is used to obtain an RGB-D video sequence;

[0015] A data transmission unit, used to transmit the RGB-D video sequence to a computer by wired or wireless means;

[0016] A computer, used to receive and store the RGB-D video sequence and provide a computing and processing platform.

[0017] Preferably, the topological feature construction module includes:

[0018] An optical flow calculation unit, used to calculate the optical flow vector and point cloud depth between adjacent frames in the RGB-D video from the current moment to the previous moment by using an improved pyramid Lucas-Kanade model;

[0019] A dynamic optical flow field construction unit, used to construct a dynamic optical flow field model based on the optical flow vector;

[0020] A key frame extraction unit, used to perform multi-level downsampling processing on the dynamic optical flow field data stream to obtain key frames.

[0021] Preferably, the processing process of the key frame extraction unit includes:

[0022] Compare the dynamic optical flow vectors selected at the current moment with those selected at the previous moment for N consecutive frames;

[0023] When the dynamic optical flow vectors at the current moment are consistent with those at the previous moment for N consecutive frames, update the dynamic optical flow vectors to those at the previous moment;

[0024] When the dynamic optical flow vectors at the current moment are inconsistent with those at the previous moment for N consecutive frames, the dynamic optical flow vector at the current moment is used as the dynamic optical flow vector at the current moment;

[0025] Perform the first downsampling process on the optical flow field data stream at the current moment to obtain the first key frame;

[0026] Perform the second downsampling process on the first key frame to obtain the second key frame, and complete the selection of key frames for all frames.

[0027] Preferably, the spatio-temporal feature fusion module includes:

[0028] A feature extraction unit for extracting the change rate of the motion direction and the proportion of the stance phase time at the edge of the target object from the key frame;

[0029] A feature fusion unit for non-linearly fusing the change rate of the motion direction and the proportion of the stance phase time to generate gait temporal features;

[0030] A feature mapping unit for mapping the gait temporal features to the topological feature space to construct high-dimensional features required for gait abnormality determination.

[0031] Preferably, the feature extraction unit extracts features through the following steps:

[0032] Determine the change rate of the motion direction at the edge of the target object in the key frame, where the change rate of the motion direction is obtained by calculating the optical flow vectors in the optical flow field model;

[0033] Determine the proportion of the stance phase time at the edge of the target object in the key frame, where the proportion of the stance phase time is obtained by calculating the average value of the number of consecutive occurrences of the optical flow vectors in the optical flow field model and the optical flow vectors at the previous moment.

[0034] Preferably, the manifold distance anomaly detection module includes:

[0035] A neural network processing unit for receiving the gait temporal features, processing them through the bidirectional long short-term memory network algorithm, and establishing a two-layer bidirectional long short-term memory network structure model including an input gate, a forget gate, an output gate, and a concatenation layer;

[0036] An SVM classification unit for constructing a support vector machine classifier based on the output result of the neural network processing unit and analyzing the specific parameters of the abnormal gait mode;

[0037] A discrimination decision unit for performing anomaly discrimination on the input gait data according to the output of the SVM classification unit in combination with the Mahalanobis distance metric method.

[0038] Preferably, the SVM classification unit constructs a training set through the following steps:

[0039] Randomly collect 10,000 - 50,000 groups of gait temporal features within 30 - 90 seconds and label them as positive samples;

[0040] Randomly collect 130,000 - 200,000 groups of gait temporal features within 30 - 90 seconds and label them as negative samples;

[0041] Use the positive samples and the negative samples as the training set of the support vector machine classifier for training to obtain a training result and form a support vector machine SVM training set.

[0042] Preferably, the system optimization module includes:

[0043] A performance monitoring unit for real - time monitoring of the difference between the system processing speed and a preset threshold;

[0044] A resource scheduling unit for dynamically adjusting the GPU computing resource allocation according to the output result of the performance monitoring unit;

[0045] A transfer learning unit for adapting system parameters between different hardware platforms to ensure system portability.

[0046] According to the gait abnormality real - time detection system based on dynamic optical flow field analysis, the processing process of the resource scheduling unit includes:

[0047] Set the system sampling frequency according to the input number of video frames;

[0048] After calculating the feature data of the current frame, calculate the average error between this frame and the current system;

[0049] When the error reaches the preset value or the error of three consecutive frames is less than 10%, do not use GPU acceleration;

[0050] When the difference between the system processing speed and the set detection speed is greater than 10%, use GPU acceleration;

[0051] When the system uses GPU acceleration, dynamically increase or decrease the number of GPU cores according to the number of GPU cores loaded and the GPU utilization rate of the current system to optimize the use of computing resources.

[0052] The present invention has the following beneficial effects:

[0053] 1. Non - contact monitoring: Collect RGB - D videos through a ceiling - mounted near - infrared depth camera, without the need for the subject to wear any devices, maintaining the naturalness of the gait and improving the reliability of the data.

[0054] 2. Multi-dimensional Feature Fusion: Extract hip-knee-ankle joint movement features from three dimensions, namely the sagittal plane, frontal plane, and cross-section, to construct a comprehensive gait representation and effectively capture subtle gait abnormality changes.

[0055] 3. Topological Feature Space Construction: Based on the dynamic optical flow field model and multi-level downsampling, map gait movements to the topological feature space to achieve precise identification of gait abnormalities in high-dimensional space.

[0056] 4. Adaptive Learning Ability: Through large-scale sample training and manifold learning, the system can adapt to individual differences and environmental changes, improving the accuracy and robustness of anomaly detection.

[0057] 5. Computational Efficiency Optimization: Adopt the GPU dynamic scheduling strategy to automatically adjust computing resources according to system load, optimizing resource utilization while ensuring real-time performance (processing speed can reach 45 frames per second).

[0058] 6. Portability and Scalability: The system adopts a modular design, supports transfer learning to adapt to different hardware platforms, and is convenient for deployment and upgrade in various application scenarios. Brief Description of the Drawings

[0059] Figure 1 It is a schematic diagram of the overall architecture of the system of the present invention;

[0060] Figure 2 It is a flowchart of the working process of the topological feature construction module in the present invention;

[0061] Figure 3 It is a schematic diagram of feature extraction of the spatio-temporal feature fusion module in the present invention;

[0062] Figure 4 It is a processing flowchart of the manifold distance anomaly detection module in the present invention;

[0063] Figure 5 It is a comparison diagram of the detection effects of the system of the present invention under different gait abnormality types. Detailed Embodiments

[0064] Please refer to the attached Figures 1-5 drawings. Below, in conjunction with the drawings and specific embodiments, the present invention will be further described in detail.

[0065] As Figure 1 shown, the real-time gait abnormality detection system based on dynamic optical flow field analysis provided by the present invention includes: a data acquisition module 10, a topological feature construction module 20, a spatio-temporal feature fusion module 30, a manifold distance anomaly detection module 40, and a system optimization module 50.

[0066] The data acquisition module 10 is used to obtain the gait RGB-D video sequence and extract the three-dimensional spatial data containing depth information. The topological feature construction module 20 is connected to the data acquisition module 10, and is used to receive the gait RGB-D video sequence, construct a dynamic optical flow field model, and extract key frame features through multi-level downsampling. The spatio-temporal feature fusion module 30 is connected to the topological feature construction module 20, and is used to extract joint motion features from three dimensions of the sagittal plane, frontal plane, and cross-section, calculate the change rate of the motion direction and the proportion of the stance phase time, and fuse the feature change rates to generate gait time series features. The manifold distance anomaly detection module 40 is connected to the spatio-temporal feature fusion module 30, and is used to receive the gait time series features, construct an abnormal gait recognition model based on the neural network algorithm and the support vector machine classifier, discriminate and classify the abnormal gait through modal division, and output the discrimination result. The system optimization module 50 is connected to the manifold distance anomaly detection module 40, and is used to dynamically adjust the GPU computing resources according to the system operation state to ensure the real-time performance of the system.

[0067] Preferably, the data acquisition module 10 includes a near-infrared depth camera 11, a data transmission unit 12, and a computer 13. The near-infrared depth camera 11 is installed on the ceiling and is used to obtain the RGB-D video sequence. In an embodiment of the present invention, the near-infrared depth camera 11 adopts an Intel RealSense D435i or a similar product, and this camera has a high depth accuracy (±2%) and a large field of view (87° horizontally and 58° vertically), and can effectively cover the walking area. The data transmission unit 12 is used to transmit the RGB-D video sequence to the computer 13 by wired or wireless means. In addition, the computer 13 is used to receive and store the RGB-D video sequence and provide a computing and processing platform.

[0068] In a preferred embodiment of the present invention, the topological feature construction module 20 includes an optical flow calculation unit 21, a dynamic optical flow field construction unit 22, and a key frame extraction unit 23. The optical flow calculation unit 21 is used to calculate the optical flow vector and the point cloud depth of adjacent frames in the RGB-D video from the current moment to the previous moment by using the improved pyramid Lucas-Kanade model.

[0069] The core idea of the improved pyramid Lucas-Kanade model is to calculate the optical flow at multiple resolution scales to handle large-scale motions. The specific algorithm is as follows:

[0070] For two given frames of images and , the optical flow vector at the position satisfies the following optical flow constraint equation:

[0071] ,

[0072] Where: is the image gray value at time at coordinate is the image gray value at time at coordinate is the displacement component in the x direction, with the unit of pixel; is the displacement component in the y direction, with the unit of pixel, represents the frame index in the video sequence, corresponding to the sampling point in the time series, with the unit of second; the coordinate system represents the pixel coordinates on the image plane, with the origin at the upper left corner of the image, the axis is horizontal to the right,

[0073] Since the above equation is non - linear, usually the first - order Taylor expansion is used for linear approximation:

[0074] ,

[0075] Where: is the partial derivative of the image in the direction, representing the gray - level change rate in the horizontal direction; is the partial derivative of the image in the direction, representing the gray - level change rate in the vertical direction.

[0076] Combined with the brightness constancy assumption, we get:

[0077] ,

[0078] Or written in vector form:

[0079] ,

[0080] Where: is the image gradient vector, representing the two - dimensional vector composed of the gray - level change rates of the image in the x and y directions; is the optical flow vector, representing the motion speed of the pixel; is the time derivative, representing the gray - level change at the same position in adjacent times.

[0081] In the pyramid structure, the optical flow is calculated starting from the coarsest level and then refined layer by layer:

[0082] ,

[0083] Where: is the The optical flow vector of the layer, a 2×1 vector, represents the pixel motion of the current layer; is the optical flow vector of the layer, also a 2×1 vector, representing the pixel motion at a coarser level; is the optical flow increment of the layer, a 2×1 vector, representing the additional motion correction amount that needs to be added to the current layer. The layer starts from the highest layer (coarsest resolution) and gradually decreases to 0 (original resolution).

[0084] In each layer, it is obtained by solving the following least squares problem :

[0085] ,

[0086] where: is a matrix containing the gradients of all pixels in the oral cavity, with a dimension of , is the number of pixels within the window; is the transpose matrix of, with a dimension of is the matrix the inverse matrix of, with a dimension of is the corresponding time derivative vector, with a dimension of , containing the time derivatives of all pixels within the window.

[0087] In the present invention, this algorithm is further combined with depth information to form an RGB-D optical flow field:

[0088] ,

[0089] where: is the RGB-D optical flow field vector, a 3D vector; is the two-dimensional optical flow vector , representing the planar motion at the position; is the depth value of the pixel

[0090] The dynamic optical flow field construction unit 22 is used to construct a dynamic optical flow field model based on the optical flow vector. The key frame extraction unit 23 is used to perform multi-level downsampling processing on the dynamic optical flow field data stream to obtain key frames.

[0091] In another embodiment of the present invention, the processing procedure of the key frame extraction unit 23 includes the following steps: First, compare the dynamic optical flow vectors selected at the current moment with those selected at the previous moment for N consecutive frames. Preferably, N is in the range of 5-10 frames, which can ensure stability without overly delaying the response time. Second, when the dynamic optical flow vectors at the current moment are consistent with those at the previous moment for N consecutive frames, update the dynamic optical flow vectors to those at the previous moment. In addition, when the dynamic optical flow vectors at the current moment are inconsistent with those at the previous moment for N consecutive frames, use the dynamic optical flow vectors at the current moment. Then, perform the first downsampling process on the optical flow field data stream at the current moment to obtain the first key frame. Finally, perform the second downsampling process on the first key frame to obtain the second key frame, completing the selection of key frames for all frames.

[0092] In this process, the downsampling strategy combines spatial uniform sampling and motion intensity weighting to ensure that key motion information is retained while reducing the data dimension. For the key frame after the second downsampling, if the sampling rate is lower than the threshold of 3 pixels per frame, output the second key frame at the previous moment to ensure the stability of features. Using 3 pixels per frame as the threshold is based on experimental findings that this value can achieve a good balance between data compression and feature retention.

[0093] The spatio-temporal feature fusion module 30 includes a feature extraction unit 31, a feature fusion unit 32, and a feature mapping unit 33. The feature extraction unit 31 is used to extract the change rate of the motion direction of the target object edge and the support phase time ratio from the key frame.

[0094] The feature extraction unit 31 extracts features through the following steps: First, determine the change rate of the motion direction of the target object edge in the key frame, where the change rate of the motion direction is obtained by calculating the optical flow vectors in the optical flow field model. Second, determine the support phase time ratio of the target object edge in the key frame, where the support phase time ratio is obtained by calculating the average value of the continuous occurrence times of the optical flow vectors in the optical flow field model and the optical flow vectors at the previous moment.

[0095] The calculation formula for the change rate of the motion direction is as follows:

[0096] ,

[0097] Where: is the change rate of the motion direction at the pixel point , with the unit of radian / second, representing the change amount of the optical flow direction per unit time; is the optical flow direction angle at the current moment at the point , with the unit of radian, and the calculation method is , where and are the and components of the optical flow vector at the current moment; is the optical flow direction angle at the previous moment at the point in radians; is the inter-frame time interval in seconds, usually set to 1 / fps. For example, when the fps is 30, seconds.

[0098] The formula for calculating the proportion of stance phase time is:

[0099] ,

[0100] where: is the proportion of stance phase time at the pixel point , which is a dimensionless ratio in the range of [0,1]; is the number of consecutive frames with consistent optical flow vector direction, that is, the change in the motion direction of this point in several consecutive frames is less than a preset threshold (usually π / 12 radians); is the total number of observed frames, usually set to the number of frames within a gait cycle (about 20 - 30 frames, depending on walking speed and camera frame rate).

[0101] The feature fusion unit 32 is used to non-linearly fuse the motion direction change rate and the proportion of stance phase time to generate gait temporal features. The fusion formula is as follows:

[0102] ,

[0103] where: is the fused gait temporal feature, which is a dimensionless scalar representing the comprehensive gait feature value of the point at time t; , and are weight coefficients, which are dimensionless scalar constants used to adjust the relative importance of different features; is the motion direction change rate of the point at time t; is the proportion of stance phase time of the point at time t; is the interaction term of the two features, capturing the non-linear relationship between the features. Preferably, take values in the ranges of 0.3 - 0.5, 0.3 - 0.5, and 0.1 - 0.3 respectively, and this configuration shows the best feature discrimination ability in experiments.

[0104] The feature mapping unit 33 is used to map gait temporal features to the topological feature space and construct high-dimensional features required for gait abnormality determination. This mapping process uses non-linear dimensionality reduction techniques such as t-SNE or UMAP to preserve the local topological structure of the data and effectively improve the separability of abnormal patterns.

[0105] The manifold distance anomaly detection module 40 includes a neural network processing unit 41, an SVM classification unit 42, and a discriminant decision-making unit 43. The neural network processing unit 41 is used to receive gait temporal features, process them through the bidirectional long short-term memory network algorithm, and establish a two-layer bidirectional long short-term memory network structure model including an input gate, a forget gate, an output gate, and a concatenation layer.

[0106] The structure of the bidirectional long short-term memory network (BiLSTM) can be expressed as:

[0107] ,

[0108] ,

[0109] ,

[0110] Where: is the hidden state of the forward LSTM, which is a d-dimensional vector, and d is the dimension of the hidden layer (usually taken as 128 or 256); is the hidden state of the backward LSTM, which is also a d-dimensional vector; is the input feature, which is an n-dimensional vector, and n is the feature dimension; is the set of network parameters of the forward LSTM, including the weights and biases of each gate; is the set of network parameters of the backward LSTM; is the weight matrix of the output layer, with dimension , and m is the output dimension; is the bias vector of the output layer, with dimension ; , represents concatenating the forward and backward hidden states into a 2d-dimensional vector; is the network output, with dimension , represents the hidden state of the forward LSTM at time, represents the hidden state of the backward LSTM at time.

[0111] The forward propagation formula of the LSTM unit is:

[0112] ,

[0113] ,

[0114] ,

[0115] ,

[0116] ,

[0117] Wherein: is the input gate, which controls the degree of new information entering the cell state. It is a d-dimensional vector with a value range of [0, 1]; is the forget gate, which controls the degree of retaining the previous cell state. It is a d-dimensional vector with a value range of [0, 1]; is the output gate, which controls the influence degree of the cell state on the current output. It is a d-dimensional vector with a value range of [0, 1]; is the cell state, which stores long-term memory and is a d-dimensional vector; is the hidden state, which serves as the output at the current moment and is a d-dimensional vector; , are weight matrices, which are respectively connected to the input and different gates or cell states; , , , are the corresponding bias vectors; is the sigmoid activation function, which compresses the input into the interval [0, 1] and is defined as ; tanh is the hyperbolic tangent function, which compresses the input into the interval [-1, 1] and is defined as ; represents the Hadamard product, that is, element-wise multiplication, represents the hidden state at time and is a dimensional vector; represents the cell state at time

[0118] In the present invention, the input feature is the gait temporal feature generated by the aforementioned spatio-temporal feature fusion module. After being processed by BiLSTM, a high-level feature representation capable of capturing long-term temporal dependence relationships is obtained.

[0119] The SVM classification unit 42 is used to construct a support vector machine classifier based on the output result of the neural network processing unit and analyze the specific parameters of the abnormal gait mode.

[0120] The goal of the SVM classifier is to find an optimal hyperplane that correctly separates the sample points of different classes and maximizes the margin. For the linearly separable case, the mathematical expression of SVM is:

[0121] ,

[0122] s.t. ,

[0123] where: w is the normal vector of the hyperplane, with the same dimension as the feature vector, which determines the direction of the hyperplane; b is the bias term, which is a scalar and determines the position of the hyperplane; is the feature vector of the i-th sample; ∈{-1,1} is the class label of the i-th sample, 1 represents normal gait, and -1 represents abnormal gait; m is the total number of training samples; is the square of the L2 norm of the vector w, that is , where n is the feature dimension; represents the dot product of the vectors w and , that is . s.t. means subject to, that is, the constraint condition is, indicating that the subsequent conditions are the constraints that the optimization problem must satisfy. The constraint condition requires that the functional margin of all sample points to the hyperplane is at least 1,

[0124] For the non-linear case, the kernel function is introduced to transform the problem into:

[0125] ,

[0126] s.t.

[0127] where: is the Lagrange multiplier, which is the decision variable of the optimization problem is the kernel function, which transforms the inner product calculation in the original feature space into the high-dimensional feature space, thus realizing non-linear classification; is the penalty parameter, which controls the penalty degree of misclassification, the larger it is, the heavier the penalty for misclassification.

[0128] Preferably, the present invention adopts the RBF kernel function:

[0129] ,

[0130] where: is the kernel parameter, which controls the complexity of the decision boundary, the larger it is, the more complex the decision boundary; is the feature vector and The squared Euclidean distance between them, i.e., . Empirically, Setting it within the range of 0.01 - 0.1 has a better effect, and the specific value can be determined through cross-validation.

[0131] The SVM classification unit constructs a training set through the following steps: First, randomly collect 10,000 - 50,000 groups of gait temporal features within 30 - 90 seconds and label them as positive samples. Second, randomly collect 130,000 - 200,000 groups of gait temporal features within 30 - 90 seconds and label them as negative samples. Finally, use the positive and negative samples as the training set of the support vector machine classifier for training to obtain the training results and form the support vector machine SVM training set.

[0132] Adopting a large-scale sample set is to improve the generalization ability of the model. The ratio of positive to negative samples is about 1:10 - 1:4. This ratio setting can effectively address the problem of class imbalance and at the same time ensure the sensitivity of the model to abnormal gaits.

[0133] The discrimination decision unit 43 is used to perform abnormal discrimination on the input gait data according to the output of the SVM classification unit in combination with the Mahalanobis distance metric method.

[0134] The formula for calculating the Mahalanobis distance is: ,

[0135] Where: is the Mahalanobis distance of the test sample, which is a scalar; is the feature vector of the test sample, and the dimension is , n is the feature dimension; is the mean vector of normal gait samples, and the dimension is also is the covariance matrix of normal samples, and the dimension is , which describes the correlation and variance between features; is the inverse matrix of the covariance matrix; is the transpose of the vector is the vector . The Mahalanobis distance takes into account the correlation and scale differences between features and can more accurately measure the degree of abnormality in high-dimensional space compared to the Euclidean distance.

[0136] The abnormal discrimination rule is:

[0137] ,

[0138] Where: The decision is the decision result for the sample , and the possible values are abnormal or normal, is the Mahalanobis distance of the sample ; is the anomaly threshold and is a scalar. Preferably, ranges from 3.0 to 5.0, and the specific value can be adjusted according to the sensitivity requirements of the application scenario. When the value is small, the system is more sensitive to anomalies but may increase the false alarm rate; when the value is large, the system is more conservative but may increase the miss rate. From a statistical perspective, for a high-dimensional normal distribution, the square of the Mahalanobis distance follows a chi-square distribution with degrees of freedom n. Therefore, = 3.0 corresponds to a confidence interval of approximately 99%, = 5.0 corresponds to a confidence interval of approximately 99.9999%.

[0139] The system optimization module 50 includes a performance monitoring unit 51, a resource scheduling unit 52, and a transfer learning unit 53. The performance monitoring unit 51 is used to monitor the difference between the system processing speed and the preset threshold in real time. The resource scheduling unit 52 is used to dynamically adjust the GPU computing resource allocation according to the output result of the performance monitoring unit. The transfer learning unit 53 is used to adapt the system parameters between different hardware platforms to ensure the portability of the system.

[0140] The processing process of the resource scheduling unit 52 includes the following steps: First, set the system sampling frequency according to the input number of video frames. In the preferred embodiment of the present invention, the sampling frequency is usually set to 25 - 30 frames per second, which can not only meet the requirements of human motion capture but also avoid generating too much redundant data. Second, after calculating the feature data of the current frame, calculate the average error between this frame and the current system. In addition, when the error reaches the preset value (such as 5%) or the error of three consecutive frames is less than 10%, GPU acceleration is not used. At the same time, when the difference between the system processing speed and the set detection speed is greater than 10%, GPU acceleration is used. Finally, when the system uses GPU acceleration, dynamically increase or decrease the number of GPU cores according to the number of GPU cores loaded by the current system and the GPU utilization rate to optimize the use of computing resources.

[0141] Preferably, the GPU utilization rate is maintained within the range of 60% - 80%, which can not only ensure the computing efficiency but also avoid excessive resource consumption leading to system heating and increased energy consumption. When the utilization rate exceeds 80%, increase the number of GPU cores; when it is lower than 60%, reduce the number of GPU cores.

[0142] In another embodiment of the present invention, the topology feature construction module 20 improves the key frame extraction strategy and introduces an adaptive sampling mechanism:

[0143] ,

[0144] where: is the sampling weight of the pixel point and is a scalar within the range of [0, 1], which determines the probability of this point being sampled; is the exercise intensity, usually represented by the magnitude of the optical flow vector, that is , the larger the value, the more intense the movement; is the gradient magnitude, representing the structural complexity of the image at this point, usually calculated from the Euclidean distance of the image gradients in the x and y directions, that is ; and are weight coefficients, satisfying , [0, 1], used to balance the importance of motion information and structural information. Preferably, , that is, paying more attention to motion information but also retaining a certain amount of structural information.

[0145] This mechanism dynamically adjusts the sampling density according to the motion intensity and edge information of the image region, ensuring that the features in the vigorously moving regions and the edge parts are better retained, while reducing the sampling points in the static regions and smooth regions, further improving the pertinence and computational efficiency of feature extraction.

[0146] In another embodiment of the present invention, the spatio-temporal feature fusion module 30 introduces a multi-scale time window analysis technique:

[0147] ,

[0148] where: is the multi-scale fusion feature, which is the eigenvalue after comprehensively considering multiple time scales; is the feature at the th time scale, calculated by the aforementioned feature fusion formula at different time window lengths; is the weight coefficient of the kth time scale, satisfying ; is the total number of time scales, usually taking 3 - 5 different scales. represents the weighted sum of the features of K time scales to achieve the fusion of multi-scale features.

[0149] By setting time windows of different lengths (such as a short window of 0.5 seconds, a medium window of 1 second, and a long window of 2 seconds), the system can simultaneously capture fast and slow gait abnormality patterns, improving the adaptability to different types of gait abnormalities. Preferably, the time window length is set to [0.5 seconds, 1 second, 2 seconds], and the corresponding weights are [0.25, 0.5, 0.25], that is, the importance of the medium time scale is the highest.

[0150] In another embodiment of the present invention, the manifold distance anomaly detection module 40 adopts an ensemble learning strategy, combining the decision results of multiple base classifiers:

[0151] ,

[0152] Wherein: is the sample The probability of being determined as abnormal (label y = 1), and the value range is [0, 1]; is the th base classifier gives the probability that the sample is abnormal, and the value range is also [0, 1]; is the number of base classifiers, and usually 3 - 7 different types of classifiers are taken; represents taking the arithmetic mean of the prediction results of M base classifiers to obtain the integrated prediction result.

[0153] This strategy reduces the bias of a single model and improves the robustness and generalization ability of the system by integrating the results of multiple classifiers such as SVM, random forest, and gradient boosting tree. The decision rule is adjusted to:

[0154] ,

[0155] Wherein: is the probability threshold, and the preferred value is 0.6 - 0.7, that is, when most classifiers consider the sample to be abnormal, the system makes an abnormal decision, reducing the false alarm risk.

[0156] In another embodiment of the present invention, the system optimization module 50 introduces an adaptive quantization technology to dynamically adjust the model accuracy according to the computing power of the hardware platform:

[0157] ,

[0158] Wherein: is the quantized weight value, which has the same dimension and physical meaning, but the accuracy is reduced; is the original weight value; is the quantization bit number, which determines the quantization accuracy. The larger n is, the higher the accuracy; round() is the rounding function that rounds the value to the nearest integer; and The operations are equivalent to scaling, quantizing, and then scaling the value back to the original range, realizing n-bit fixed-point representation. For example, when n = 8, the model weights are represented by 8-bit fixed-point numbers, and the accuracy is 1 / 256.

[0159] By dynamically adjusting the quantization bits, the system can maintain the best performance-accuracy balance on different computing platforms and operate efficiently from high-performance workstations to embedded devices. Preferably, different quantization bits are set according to the platform performance: 16-32-bit floating-point calculation is used on high-performance GPU platforms; 8-bit fixed-point calculation is used on medium-performance platforms; 4-6-bit fixed-point calculation is used on low-performance embedded devices. Experiments show that when reducing from 32-bit floating-point to 8-bit fixed-point, the calculation speed can be increased by 3-4 times, while the accuracy loss is only 1%-2%, which is a very effective optimization strategy.

[0160] The real-time gait abnormality detection system based on dynamic optical flow field analysis of the present invention has been verified in multiple clinical application scenarios. In the detection of gait abnormalities in Parkinson's disease patients, the system achieved a detection accuracy rate of 93.2% and an early warning rate of 89.5%, which is 15-20% higher than traditional methods. In the fall risk assessment of the elderly, the system's prediction accuracy rate reached 87.6%, and the average early warning time was 2.3 seconds, providing sufficient time for intervention measures. The system of the present invention also detected gait abnormalities after stroke, with an accuracy rate of 90.1%, which is 16.6% higher than traditional methods. Similarly, for gait abnormalities caused by orthopedic diseases, the system also achieved a detection accuracy rate of 86.3%, superior to traditional methods.

[0161] In addition, in terms of system resource utilization, the optimized GPU dynamic scheduling strategy reduces the average energy consumption by 32% while maintaining a real-time processing ability of 45 frames per second, meeting the real-time requirements of clinical monitoring.

[0162] In summary, the real-time gait abnormality detection system based on dynamic optical flow field analysis provided by the present invention realizes high-precision, real-time, and non-contact gait abnormality detection through innovative technologies such as multi-dimensional feature fusion, topological feature space construction, and manifold distance anomaly detection, providing an effective technical means for medical rehabilitation, elderly care, and early screening of neurological diseases.

[0163] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A real-time gait abnormality detection system based on dynamic optical flow field analysis, characterized in that, Including: A data acquisition module, configured to acquire a gait RGB-D video sequence and extract three-dimensional spatial data containing depth information; A topological feature construction module, connected to the data acquisition module, configured to receive the gait RGB-D video sequence, construct a dynamic optical flow field model, and extract key frame features through multi-level downsampling; A spatio-temporal feature fusion module, connected to the topological feature construction module, configured to extract joint motion features from three dimensions of the sagittal plane, frontal plane, and cross-section, calculate the motion direction change rate and the proportion of stance phase time, and perform non-linear fusion on the motion direction change rate and the proportion of stance phase time to generate gait temporal features; A manifold distance anomaly detection module, connected to the spatio-temporal feature fusion module, configured to receive the gait temporal features, construct an abnormal gait recognition model based on a neural network algorithm and a support vector machine classifier, perform discriminant classification on abnormal gaits through modal division, and output a discriminant result; A system optimization module, connected to the manifold distance anomaly detection module, configured to dynamically adjust GPU computing resources according to the system operation state to ensure the real-time performance of the system; The spatio-temporal feature fusion module includes: A feature extraction unit, configured to extract the motion direction change rate and the proportion of stance phase time of the edge of the target object from the key frames; A feature fusion unit, configured to perform non-linear fusion on the motion direction change rate and the proportion of stance phase time to generate gait temporal features; A feature mapping unit, configured to map the gait temporal features to a topological feature space to construct high-dimensional features required for gait anomaly determination; The fusion formula is as follows: ， Wherein: is the fused gait temporal feature, which is a dimensionless scalar representing the point The comprehensive gait feature value at time t; , β, and γ are weight coefficients, which are dimensionless scalar constants used to adjust the relative importance of different features; is the point The rate of change of the movement direction at time t; is the point The proportion of the stance phase time at time t; is the interaction term of the two features, capturing the non-linear relationship between the features; The feature extraction unit extracts features through the following steps: Determine the motion direction change rate of the edge of the target object in the key frame, where the motion direction change rate is obtained by calculating the optical flow vector in the optical flow field model; Determine the proportion of stance phase time of the edge of the target object in the key frame, where the proportion of stance phase time is obtained by calculating the average number of consecutive occurrences of the optical flow vector in the optical flow field model and the optical flow vector of the previous moment; The formula for calculating the proportion of stance phase time is: ， Wherein: is the time occupancy ratio of the support phase at the pixel point, which is a dimensionless ratio with a range of [0, 1]; is the number of consecutive frames in which the optical flow vector direction remains consistent, that is, the change in the motion direction of this point in several consecutive frames is less than the preset threshold, which is π / 12 radians; is the total number of observed frames, which is set to the number of frames within the gait cycle; ​ The manifold distance anomaly detection module includes: A neural network processing unit, configured to receive the gait temporal features, process them through a bidirectional long short-term memory network algorithm, and establish a two-layer bidirectional long short-term memory network structure model including an input gate, a forgetting gate, an output gate, and a splicing layer; An SVM classification unit, configured to construct a support vector machine classifier based on the output result of the neural network processing unit and analyze the specific parameters of the abnormal gait mode; A discriminant decision unit, configured to perform anomaly discrimination on the input gait data according to the output of the SVM classification unit in combination with the Mahalanobis distance metric method; The formula for calculating the Mahalanobis distance is: , Wherein: is the Mahalanobis distance of the test sample, which is a scalar; is the feature vector of the test sample, with the dimension of , and n is the feature dimension; is the mean vector of the normal gait samples, with the same dimension of is the covariance matrix of the normal samples, with the dimension of , which describes the correlation and variance between features; is the inverse matrix of the covariance matrix; is the transpose of the vector ; ; The anomaly discrimination rule is: ， Where: the judgment is the judgment result of the sample , and the possible values are abnormal or normal, is the Mahalanobis distance of the sample ; is the anomaly threshold, which is a scalar.

2. The real-time gait abnormality detection system based on dynamic optical flow field analysis according to claim 1, wherein The data acquisition module includes: A near-infrared depth camera, installed on the ceiling, configured to acquire an RGB-D video sequence; A data transmission unit, configured to transmit the RGB-D video sequence to a computer by wire or wirelessly; A computer, configured to receive and store the RGB-D video sequence and provide a computing and processing platform.

3. The real-time gait abnormality detection system based on dynamic optical flow field analysis according to claim 1, characterized in that, The topological feature construction module includes: An optical flow calculation unit, which is used to calculate the optical flow vector and point cloud depth of adjacent frames in the RGB-D video from the previous moment to the current moment by using the improved pyramid Lucas-Kanade model; A dynamic optical flow field construction unit, which is used to construct a dynamic optical flow field model based on the optical flow vector; A key frame extraction unit, which is used to perform multi-level downsampling processing on the data stream of the dynamic optical flow field to obtain key frames.

4. The real-time gait abnormality detection system based on dynamic optical flow field analysis according to claim 3, wherein The processing process of the key frame extraction unit includes: Comparing the dynamic optical flow vectors selected at the current moment with those selected at the previous moment for N consecutive frames; When the dynamic optical flow vectors at the current moment are consistent with those at the previous moment for N consecutive frames, updating the dynamic optical flow vectors to those at the previous moment; When the dynamic optical flow vectors at the current moment are inconsistent with those at the previous moment for N consecutive frames, taking the dynamic optical flow vectors at the current moment as the dynamic optical flow vectors at the current moment; Performing the first downsampling processing on the data stream of the optical flow field at the current moment to obtain the first key frame; Performing the second downsampling processing on the first key frame to obtain the second key frame, and completing the selection of key frames for all frames.

5. The real-time gait abnormality detection system based on dynamic optical flow field analysis according to claim 1, characterized in that The SVM classification unit constructs a training set through the following steps: Randomly collecting 10,000 - 50,000 groups of gait timing features within 30 - 90 seconds and labeling them as positive samples; Randomly collecting 130,000 - 200,000 groups of gait timing features within 30 - 90 seconds and labeling them as negative samples; Using the positive samples and the negative samples as the training set of the support vector machine classifier for training to obtain a training result, and forming a support vector machine SVM training set.

6. The real-time gait abnormality detection system based on dynamic optical flow field analysis according to claim 1, characterized in that The system optimization module includes: A performance monitoring unit, which is used to monitor the difference between the system processing speed and the preset threshold in real time; A resource scheduling unit, which is used to dynamically adjust the GPU computing resource allocation according to the output result of the performance monitoring unit; A transfer learning unit, which is used to adapt system parameters between different hardware platforms to ensure the portability of the system.

7. The real-time gait abnormality detection system based on dynamic optical flow field analysis according to claim 6, characterized in that, The processing process of the resource scheduling unit includes: Setting the system sampling frequency according to the input number of video frames; After calculating the feature data of the current frame, calculating the average error between this frame and the current system; When the error reaches the preset value or the errors of three consecutive frames are all less than 10%, not using GPU acceleration; When the difference between the system processing speed and the set detection speed is greater than 10%, using GPU acceleration; When the system uses GPU acceleration, dynamically increasing or decreasing the number of GPU cores according to the number of GPU cores loaded in the current system and the GPU utilization rate to optimize the utilization of computing resources.

Images