Rehabilitation patient gait image analysis and training system based on deep learning

Abstract

The application discloses a rehabilitation patient gait image analysis and training system based on deep learning, relates to the technical field of medical rehabilitation, can realize comprehensive capture on the gait process through side view and rear view dual-view video acquisition, and generate joint sequence data through pretreatment to provide accurate input for analysis. The posture stability index is calculated and the exercise tolerance index is generated to quantify the exercise adaptability of the patient in the gait cycle. Through the adjustment coefficient mapping and threshold adjustment mechanism, the internal threshold is dynamically generated, so that the feature extraction process can adaptively change with the patient state, breaking through the limitations of the traditional fixed threshold method. The fused gait feature vector is analyzed by the classifier, and the probability distribution of multiple abnormal gait patterns is output, and an analysis report containing training suggestions is generated. It not only improves the accuracy and reliability of gait recognition, but also provides personalized guidance for rehabilitation training, and has significant clinical application value and popularization potential.

Description

Technical Field

[0001] This invention relates to the field of medical rehabilitation technology, specifically a deep learning-based system for analyzing and training gait images of rehabilitation patients. Background Technology

[0002] With the development of rehabilitation medicine and intelligent monitoring technology, gait analysis has gradually become an important means of evaluating the motor function of rehabilitation patients and developing personalized training programs. Existing gait analysis systems are mostly based on video acquisition or sensor monitoring, identifying and assessing abnormal gait by extracting indicators such as joint movement trajectories and body posture stability during the patient's walking process. However, due to individual differences among rehabilitation patients during training, their gait performance often changes dynamically with tolerance and training load. Traditional systems often rely on fixed judgment thresholds or static evaluation rules, making it difficult to comprehensively reflect the patient's motor adaptability and training capacity at different times.

[0003] In this context, existing methods lack flexibility in threshold setting, easily leading to mismatches between assessment results and the patient's actual condition. For example, some patients experience a decline in postural stability after a short period of training, but because the judgment threshold is fixed, the system fails to provide timely and appropriate feedback, resulting in delayed rehabilitation training guidance. Therefore, how to introduce a dynamic adjustment mechanism related to the patient's exercise tolerance during gait feature analysis and training suggestion generation, thereby achieving adaptation between the assessment process and the patient's actual recovery state, has become an urgent technical problem to be solved. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a deep learning-based system for analyzing and training gait images of rehabilitation patients.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] This invention discloses a deep learning-based gait image analysis and training system for rehabilitation patients, comprising:

[0007] The data acquisition and preprocessing module is used to acquire and preprocess dual-view gait video data of the user walking from the side and the rear, and generate time-synchronized side view joint point sequence data and rear view joint point sequence data.

[0008] The index calculation module is used to calculate the user's posture stability index in real time during walking based on the side-view joint sequence data and the back-view joint sequence data, and to generate the exercise tolerance index according to the changing trend of the posture stability index within the complete gait cycle.

[0009] An adjustment coefficient mapping module is configured to obtain a real-time adjustment coefficient according to the exercise tolerance index and a preset mapping relationship, wherein the preset mapping relationship is configured such that when the value of the exercise tolerance index increases, the output real-time adjustment coefficient tends to a first preset value; and when the value of the exercise tolerance index decreases, the output real-time adjustment coefficient tends to a second preset value.

[0010] A threshold adjustment module is configured to perform operation on the real-time adjustment coefficient and a predefined reference internal threshold in the multi-view feature fusion model to generate a dynamic internal threshold.

[0011] A model processing module is configured to input the side-view joint sequence data and the rear-view joint sequence data into the multi-view feature fusion model, adopt the dynamic internal threshold as a feature saliency criterion, and output a fused gait feature vector.

[0012] An abnormality analysis and report generation module is configured to input the gait feature vector into an abnormal gait classifier to obtain probability distribution data for indicating multiple abnormal gait modes, and generate a gait analysis report containing abnormal mode recognition results and training suggestions.

[0013] Compared with the prior art, the present application has the following beneficial effects:

[0014] 1. By associating the exercise tolerance index with the preset mapping relationship, the real-time adjustment coefficient is output, and the dynamic internal threshold is generated in combination with the reference threshold in the multi-view feature fusion model, so that the model can be adaptively adjusted according to the patient's state; the rigidification problem caused by the dependence of the traditional method on the fixed threshold is effectively avoided, and the gait feature extraction and saliency determination are more in line with the actual recovery of the patient.

[0015] 2. The dynamic threshold is used as the basis to improve the accuracy of the analysis results and the training suggestions in the final report, which not only helps doctors and therapists to timely grasp the recovery progress of the patient, but also provides targeted training feedback for the patient, significantly improving the gait evaluation accuracy and individualized guidance level in the rehabilitation training. BRIEF DESCRIPTION OF DRAWINGS

[0016] The disclosure of the present application will be described with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present application. In the drawings, the same reference numerals are used to refer to the same components. Among them:

[0017] Figure 1 is a system architecture diagram of the present application;

[0018] Figure 2 is a data flow diagram of the present application;

[0019] Figure 3The logical diagram of the preset mapping relationship in the application. DETAILED DESCRIPTION

[0020] It is easy to understand that, according to the technical scheme of the application, a person skilled in the art can propose a plurality of structure modes and implementation modes which can be replaced with each other without changing the essential spirit of the application. Therefore, the following specific embodiments and drawings are only exemplary descriptions of the technical scheme of the application, and should not be regarded as the whole or as a limitation or restriction on the technical scheme of the application.

[0021] SUMMARY

[0022] In the conventional existing gait analysis system, the fixed determination threshold and static evaluation rules cannot adapt to the dynamic changes of the exercise tolerance and training load of the rehabilitation patient. When the patient's posture stability has nonlinear fluctuations within the gait cycle, the system cannot establish a dynamic correlation between the evaluation parameters and the physiological state, resulting in a mismatch between the feature significance criterion and the real-time exercise ability. This static threshold mechanism reduces the accuracy of multi-view feature fusion, causing the feature vector received by the abnormal gait classifier to contain redundant noise, ultimately affecting the timeliness and pertinence of the training recommendations.

[0023] For example, in the gait rehabilitation training scene, after the patient completes three sets of weight-bearing walking training, the standard deviation of the pelvic swing angle increases from the initial 2.3 degrees to 5.1 degrees, and the absolute value of the linear fitting slope of the trunk lateral tilt angle increases by 40%. At this time, the conventional system still uses the preset baseline internal threshold to screen the cross-view feature correlation, causing the key pelvic rotation features to be misjudged as noise and excluded. The gait feature vector output by the multi-view feature fusion model loses the key data dimension representing the decline in tolerance, and the abnormal classifier incorrectly identifies the compensatory gait as a normal pattern, generating training recommendations for continuously increasing the load of the deep squat action.

[0024] If the above problems are not solved, the misidentification of abnormal gait patterns will cause the training program to be misaligned with the actual recovery stage of the patient, exacerbating the risk of exercise injury. The rigidity of the feature screening mechanism will hinder the system from capturing the critical state of the transformation of compensatory gait to pathological gait, delaying the optimal timing of rehabilitation intervention. The time asynchronous processing defect of multi-modal data will also cause phase deviation of kinematic parameters and physiological indicators, reducing the reliability of gait cycle segmentation and stability index calculation, and ultimately forming a negative feedback loop to affect the iterative optimization of individualized rehabilitation programs.

[0025] To address the aforementioned issues, this application first considers establishing a dynamic correlation mechanism between assessment parameters and physiological states. Traditional systems use fixed thresholds to screen cross-view feature correlations, leading to key kinematic parameters being misclassified as noise and failing to reflect changes in patient tolerance. To resolve this, this application attempts to dynamically couple the exercise tolerance index with the feature screening threshold, adjusting the internal criteria of the multi-view feature fusion model through real-time adjustment coefficients. Further analysis reveals that relying solely on single-view data or static mapping relationships is insufficient to capture nonlinear fluctuations within the gait cycle; therefore, dual-view time-series data needs to be introduced to collaboratively calculate the tolerance index, and a dynamic threshold is generated based on its changing trend. By designing a linkage mechanism between the adjustment coefficient mapping module and the threshold adjustment module, the feature significance criterion is adaptively adjusted according to the patient's actual state, thereby resolving the problems of assessment lag and feature mismatch.

[0026] In this regard, such as Figure 1 As shown, this application proposes a deep learning-based gait image analysis and training system for rehabilitation patients, including:

[0027] The data acquisition and preprocessing module is used to acquire and preprocess dual-view gait video data of the user walking from the side and rear, generating time-synchronized side-view joint point sequence data and rear-view joint point sequence data. The data acquisition and preprocessing module refers to the component used to synchronously acquire and process user walking video data from both the side and rear views. Specifically, it can be implemented using a dual-camera array combined with a timestamp alignment algorithm. Its role is to ensure the spatiotemporal consistency of multi-view joint point data and provide accurate input for subsequent analysis.

[0028] The index calculation module is used to calculate the user's postural stability index in real time during walking based on lateral and posterior joint sequence data. It then generates an exercise tolerance index based on the changing trend of the postural stability index within the complete gait cycle. The postural stability index is a quantitative parameter reflecting the user's body balance during walking, specifically achieved by calculating dynamic characteristics such as trunk tilt angle and pelvic swing amplitude. Its function is to monitor changes in the patient's exercise stability in real time, providing a basis for the tolerance index calculation. The exercise tolerance index is a dynamic assessment value characterizing the user's sustained exercise ability. It is specifically calculated by weighting the standard deviation and trend slope of the stability index within the gait cycle, and its function is to quantify the patient's adaptation to the current training load.

[0029] An adjustment coefficient mapping module is configured to obtain a real-time adjustment coefficient according to the motion tolerance index and a preset mapping relationship, wherein the preset mapping relationship is configured such that when the value of the motion tolerance index increases, the output real-time adjustment coefficient tends to a first preset value; when the value of the motion tolerance index decreases, the output real-time adjustment coefficient tends to a second preset value; the real-time adjustment coefficient refers to a scaling factor dynamically adjusted according to the tolerance index, and can be specifically implemented by combining the preset mapping relationship with a change rate correction mechanism, and functions to convert the change in the patient state into a quantitative parameter for threshold adjustment.

[0030] A threshold adjustment module is configured to perform operation on the real-time adjustment coefficient and a predefined reference internal threshold in the multi-view feature fusion model to generate a dynamic internal threshold; the dynamic internal threshold refers to a feature screening criterion that changes with the adjustment coefficient, and can be specifically implemented by combining a linear combination of the reference threshold and the adjustment coefficient with a change rate correction term, and functions to adapt the feature fusion process to the real-time state fluctuation of the patient.

[0031] A model processing module is configured to input the side-view joint sequence data and the rear-view joint sequence data into a multi-view feature fusion model, adopt the dynamic internal threshold as a feature saliency criterion, and output a fused gait feature vector; the multi-view feature fusion model refers to a processing architecture integrating side-view and rear-view joint data, and can be specifically implemented by adopting a cross-view correlation matrix screening and weighted fusion algorithm, and functions to improve the robustness of abnormal gait detection.

[0032] An abnormality analysis and report generation module is configured to input the gait feature vector into an abnormal gait classifier to obtain probability distribution data indicating a plurality of abnormal gait patterns, and generate a gait analysis report containing abnormal pattern recognition results and training suggestions; the abnormal gait classifier refers to a gait pattern recognition component based on probability distribution, and can be specifically implemented by adopting a prototype vector similarity matching combined with a normalization processing algorithm, and functions to accurately identify a plurality of abnormal gait types and generate targeted suggestions.

[0033] The core innovation of the present application lies in that a dynamic threshold adjustment mechanism driven by the motion tolerance index is used to quantize the real-time motion stability change of the patient into an adjustment coefficient, and then dynamically control the feature screening threshold in the multi-view feature fusion process, so that the system can adaptively adjust the analysis strategy according to the current tolerance degree of the patient, and solve the technical defect that the traditional fixed threshold method cannot adapt to the dynamic change of the patient state.

[0034] As shown in Figure 2 , it is a data flowchart in the present application; as a preferred embodiment, the scheme of the present application is implemented as follows:

[0035] The data acquisition preprocessing module uses a high-speed camera to collect video data of the user walking on the treadmill from the side and rear, respectively. The video frame rate is set to 120 fps, and the resolution is 1920x1080. The collected video data is processed in time synchronization to ensure the time alignment of the side and rear data. Then the 18 human body key point coordinates in each frame of image are extracted by the human body key point detection algorithm, and the side and rear joint sequence data are generated.

[0036] The index calculation module calculates the trunk inclination angle based on the side joint sequence data and the pelvic swing angle based on the rear joint sequence data. The two sets of angle data are normalized and weighted according to the preset weight (e.g. trunk inclination 0.6, pelvic swing 0.4) to obtain the comprehensive stability index time series data. In a complete gait cycle (usually 1-2 seconds), the standard deviation of the index is calculated as the instantaneous fluctuation amplitude, and the absolute value of the linear fitting slope is calculated as the stability change trend value. These two values are input into the pre-defined tolerance index calculation function to obtain the exercise tolerance index.

[0037] The adjustment coefficient mapping module obtains the real-time adjustment coefficient according to the current exercise tolerance index.

[0038] The threshold adjustment module calculates the dynamic internal threshold value by operating the real-time adjustment coefficient with the pre-defined baseline internal threshold value.

[0039] The model processing module uses a pre-trained multi-view feature fusion neural network model to process the input joint sequence data.

[0040] The abnormality analysis and report generation module uses an abnormal gait classifier to perform multi-classification on the gait feature vector to obtain the probability distribution of various abnormal gait patterns (such as limping, foot drop, etc.). Based on the highest probability abnormal pattern, the system selects the corresponding rehabilitation training action from the pre-set training action library to generate a personalized gait analysis report and training suggestion.

[0041] Through the above scheme, this application achieves dynamic adaptation between the gait analysis process and the patient's actual recovery state. The introduction of an exercise tolerance index and a real-time adjustment coefficient mechanism allows the feature significance criterion to adaptively adjust according to the patient's state, solving the problem of dynamic correlation between assessment parameters and physiological state. Collaborative computation of dual-view time-series data improves the ability to capture nonlinear fluctuations within the gait cycle. The application of dynamic internal thresholds improves the accuracy of multi-view feature fusion and reduces redundant noise in the feature vectors received by the abnormal gait classifier. This adaptive mechanism improves the accuracy of abnormal gait pattern recognition, making the generated training suggestions more timely and targeted, thereby reducing the risk of misalignment between the training plan and the patient's actual recovery stage. It helps to promptly capture the critical state of the transformation from compensatory gait to pathological gait, providing more accurate timing judgments for rehabilitation intervention.

[0042] This application further proposes the following processing logic for the preset mapping relationship:

[0043] The real-time adjustment coefficient is weighted and corrected based on the rate and direction of change of the exercise tolerance index.

[0044] Specifically, when the exercise tolerance index continues to increase and the rate of change exceeds a first threshold, the real-time adjustment coefficient is accelerated to approach a first preset value; when the exercise tolerance index continues to decrease and the rate of change exceeds a second threshold, the real-time adjustment coefficient is accelerated to approach a second preset value; when the exercise tolerance index fluctuates between the first threshold and the second threshold, the real-time adjustment coefficient remains smoothly adjusted.

[0045] The preset mapping relationship is the core algorithm of the adjustment coefficient mapping module, and its purpose is to adjust the exercise tolerance index. The dynamic change characteristics (including the direction and rate of change) are converted into real-time adjustment coefficients. A sophisticated control strategy. The following combines... Figure 3 The processing logic is described in detail.

[0046] The direction of change in the exercise tolerance index determines whether the user's condition is continuously improving or deteriorating, while the rate of change in the exercise tolerance index is used to quantify the speed of improvement or deterioration.

[0047] Based on these two dimensions, three differentiated adjustment methods can be implemented:

[0048] When a user's condition is detected to be rapidly and continuously improving or deteriorating, the model's sensitivity is adjusted at an accelerated rate instead of a uniform adjustment. This is equivalent to an "emergency response" mechanism, enabling it to adapt more quickly to drastic changes in the user's condition and provide more accurate analysis or stronger alerts in a timely manner.

[0049] When the user state is in a fluctuation or stable period, i.e., the change rate is within the threshold range, the system adopts a smooth adjustment strategy. This ensures the stability of the analysis and prevents the output recommendations from frequently jumping due to minor fluctuations in the data, providing a consistent and reliable experience for the user.

[0050] The first threshold and the second threshold are set according to the fluctuation range of the exercise tolerance index in historical data, and the first preset value and the second preset value correspond to the upper limit and the lower limit of the adjustment coefficient, respectively.

[0051] Through the above technical solutions, the present application realizes dynamic adjustment of the real-time adjustment coefficient. According to the change trend of the exercise tolerance index, the system can quickly respond to changes in the patient's state and adjust the internal threshold in a timely manner. When the patient's exercise tolerance improves, the system increases the internal threshold to increase the tolerance of minor gait abnormalities; when the patient's exercise tolerance decreases, the system reduces the internal threshold to increase the sensitivity to gait abnormalities. This dynamic adjustment mechanism enables the system to more accurately reflect the real-time state of the patient, avoiding the evaluation lag problem caused by fixed thresholds and providing more accurate guidance for rehabilitation training.

[0052] The real-time adjustment coefficient is calculated according to the above logic , which is an iterative calculation based on time series. The core is to intelligently adjust the coefficient according to the real-time changes of the user's exercise state , please combine the following content for detailed explanation:

[0053] First, determine whether the current time point is the initial moment of calculation (e.g., after the system starts or after the first calculation period after starting to track a new user).

[0054] If it is the initial time point: due to the lack of historical data (the real-time adjustment coefficient of the previous moment, the exercise tolerance index does not exist), the table lookup method is used for initialization.

[0055] The system pre-stores a tolerance index-adjustment coefficient mapping table, which defines the initial correspondence between the exercise tolerance index and the real-time adjustment coefficient . This relationship is usually set as a monotonic function, for example, The higher the initial value of tends to the first preset value (e.g., 1.2), indicating that the initial state gives higher trust to stable gait; The lower the initial value of tends to the second preset value (e.g., 0.8), indicating that the initial state is more sensitive to abnormal gait.

[0056] If not the initial time point: then perform the following dynamic iteration calculation process.

[0057] Obtain the real-time adjustment coefficient of the last time , the exercise tolerance index of the last time ;

[0058] Calculate the exercise tolerance index relative to the change rate of the last time :

[0059] , is the time interval between the current time and the last time, in the system based on fixed frame rate video stream, is a constant (such as 1 / 30 seconds). The formula calculation obtained a clear physical meaning (unit: index / s) change rate.

[0060] At the same time, by the sign function get the direction of change:

[0061] When , at this time, it means that the state is improving.

[0062] When , at this time, it means that the state is deteriorating.

[0063] When , at this time, it means that the state is stable.

[0064] The real-time adjustment coefficient of the current time is calculated by the following formula:

[0065] ;

[0066] The above formula is composed of three parts of accumulation, which together determine the value of the real-time adjustment coefficient ;

[0067] The real-time adjustment coefficient of the last time as the basis of calculation, ensures the continuity and smoothness of the change between the real-time adjustment coefficient at different times.

[0068] As an acceleration adjustment term, this term is responsible for accelerating the response to rapid changes in state;

[0069] Where only when the absolute value of the change rate Exceeding a preset threshold When the item is not 0; the threshold value changes according to the direction:

[0070] When the value is positive, the value is the first threshold value;

[0071] When the value is negative, the value is the second threshold value;

[0072] is the first preset weight factor, which is configured by the staff according to historical data experience value; when the change is violent, this item will produce a larger adjustment amount, whose positive and negative are determined by , so as to drive to change rapidly towards the target value .

[0073] As the smoothing convergence item of the formula, this item is responsible for driving to converge smoothly to the target value , preventing excessive fluctuations in the final result; is the difference between the current value and the target value, is the second preset weight factor, which is a parameter between 0 and 1, configured by the staff according to historical data experience value.

[0074] Whether the change rate exceeds the threshold, this item always exists. It is like a "damper" to ensure can stably tend to the target value . The value of the target value is determined by the change direction:

[0075] When the value is positive, is the first preset value (a higher value, such as 1.2).

[0076] When the value is negative, is the second preset value (a lower value, such as 0.8).

[0077] By the technical solution, the dynamic calculation of the real-time adjustment coefficient is realized. Thus, the system can adaptively adjust the real-time adjustment coefficient according to the change trend and rate of the exercise tolerance index. When the exercise tolerance index rapidly rises, the real-time adjustment coefficient will accelerate to approach the first preset value; when the exercise tolerance index rapidly falls, the real-time adjustment coefficient will accelerate to approach the second preset value. The dynamic adjustment mechanism makes the system more sensitive to capture the change of the patient's exercise state, thereby providing more accurate reference for subsequent gait analysis and training suggestion generation. Further, by introducing the first preset weight factor and the second preset weight factor, the system can flexibly control the response degree of the real-time adjustment coefficient to the change of the exercise tolerance index, thereby balancing the sensitivity and stability. The adaptive adjustment mechanism effectively improves the adaptability of the system to the change of the patient's exercise state, and lays a foundation for realizing personalized and precise rehabilitation training guidance.

[0078] The dynamic internal threshold is obtained by operating the real-time adjustment coefficient and a predefined reference internal threshold in the multi-view feature fusion model by the threshold adjustment module; the threshold is not a fixed value, but a dynamic variable that is adaptively adjusted according to the real-time state and state change trend of the user, and is a key to realizing intelligent screening of the multi-view feature fusion model. The calculation process is as follows:

[0079] The real-time adjustment coefficient at the current time is obtained , and the change rate of the exercise tolerance index relative to the last time ;

[0080] The dynamic internal threshold is calculated by the following formula:

[0081] ;

[0082] is a correction amount calculated based on the change rate of the real-time adjustment coefficient, and the data is used to quantify the change speed of the real-time adjustment coefficient, and the value , wherein is the difference between the real-time adjustment coefficients at the current time and the last time, is the time interval between the two times; the absolute value operation ensures that is always a non-negative value, and the physical meaning is the degree of change of the adjustment coefficient.

[0083] As a steady-state adjustment item, the reference internal threshold is linearly scaled according to the absolute state level (represented by ) of the user at the current time, and is the main body and basis of the threshold adjustment. is a predefined baseline internal threshold of the multi-view feature fusion model, representing the baseline strictness of judging feature saliency in normal or average state. is a preset linear amplification parameter, used to control the influence amplitude of the baseline threshold. as a scaling factor. When increases (usually indicating that the user's state is stable or improving), the factor is greater than 1, causing the dynamic internal threshold to rise. This means that the model will become more "strict" or "conservative", requiring higher correlation between features to be considered salient, thereby reducing the likelihood of false positives. Conversely, when decreases, the dynamic internal threshold decreases, and the model becomes more "sensitive", able to capture more subtle abnormal features.

[0084] as a correction term in the formula, which instantaneously corrects the dynamic internal threshold according to the user's change trend (represented by ), providing the system with rapid response capability.

[0085] The larger the value, the faster the user's functional state is changing (whether improving or deteriorating). This term is always positive. Therefore, when increases, the value of this term increases, resulting in a positive increment to the final dynamic internal threshold .

[0086] The way the dynamic internal threshold is calculated gives the system the ability to respond in advance to sudden changes in state. For example, when the user's state begins to deteriorate rapidly (at this time may not have decreased to a very low level, but has already become very large), the dynamic correction term will immediately raise the threshold, forcing the fusion model to be more vigilant and capture abnormal feature patterns resulting from sudden changes in state earlier, thereby achieving early warning. This greatly enhances the safety protection performance of the system.

[0087] Specifically, in one embodiment, the baseline internal threshold can be set to 0.5, and the linear amplification parameter can be set to 0.2. Assuming that the real-time adjustment coefficient at the current time is 0.8, the real-time adjustment coefficient at the previous time is 0.7, and the time interval is 0.1 seconds, then the value of is

[0088] Assuming the rate of change of the exercise tolerance index relative to the previous time is 0.05, then the dynamic internal threshold can be calculated as follows:

[0089] .

[0090] Through the above technical solutions, the present application realizes the adaptive adjustment of the dynamic internal threshold. By introducing the real-time adjustment coefficient and the rate of change of the exercise tolerance index, the dynamic internal threshold can be adjusted in real time according to the exercise state of the patient. When the patient's exercise tolerance improves, the threshold will be increased accordingly, making the system more stringent in screening gait features. Conversely, when the patient's exercise tolerance decreases, the threshold will be lowered, allowing the system to capture more potential abnormal features. This dynamic adjustment mechanism improves the accuracy and sensitivity of gait analysis, enabling the system to better adapt to the state changes of the patient during rehabilitation, providing a more reliable basis for developing personalized rehabilitation training programs.

[0091] Linear amplification parameter is set to be a key parameter for personalized dynamic adjustment according to individual differences and the current state of the user, enabling the system to adapt to the gait stability features of different users and respond more accurately to abnormal states that deviate from their normal state. The following details the process.

[0092] First, the historical exercise tolerance index sequence of the current user is obtained from the storage unit. This sequence records multiple exercise tolerance index values of the user over a period of time (e.g., the past week or the past 10 training sessions).

[0093] Based on this historical data sequence, the system calculates two core statistics: mean , standard deviation ; the mean represents the average exercise function level under normal conditions of the user, which is an individualized baseline; the standard deviation quantifies the volatility of the user's historical state. The larger the value, the more intense the gait stability fluctuations of the user; the smaller the value, the more stable the state.

[0094] Then calculate the offset of the current exercise tolerance index from the mean : ; this offset indicates the specific degree to which the user's current state deviates from their own normal average level. The larger the value, the more abnormal the current state.

[0095] Finally, linear amplification parameter is dynamically calculated by the following formula:

[0096] ;

[0097] As the preset linear amplification base value, determines the adjustment reference level of parameter .

[0098] As a whole, it is adjusted according to the individual situation of the user to scale the base value.

[0099] The hyperbolic tangent function maps the input to the output interval of , and has the characteristics of smooth saturation. Specifically:

[0100] When the value of is near 0, the output of approximately linearly increases.

[0101] When the value of is very large (much larger than 1) or very small (much smaller than -1), the output of will gradually tend to 1 or -1, and will not increase indefinitely.

[0102] In actual application, since and are both positive numbers, the output range of is between

[0103] As a preset reference offset, it is a preset constant used to normalize the offset . Its role is to define a reference standard for "significant offset". For example, it can be set as a multiple of the historical standard deviation (such as ), as a dimensionless ratio, is used to judge the severity of the current offset.

[0104] The standard deviation is the coefficient of the output value of . For users with large historical state fluctuations ( large), the result will be amplified, making the adjustment range of larger. This implements a "tolerance" strategy: for users whose state is inherently unstable, the system allows parameter to have a larger range of variation, avoiding overreaction to their single state fluctuations.

[0105] For a user with stable state (S small), if its current state is abnormal (A large), the formula will produce a moderate increase in the system threshold, thereby acutely capturing its abnormal state.

[0106] For a user with large state fluctuations (S large), the same abnormal state (A large) will result in a significant increase. However, the increase in the dynamic threshold will cause the sensitivity of the system to its state changes to be relatively reduced. This is a reasonable "inertia" strategy: for a user with large fluctuations, the system will not be alarmed by every fluctuation, and will only respond strongly when the fluctuation is significantly abnormal, preventing false positives for a specific user group.

[0107] Through the above technical solutions, the application realizes dynamic adjustment of the linear amplification parameter. Thus, the system can adaptively adjust the linear amplification parameter according to the fluctuation degree of the user's historical gait data and the deviation degree of the current real-time adjustment coefficient from the historical mean value. Further, this dynamic adjustment mechanism enables the system to more sensitively respond to changes in the user's gait characteristics when calculating the dynamic internal threshold, thereby improving the accuracy and individualization of gait analysis.

[0108] The application further proposes a generation process for the side-view joint sequence data and the rear-view joint sequence data as follows:

[0109] Through the side-view image acquisition device and the rear-view image acquisition device, the side-view gait video sequence and the rear-view gait video sequence of the user on the preset walking path are obtained; the side-view image acquisition device and the rear-view image acquisition device are arranged at fixed positions on the user's walking path, ensuring that the dual-view covers the complete posture changes in the user's walking process.

[0110] The side-view gait video sequence and the rear-view gait video sequence are subjected to time synchronization processing; the time synchronization processing includes a periodic gait signal extraction module that generates a synchronization reference signal by detecting the periodic characteristics of the foot ground contact event in the video sequence.

[0111] From the synchronized video data, the time sequence coordinate data of the predefined multiple human body key points are extracted, thereby generating the side-view joint sequence data and the rear-view joint sequence data. The predefined human body key points include the ankle joint, the knee joint, the hip joint, and the pelvic center point, and their coordinate data are extracted frame by frame through a key point detection model based on a convolutional neural network.

[0112] After the lateral and rear-view gait video sequences are simultaneously acquired, periodic gait signals are extracted from the foot movement trajectories of the two video sequences. A cross-correlation calculation module analyzes the phase difference between the two periodic signals to determine the frame offset. Based on the frame offset, the frame rate of the rear-view video sequence is converted to align its gait period phase with that of the lateral view video sequence. The synchronized video data is input into a keypoint detection model, which outputs two-dimensional coordinate temporal data containing the ankle, knee, hip, and pelvic center points. The keypoint coordinate data undergoes filtering and smoothing to eliminate coordinate jitter caused by acquisition noise, ultimately generating time-aligned lateral and rear-view keypoint sequence data. This process eliminates the temporal deviation between the two viewpoints, ensuring the spatiotemporal consistency of subsequent posture stability index calculations and improving the accuracy of multi-view feature fusion.

[0113] Through the above technical solution, this application achieves high-precision acquisition and synchronization of multi-view gait data. By employing high-frame-rate image acquisition equipment and a cross-correlation-based time synchronization method, the temporal consistency of keypoint sequence data from different perspectives is ensured. This provides a reliable data foundation for subsequent multi-view feature fusion and abnormal gait analysis, helping to improve the accuracy and comprehensiveness of gait analysis. Simultaneously, the extraction of predefined key points simplifies the data processing flow and improves the system's real-time performance. This multi-view, highly synchronized data acquisition method can more comprehensively capture the gait characteristics of rehabilitation patients, providing richer information support for the development of personalized training programs.

[0114] This application further proposes time synchronization processing for the side-view gait video sequence and the rear-view gait video sequence, including:

[0115] Periodic gait signals are extracted from the side-view gait video sequence and the rear-view gait video sequence, respectively. The periodic gait signals are generated by detecting key movements such as heel touching the ground or toe leaving the ground in consecutive video frames, and are represented by waveforms with a fixed frequency.

[0116] The frame offset between the two video sequences is determined by calculating the cross-correlation between the periodic gait signals. The cross-correlation is calculated using the sliding window method, where the correlation coefficients of the two signals at different time offsets are plotted as cross-correlation function curves, and the offset corresponding to the peak of the curve is taken as the frame offset.

[0117] Frame rate conversion is performed on one of the video sequences based on the frame offset to align the gait cycle phases of the two video sequences. The frame rate conversion process adjusts the time interval between video frames using an interpolation algorithm to ensure that the start point and duration of the gait cycle in the two video sequences are consistent.

[0118] Periodic gait signal extraction employs foot motion trajectory tracking technology based on optical flow to capture displacement changes at key foot points and generate periodic waveforms. During cross-correlation calculation, the sliding window length is set to the average duration of a complete gait cycle, ensuring signal comparison is performed on the same time scale. Frame rate conversion utilizes a cubic spline interpolation algorithm to dynamically adjust the frame rate of the target video sequence, for example, converting a 30 fps video to 25 fps while maintaining gait cycle phase alignment with the video from the other viewpoint. Through these steps, the video sequences from the two viewpoints achieve precise synchronization on the time axis, eliminating gait phase deviations caused by differences in device acquisition, and providing a foundation for time alignment for subsequent cross-viewpoint keypoint data fusion.

[0119] Through the above technical solution, this application achieves precise time synchronization of multi-view gait video sequences, providing a foundation for subsequent multi-view feature fusion. This improves the accuracy and reliability of gait analysis and helps to more comprehensively capture changes in patients' gait characteristics from different perspectives.

[0120] The process for generating the further exercise tolerance index in this application is as follows:

[0121] Based on the side-view key point sequence data, the tilt angle of the torso key points relative to the vertical direction in each frame is calculated to obtain the tilt angle time sequence data.

[0122] Based on the rear-view joint sequence data, the angle between the line connecting the key points of the pelvis in each frame and the horizontal direction is calculated to obtain the time sequence data of the pelvic swing angle.

[0123] The time series data of lateral tilt angle and pelvic swing angle are normalized and then weighted and fused based on predefined weight coefficients to generate time series data of comprehensive stability index.

[0124] Extract the time series data of the comprehensive stability index within a complete gait cycle time window, calculate the standard deviation of the data within the window as the instantaneous fluctuation amplitude, and calculate the absolute value of the linear fitting slope of the data within the window as the stability change trend value.

[0125] The instantaneous fluctuation amplitude and the stability change trend value are input into a predefined tolerance index calculation function to calculate the sports tolerance index; the tolerance index calculation function is configured such that an increase in the instantaneous fluctuation amplitude and the stability change trend value both lead to a decrease in the output value of the sports tolerance index.

[0126] The tilt angle of the trunk key points was calculated using the coordinates of mid-spine markers, and the direction of the line connecting the left and right anterior superior iliac spines was calculated using the coordinates of the pelvic key points. Normalization was performed using a minimum-maximum scaling method to map the angle data to the 0-1 interval. Weighting coefficients were set to 0.6 for lateral tilt angle and 0.4 for pelvic swing angle based on clinical data statistics. The complete gait cycle time window was determined by detecting heel strike events, and the standard deviation was calculated using a sliding window method to eliminate boundary effects. The linear fitting slope was calculated using the least squares method. The tolerance index calculation function was implemented as an exponential decay function, with its input parameters assigned negative weighting coefficients.

[0127] Specifically, by extracting two complementary stability indices—trunk lateral tilt and pelvic coronal plane sway—from dual-view joint point data, the limitations of single-view assessment are eliminated. Normalization ensures the comparability of data with different dimensions, and weighted fusion enhances the contribution of key indicators. Time window analysis based on the complete gait cycle captures the stability fluctuation characteristics in periodic movements; standard deviation quantifies the instantaneous degree of posture deviation, and the absolute value of the slope reflects the trend of stability deterioration. The tolerance index calculation function nonlinearly combines the fluctuation amplitude and the trend of change; when the patient's posture stability decreases, the output index decreases, triggering subsequent modules to lower the training intensity threshold. This calculation process quantifies dynamically changing parameters, enabling the exercise tolerance assessment to respond in real time to changes in the patient's training status.

[0128] As a preferred embodiment, the solution of this application is implemented as follows: During gait analysis, the lateral view image acquisition device and the rear view image acquisition device simultaneously acquire the user's walking video sequence, and use a human posture estimation algorithm to extract the temporal coordinates of key points of the torso and key points of the pelvis. Based on the lateral view data, the angle between the midline of the torso and the vertical line is calculated in each frame to form a lateral tilt angle sequence; based on the rear view data, the angle between the line connecting the left and right anterior superior iliac spines of the pelvis and the horizontal plane is calculated in each frame to form a pelvic swing angle sequence. The two sets of angle data are respectively processed by maximum value normalization, and linearly weighted with a weight coefficient of 0.6:0.4 to generate a comprehensive stability index curve. After detecting a complete gait cycle, the comprehensive stability data segment within that cycle is extracted, its standard deviation is calculated as the instantaneous fluctuation amplitude, and the least squares linear fitting is performed on the data segment to extract the absolute value of the slope as the stability change trend value.

[0129] Input the fluctuation amplitude and trend value into the exponential function The output range is the exercise tolerance index, which is between 0 and 1.

[0130] The above technical solution effectively solves the feedback lag problem caused by fixed thresholds in traditional gait assessment systems. By dynamically fusing multi-view posture stability indicators and quantifying their fluctuation characteristics, it can accurately capture the instantaneous changes in the patient's tolerance during training. Combined with the tolerance index function's inverse adjustment mechanism for fluctuation amplitude and trend value, the output index can reflect the patient's physical load status in real time, providing a reliable basis for subsequent training intensity adjustments.

[0131] The core function of the model processing module is to fuse keypoint sequence data from both lateral and rearward perspectives into a feature vector that comprehensively and efficiently represents the gait pattern. This process is not a simple data concatenation, but an intelligent fusion based on dynamic feature saliency discrimination; the specific process is as follows:

[0132] First, the correlation strength between keypoints from different perspectives is quantified. This is done by calculating a cross-perspective feature correlation matrix. This is achieved through [the following]. The matrix [is used]. The size is ,in It is the number of selected features in the side-view keypoint sequence. It is the number of selected features in the rear-view keypoint sequence.

[0133] Matrix elements Represents the first in the side-view sequence The first feature and the first in the back-look sequence The correlation strength between features. Features can be joint coordinates, joint angles, velocities, etc. The correlation can be calculated using methods such as Pearson correlation coefficient and cosine similarity.

[0134] Obtain the correlation matrix Subsequently, the system does not fuse all feature pairs, but instead uses the dynamic internal threshold at the current moment. Perform significance screening. Analyze the correlation matrix. Each matrix element With dynamic internal threshold Compare and filter out all that meet the criteria. The elements are defined. The feature pairs corresponding to these elements are considered "significantly relevant" feature pairs in the current user state. The system also records the matrix indices of these significant elements. .

[0135] Then, based on all the matrix indices recorded in the previous step... The system extracts features from the side-view keypoint sequence data. Extracting the corresponding features from the rear-view key point sequence data These data pairs form key data points. .

[0136] Weighted summation generates preliminary fusion features: for each selected data pair The system then fuses these features according to predefined weights. These predefined weights can be uniform weights or different weights assigned based on prior knowledge (e.g., the knee joint angle has a higher weight than the ankle joint). Initial feature fusion. It is obtained by summing or averaging the fusion results of all significant data pairs, and its formal representation is as follows:

[0137] ;

[0138] Where k iterates through all salient feature pairs. These are the weights of the corresponding data pairs. It is a fusion function: for example, direct concatenation. Or calculate its average .

[0139] Finally, in order to preserve the global information of the original data and avoid losing useful background information during the filtering process, the system performs the following operations:

[0140] Pooling operations, such as global average pooling or global max pooling, are performed on the original side-view and rear-view key sequence data respectively. This step reduces the dimensionality of the temporal data for each viewpoint to a fixed global feature vector. and .

[0141] Preliminary fusion features representing significant local features Pooled features representing global information and The features are concatenated to form a composite feature vector containing information at multiple granularities. .

[0142] Finally, the concatenated composite feature vector Dimensionality reduction is performed. This can be achieved through Principal Component Analysis (PCA) or a lightweight fully connected neural network layer. The goal is to eliminate redundant information, reduce feature dimensionality, decrease the computational burden on subsequent classifiers, and improve feature representation efficiency. The vector output after dimensionality reduction is the final, fused gait feature vector, which will serve as input to the abnormal gait classifier.

[0143] The process for generating the further probability distribution data in this application is as follows:

[0144] The fused gait feature vector is compared with multiple predefined abnormal gait pattern prototype vectors to calculate the similarity, resulting in a set of similarity scores. The similarity calculation uses cosine similarity or Euclidean distance as a metric, and the normalization is achieved using the softmax function.

[0145] The similarity scores are normalized and converted into a probability distribution.

[0146] Each probability value in the probability distribution data corresponds to an abnormal gait pattern, and its magnitude represents the confidence level that the input gait feature vector belongs to the corresponding abnormal gait pattern.

[0147] The process of generating the training suggestions is as follows:

[0148] The abnormal gait pattern corresponding to the maximum value in the probability distribution data is determined as the main abnormal pattern.

[0149] Query the predefined training action library associated with the main anomaly pattern; the predefined training action library stores the mapping relationship between anomaly patterns and training actions, with each main anomaly pattern mapped to more than three training actions;

[0150] At least one training action is selected from the training action library, and a descriptive instruction containing the training action is generated. The descriptive instruction includes the action execution frequency, intensity parameters, and precautions.

[0151] When the fused gait feature vector is input into the anomaly classifier, its similarity to the prototype vectors of each anomaly pattern is calculated to form a multi-dimensional similarity score set. For example, when using cosine similarity calculation, the directional consistency between the feature vector and the prototype vector in 128-dimensional space is quantified. After the similarity scores are normalized by the softmax function, they are converted into a probability distribution form, with the sum of probability values ​​being 1. The dominant anomaly pattern is determined by comparing the maximum value in the probability distribution; for example, the hemiplegic gait with a probability value of 0.65 is identified as the dominant anomaly pattern. The training action library pre-stores actions such as four-point support training and center of gravity transfer training corresponding to the hemiplegic gait. The system automatically selects the first two training actions to generate instructions. The action intensity parameter in the instructions is dynamically adjusted according to the probability value. When the probability of the dominant anomaly pattern exceeds 0.6, the training intensity is set to a moderate load. Thus, the training suggestion generation process realizes the quantitative assessment of the confidence of the anomaly pattern and the targeted action recommendation, effectively improving the adaptability of the rehabilitation training program.

[0152] As a preferred embodiment, the solution of this application is implemented as follows: During gait analysis, the fused gait feature vector is input to the anomaly classification module. This module stores seven predefined prototype vectors of abnormal gait patterns, including typical patterns such as hemiplegic dribble gait, Parkinson's shuffling gait, and quadriceps weakness gait. An initial similarity score set is obtained by calculating the cosine similarity between the input feature vector and each prototype vector. This score set is normalized using a softmax function and converted into probability distribution data, where each probability value corresponds to the confidence level of an abnormal gait pattern.

[0153] When the probability value corresponding to the quadriceps weakness gait in the probability distribution data reaches 0.68, this pattern is identified as the primary abnormal pattern. The system accesses the preset training action library and retrieves three training actions associated with this pattern: straight leg raise training, resisted knee extension training, and step gait training. Two of these training actions are selected, and descriptive instructions containing the action name, training duration, and number of repetitions are generated, such as "Perform straight leg raise training, 10 repetitions per set, 3 sets per day" and "Step gait training, 5 minutes each time, twice a day".

[0154] Through the above technical solution, this application achieves dynamic recognition of abnormal gait patterns based on probability distribution, solving the problem of insufficient classification sensitivity caused by fixed judgment rules in traditional systems. By matching the main abnormal pattern with a pre-set training action library, targeted rehabilitation training suggestions can be automatically generated, avoiding the lag of manual intervention and effectively improving the real-time performance and adaptation accuracy of personalized rehabilitation guidance.

[0155] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A deep learning-based gait image analysis and training system for rehabilitation patients, characterized in that: include: The data acquisition and preprocessing module is used to acquire and preprocess dual-view gait video data of the user walking from the side and the rear, and generate time-synchronized side view joint point sequence data and rear view joint point sequence data. The index calculation module is used to calculate the user's posture stability index in real time during walking based on the side-view joint sequence data and the back-view joint sequence data, and to generate the exercise tolerance index according to the changing trend of the posture stability index within the complete gait cycle. The adjustment coefficient mapping module is used to obtain the real-time adjustment coefficient based on the exercise tolerance index and the preset mapping relationship. The preset mapping relationship is configured such that when the value of the exercise tolerance index increases, the output real-time adjustment coefficient approaches the first preset value; when the value of the exercise tolerance index decreases, the output real-time adjustment coefficient approaches the second preset value. The threshold adjustment module is used to calculate the real-time adjustment coefficient with the predefined benchmark internal threshold in the multi-view feature fusion model to generate a dynamic internal threshold. The model processing module is used to input the side-view joint sequence data and the rear-view joint sequence data into the multi-view feature fusion model, use the dynamic internal threshold as the feature salience criterion, and output the fused gait feature vector. The anomaly analysis and report generation module is used to input the gait feature vector into the abnormal gait classifier to obtain probability distribution data for indicating multiple abnormal gait patterns, and generate a gait analysis report containing abnormal pattern recognition results and training suggestions.

2. The deep learning-based gait image analysis and training system for rehabilitation patients according to claim 1, characterized in that: The processing logic for the preset mapping relationship is as follows: The real-time adjustment coefficient is weighted and corrected based on the rate and direction of change of the exercise tolerance index. Specifically, when the exercise tolerance index continues to increase and the rate of change exceeds a first threshold, the real-time adjustment coefficient is accelerated to approach a first preset value; when the exercise tolerance index continues to decrease and the rate of change exceeds a second threshold, the real-time adjustment coefficient is accelerated to approach a second preset value; when the exercise tolerance index fluctuates between the first threshold and the second threshold, the real-time adjustment coefficient remains smoothly adjusted.

3. The deep learning-based gait image analysis and training system for rehabilitation patients according to claim 2, characterized in that: The calculation process for the real-time adjustment coefficient is as follows: Determine if the current time point is the initial time point; If the judgment result is yes, then the real-time adjustment coefficient is obtained by querying the preset tolerance index-adjustment coefficient mapping table based on the current exercise tolerance index. If the result is negative, then perform the following operations: Obtain the real-time adjustment coefficient of the previous moment. The exercise tolerance index at the previous moment ; Calculate the exercise tolerance index rate of change relative to the previous moment : , The time interval between the current moment and the previous moment; Calculate the real-time adjustment coefficient at the current moment using the following formula. : ； in, This is a sign function used to characterize the direction of change; As the first preset weighting factor, This is the second preset weighting factor; when When the value is positive, The first preset value, The first threshold; when When the value is negative, The second preset value, This is the second threshold.

4. The deep learning-based gait image analysis and training system for rehabilitation patients according to claim 1, characterized in that: The calculation process for the dynamic internal threshold is as follows: Obtain the real-time adjustment coefficient at the current moment. and the rate of change of the exercise tolerance index relative to the previous moment. ; Calculate dynamic internal threshold : ； in, As the baseline internal threshold, These are the preset linear amplification parameters; The correction amount is calculated based on the rate of change of the real-time adjustment coefficient, and its value is... ,in This is the difference between the real-time adjustment coefficient at the current time and the previous time. The time interval between the two moments is denoted as .

5. The deep learning-based gait image analysis and training system for rehabilitation patients according to claim 4, characterized in that: The linear amplification parameter The calculation is based on the user's historical gait data, and the specific process is as follows: Obtain historical data on the user's exercise tolerance index, and calculate the standard deviation of the exercise tolerance index based on the historical data. and mean ; Calculate the current exercise tolerance index with the mean offset : ； Based on the offset Calculate the linear amplification parameters : ； in, The preset linear amplification base value, This is the preset reference offset. It is the hyperbolic tangent function.

6. The deep learning-based gait image analysis and training system for rehabilitation patients according to claim 1, characterized in that: The generation process of the side-view joint sequence data and the rear-view joint sequence data is as follows: The system acquires side-view gait video sequences and rear-view gait video sequences of the user on a preset walking path using a side-view image acquisition device and a rear-view image acquisition device. The side-view gait video sequence and the rear-view gait video sequence are time-synchronized. From the synchronized video data, the temporal coordinate data of multiple predefined human body key points are extracted to generate the side view joint point sequence data and the rear view joint point sequence data.

7. The deep learning-based gait image analysis and training system for rehabilitation patients according to claim 6, characterized in that: The time synchronization processing of the side-view gait video sequence and the rear-view gait video sequence includes: Periodic gait signals are extracted from the side-view gait video sequence and the rear-view gait video sequence, respectively; The frame offset between the two video sequences is determined by calculating the cross-correlation between the periodic gait signals. The frame rate of one of the video sequences is converted based on the frame offset to align the gait cycles of the two video sequences.

8. The deep learning-based gait image analysis and training system for rehabilitation patients according to claim 1, characterized in that: The process of generating the exercise tolerance index is as follows: Based on the side-view key point sequence data, the tilt angle of the torso key points relative to the vertical direction in each frame is calculated to obtain the tilt angle time sequence data. Based on the rear-view joint sequence data, the angle between the line connecting the key points of the pelvis in each frame and the horizontal direction is calculated to obtain the time sequence data of the pelvic swing angle. The time series data of the tilt angle and the pelvic swing angle are normalized and then weighted and fused based on predefined weight coefficients to generate time series data of comprehensive stability index. Extract the time series data of the comprehensive stability index within a complete gait cycle time window, calculate the standard deviation of the data within the window as the instantaneous fluctuation amplitude, and calculate the absolute value of the linear fitting slope of the data within the window as the stability change trend value. The instantaneous fluctuation amplitude and the stability change trend value are input into a predefined tolerance index calculation function to calculate the sports tolerance index; the tolerance index calculation function is configured such that an increase in the instantaneous fluctuation amplitude and the stability change trend value both lead to a decrease in the output value of the sports tolerance index.

9. The deep learning-based gait image analysis and training system for rehabilitation patients according to claim 1, characterized in that: The process of generating the gait feature vector is as follows: Calculate the cross-view feature correlation matrix between the side-view keypoint sequence data and the rear-view keypoint sequence data; Each element in the cross-view feature correlation matrix is ​​compared with the dynamic internal threshold; Filter out all correlation matrix elements that are greater than the dynamic internal threshold and record their matrix indices; Based on the recorded matrix index, the corresponding key point data pairs are extracted from the side-view key point sequence data and the rear-view key point sequence data; The selected key point data are weighted and summed according to predefined weight values ​​to generate preliminary fusion features; The preliminary fused features are concatenated with the original joint sequence data processed by pooling, and the dimensionality of the concatenated feature vector is reduced to output the final fused gait feature vector.

10. The deep learning-based gait image analysis and training system for rehabilitation patients according to claim 1, characterized in that: The process of generating the probability distribution data is as follows: The fused gait feature vector is compared with multiple predefined abnormal gait pattern prototype vectors to calculate the similarity, resulting in a set of similarity scores. The similarity scores are normalized and converted into a probability distribution. Each probability value in the probability distribution data corresponds to an abnormal gait pattern, and its magnitude represents the confidence level that the input gait feature vector belongs to the corresponding abnormal gait pattern. The process of generating the training suggestions is as follows: The abnormal gait pattern corresponding to the maximum value in the probability distribution data is determined as the main abnormal pattern. Query the predefined training action library associated with the main anomaly pattern; Select at least one training action from the training action library and generate a descriptive instruction containing the training action.

Images