Improved YOLOv8-Pose artificial limb user gait coordination scoring method

Abstract

The invention relates to an improved YOLOv8-Pose-based artificial limb user gait coordination scoring method, which comprises the following steps of: firstly, acquiring coordinates and confidence coefficients of 17 key points of a human body in the whole process by using an improved YOLOv8-Pose model frame by frame for a video of finishing a TUG test of an artificial limb user, and acquiring diagonal vertex coordinates of a detection frame; thirdly, dividing the video into a forward walking part and a reverse walking part by utilizing the height change of the detection frame and the shoulder breadth change of the artificial limb user, dividing a complete gait cycle in forward walking and a complete gait cycle in reverse walking by utilizing the coordinate change of the ankle joint, and calculating gait conventional parameters; and then, calculating a gait coordination index by using the collected key point coordinates. And finally, independently scoring each item of part of gait conventional parameters and all gait coordination indexes by referring to the standard of a normal person. And standardizing the gait routine parameters and the gait coordination indexes, inputting the standardized gait routine parameters and the standardized gait coordination indexes into the trained MLP classifier, and calculating a comprehensive score by using a classification probability.

Description

Technical Field

[0001] The present invention relates to the field of human pose estimation in computer vision, and in particular to a gait coordination scoring method for prosthetic users based on improved YOLOv8-Pose Background Art

[0002] In recent years, key point detection models have developed rapidly in the field of artificial intelligence and have important significance in fields such as medical health, sports, and human-computer interaction. The human pose estimation task focuses on detecting the coordinates and confidence of important human joints to understand human movements. It should be noted that different from general key point detection tasks, the uniqueness of the human pose estimation task lies in: (1) It is necessary to complete the connection of important joints based on the human body structure and evaluate the human pose on this basis. (2) Usually, constraint conditions of relevant human body structure knowledge, such as reasonable connection relationships of joints, are used to optimize the detection results of key points. In specific usage scenarios, professional standards need to be combined, such as determining the normal standard of joint angles in the field of medical rehabilitation. (3) Top-down and bottom-up methods are commonly used. The top-down method first detects the human body frame and then estimates the pose of the human body within each frame, but the computational complexity increases rapidly with the increase in the number of people; while the bottom-up method first detects all key points and then groups the key points into different human body instances through post-processing, but the post-processing is complex. In addition, the human pose evaluation model needs to adapt to human pose changes in different scenarios. Therefore, it is difficult to evaluate the human pose in complex scenarios

[0003] Benefiting from the rapid development of deep learning, convolutional neural networks play a key role in human key point detection models, and the detection functions of various key point detection models are excellent. YOLOv8-Pose has a fast inference speed while maintaining high accuracy. Due to the special application scenario, in order to better apply to the scenario of scoring the gait coordination of prosthetic users, it is very important to improve the YOLOv8-Pose model and analyze and score the gait parameters of prosthetic users Summary of the Invention

[0004] The purpose of the present invention is to propose a gait coordination scoring method for prosthetic users based on improved YOLOv8-Pose, which can accurately and effectively score gait coordination

[0005] To achieve the above purpose, the technical solution of the present invention is: A gait coordination scoring method for prosthetic users based on improved YOLOv8-Pose, comprising the following steps:

[0006] Step S1: Obtain the video of the prosthetic user completing the TUG test. Use the improved YOLOv8-Pose model to perform human key point detection frame by frame on the video, obtain the coordinates and confidence levels of 17 human key points throughout the process, and obtain the diagonal vertex coordinates of the detection box.

[0007] Step S2: Use the change in the height of the detection box and the change in the shoulder width of the prosthetic user to divide the walking state of the prosthetic user, divide it into the forward walking part and the reverse walking part, and then use the change in the ankle joint coordinates to divide a complete gait cycle in forward walking and a complete gait cycle in reverse walking, and calculate the conventional gait parameters.

[0008] Step S3: Calculate the gait coordination index of the prosthetic user using the collected key point coordinates.

[0009] Step S4: Refer to the standards of normal people for some conventional gait parameters and all gait coordination indexes, and perform individual scoring for each item; standardize the measured conventional gait parameters and gait coordination indexes, input them into the trained MLP classifier, and use the classification probability to calculate the comprehensive score.

[0010] Preferably, the video of the TUG test is taken from the front right side of the subject and ensure that the subject is within the frame throughout the TUG test. In the video, the part from the subject standing up to preparing to turn around is defined as the forward walking part, and the part from the end of turning around to returning to the front of the chair and preparing to sit down is defined as the reverse walking part. The video shooting stops when the subject starts to move until completely sitting down.

[0011] Preferably, the improved YOLOv8-Pose model is specifically as follows:

[0012] Add a CBAM introducing residual connection after the SPPF module in the Backbone layer of the YOLOv8-Pose model. CBAM is the convolutional block attention mechanism. Through the cascaded structure of the channel attention module and the spatial attention module, it learns the importance of different channels and spatial positions in the input feature map to improve the feature extraction ability of the model. The main calculation method of CBAM after introducing residual connection is as follows:

[0013]

[0014] Figure is the input feature map, satisfying Figure ∈ R size_C×size_H×size_W , where R represents the set of real numbers, size_C is the number of channels of the input feature map, size_H is the height of the input feature map, size_W is the width of the input feature map, is the element-wise multiplication operation combined with the broadcast mechanism, and is the result of the operation of the channel attention module on Figure, Figure coutIt is a figure with a residual connection introduced after channel attention enhancement, which is for Figure cout The result of performing operations on the spatial attention module, Figure sout is a figure with a residual connection introduced after spatial attention enhancement;

[0015] Optimize the Neck layer. Add CBAM with a residual connection also introduced after the C2f module in the P3 / 8 branch, and the input channel number of CBAM is 256 to enhance feature representation; add CBAM with a residual connection also introduced after the C2f module in the P4 / 16 branch, and the input channel number of CBAM is 512 to further improve the quality of features; add CBAM with a residual connection also introduced after the C2f module in the P5 / 32 branch, and the input channel number of CBAM is 1024 to enable the model to better focus on key features;

[0016] Replace all convolutional modules Conv in the Neck layer with GSConv, and the GSConv is a lightweight convolution method that combines standard convolution and depthwise separable convolution;

[0017] During the model training stage, use the loss function Loss optimized based on MPDIoU, where MPDIoU is the intersection over union based on the minimum point distance. The optimization method is to add a correction term to the original MPDIoU calculation formula, and the specific calculation method is as follows:

[0018]

[0019] Loss = 1 - MPDIoU rev

[0020] MPDIoU rev is the MPDIoU with a correction term added. IoU is the traditional intersection over union. distance1 is the distance between the upper left corner point of the prediction box and the upper left corner point of the ground truth box. distance2 is the distance between the lower right corner point of the prediction box and the lower right corner point of the ground truth box. distance c is the distance between the center point of the prediction box and the center point of the ground truth box. τ is the weight of the correction term. width is the width of the photo used for training. height is the height of the photo used for training;

[0021] Input the video into the improved YOLOv8 - Pose model. The model processes each frame of the video to obtain the coordinates and confidence levels of 17 key points of the prosthetic user in each frame of the photo, and the coordinates of the upper left corner point and the lower right corner point of the detection box.

[0022] Preferably, divide the forward walking part in the video. The specific division method is as follows:

[0023] (1) Calculate the starting frame frame_s1 of forward walking

[0024] box_h is a list of detection box heights. First, perform median filtering on box_h with a window size of 5, and then perform average filtering with a window size of 5 to smooth the box_h curve. The first 1 / 3 of the box_h data is intercepted to form a judgment list box_h_choose, where the start frame of forward walking, frame_s1, is in box_h_choose. The maximum value of the detection box height in the box_h_choose list is box_h_max, and the minimum value of the detection box height in the box_h_choose list is box_h_min. The calculation formula for frame_s1 is as follows:

[0025] box_h_choose frame_s1 ≥box_h_min+0.5×(box_h_max-box_h_min)

[0026] where box_h_choose frame_s1 Represents the frame_s1th element of box_h_choose. frame_s1 increases incrementally from 1 and is an integer. When the above formula is satisfied for the first time, the value of frame_s1 is determined.

[0027] (2) Calculate the end frame frame_e1 of the forward walk

[0028] shoulderw is a list of shoulder widths, len_shoulderw is the length of the shoulderw list, that is, the number of collected shoulder width data. First, perform median filtering on shoulderw with a window size of 0.05 times len_shoulderw, and then perform mean filtering with a window size of 0.05 times len_shoulderw. After filtering, the maximum shoulder width is shoulder_w_max, and the minimum shoulder width is shoulder_w_min. The calculation formula is as follows:

[0029] frame_e1>frame_s1

[0030] shoulderw frame_e1 ≥shoulder_w_min+0.83×(shoulder_w_max-shoulder_w_min)

[0031] shoulderw frame_e1 It is the frame_e1th element in shoulderw. frame_e1 increases incrementally from 1 and is an integer. The first time the above two formulas are satisfied at the same time, the value of frame_e1 is determined.

[0032] Preferably, the reverse walking part in the video is divided, and the specific division method is as follows:

[0033] (1) Calculate the start frame frame_s2 of reverse walking, and the calculation formula is as follows:

[0034] frame_s2 > frame_e1

[0035] shoulderw frame_s2 ≤ shoulder_w_min + 0.5×(shoulder_w_max - shoulder_w_min)

[0036] shoulderw frame_s2 is the frame_s2-th element in shoulderw, frame_s2 starts from 1 and increases as an integer. When the above two formulas are satisfied simultaneously for the first time, the value of frame_s2 is determined;

[0037] (2) Calculate the end frame frame_e2 of reverse walking, and the calculation formula is as follows:

[0038] frame_e2 > frame_s2

[0039] shoulderw frame_e2 ≥ shoulder_w_min + 0.83×(shoulder_w_max - shoulder_w_min)

[0040] shoulderw frame_c2 is the frame_e2-th element in shoulderw, frame_e2 starts from 1 and increases as an integer. When the above two formulas are satisfied simultaneously for the first time, the value of frame_e2 is determined;

[0041] The interval of the forward walking part is [frame_s1, frame_e1], and the interval of the reverse walking part is [frame_s2, frame_e2].

[0042] Preferably, the specific method of using the ankle joint coordinate change to divide a complete gait cycle in forward walking and a complete gait cycle in reverse walking is as follows:

[0043] left_anklex is the abscissa of the left ankle joint, right_anklex is the abscissa of the right ankle joint, and left_step is the left step length. The calculation method is as follows:

[0044] left_step = left_anklex - right_anklex

[0045] Collect the left step lengths of each frame in the forward walking section and the reverse walking section respectively to form a left step length list. Determine a complete gait cycle based on adjacent peaks in the left step length list, and divide a complete forward walking cycle into

[0046] forward_s, forward_e], and a complete reverse walking cycle into [back_s, back_e].

[0047] Preferably, the calculation of gait conventional parameters is specifically as follows:

[0048] Calculate the parameters on the right side of the prosthetic user using the key points in the interval [frame_s1, frame_e1] of the forward walking section, and calculate the parameters on the left side of the prosthetic user using the key points in the interval [frame_s2, frame_e2] of the reverse walking section; the gait conventional parameters include:

[0049] (1) The maximum value of the left hip joint angle and the maximum value of the right hip joint angle; the hip joint angle is defined as the angle between the line connecting the hip joint and the knee joint and the vertical axis, with the angle being positive during flexion and negative during extension, where flexion is the forward movement of the thigh and extension is the backward movement of the thigh;

[0050] (2) The maximum value of the left knee joint angle and the maximum value of the right knee joint angle; the knee joint angle is defined as the angle between the extension line of the line connecting the hip joint and the knee joint and the line connecting the knee joint and the ankle joint;

[0051] (3) The variance of the maximum value of the hip joint angle; collect the maximum values of multiple right hip joint angles in the forward walking section and the maximum values of multiple left hip joint angles in the reverse walking section, and calculate the variance of the collected angles;

[0052] (4) The variance of the maximum value of the knee joint angle; collect the maximum values of multiple right knee joint angles in the forward walking section and the maximum values of multiple left knee joint angles in the reverse walking section, and calculate the variance of the collected angles;

[0053] (5) The standard deviation of all left hip joint angles in the reverse walking section and the standard deviation of all right hip joint angles in the forward walking section;

[0054] (6) The standard deviation of all left knee joint angles in the reverse walking section and the standard deviation of all right knee joint angles in the forward walking section;

[0055] (7) The ratio of step length to leg length is ratio_step_leg; the step length step is defined as the maximum horizontal distance between the left and right ankle joints during walking, and the leg length leg is the sum of the thigh length and the calf length. The thigh length is defined as the length of the line connecting the hip joint and the knee joint, and the calf length is defined as the length of the line connecting the knee joint and the ankle joint. The calculation formula of ratio_step_leg is as follows:

[0056]

[0057] (8) The ratio of walking speed to leg length ratio_speed_leg; fps is the frame rate of the video, centerx is the list of the abscissas of the center points of the detection frames, dist is the forward walking distance, and the calculation formula is as follows:

[0058] dist = centerx frame_e1 -centerx frame_s1

[0059] where centerx frame_e1 is the frame_e1-th element in centerx, and centerx frame_s1 is the frame_s1-th element in centerx; the calculation formula of ratio_speed_leg is as follows:

[0060]

[0061] (9) The ratio of the left and right thigh lengths;

[0062] (10) The ratio of the left and right calf lengths.

[0063] Preferably, the step S3 specifically includes the following steps:

[0064] Step S31: Use the DTW algorithm to calculate the DTW Manhattan distance of the vertical coordinates of the left and right knee joints and the DTW Manhattan distance of the vertical coordinates of the left and right ankle joints during walking:

[0065] Define the left time series ListA and the right time series ListB. totala is the total number of elements in ListA, and totalb is the total number of elements in ListB. The calculation process of the DTW Manhattan distance between ListA and ListB is as follows:

[0066] (1) Construct the Manhattan distance matrix Manhattan_DMatrix. indexa is the order of the element in ListA, and indexb is the order of the element in ListB. The element M in the Manhattan distance matrix indexa,indexbIt is the Manhattan distance between the indexa-th element in ListA and the indexb-th element in ListB, corrected by the weight factor, and the calculation formula is as follows:

[0067]

[0068] where a indexa is the indexa-th element of ListA, and b indexb is the indexb-th element of ListB, is the weight factor of a indexa and β indexb is the weight factor of b indexb . The calculation formula of β indexb is as follows:

[0069]

[0070] indexa, indexb ∈ [frame_s1, frame_e1] ∪ [frame_s2, frame_e2]

[0071]

[0072]

[0073] (2) Construct the cumulative cost matrix Cost_Matrix. The element C indexa,indexb in the cumulative cost matrix is the minimum cumulative distance for aligning the first indexa elements of ListA with the first indexb elements of ListB. The calculation of C indexa,indexb adopts the dynamic programming method in the DTW algorithm, and a window constraint is added during the calculation. The formula of the window constraint is as follows:

[0074] -0.25 × max{totala, totalb} < indexa - indexb < 0.25 × max{totala, totalb}

[0075] where max{·} is the operation of taking the maximum value;

[0076] (3) The calculation formula of the DTW Manhattan distance dtw_manhattan between the left time series ListA and the right time series ListB is as follows:

[0077] dtw_manhattan = C totala,totalb

[0078] Calculate the DTW Manhattan distance dtw_knee of the left and right knee joint vertical coordinates using the list left_kneey of the left knee joint vertical coordinates and the list right_kneey of the right knee joint vertical coordinates; calculate the DTW Manhattan distance dtw_ankle of the left and right ankle joint vertical coordinates using the list left_ankley of the left ankle joint vertical coordinates and the list right_ankley of the right ankle joint vertical coordinates;

[0079] Step S32: Perform proportional normalization on dtw_knee and dtw_ankle;

[0080] The calculation formula for the proportionally normalized DTW Manhattan distance dtw_knee_leg of the left and right knee joint vertical coordinates is as follows:

[0081]

[0082] The calculation formula for the proportionally normalized DTW Manhattan distance dtw_ankle_leg of the left and right ankle joint vertical coordinates is as follows:

[0083]

[0084] Step S33: Calculate the perimeter and envelope area of the planar periodic cyclic curves of the left and right hip-knee based on the spatio-temporal relationship of the hip-knee joint angles; the specific calculation method is as follows:

[0085] The hip joint angle and the ipsilateral knee joint angle form a two-dimensional spatio-temporal map of the hip-knee joint angles. With the hip joint angle as the abscissa and the knee joint angle as the ordinate, the point coordinates on the two-dimensional spatio-temporal map of the hip-knee joint angles are (hipangle i , kneeangle i ), where i is the frame sequence number in a period, higangle i is the hip joint angle of the i-th frame in the period, and kneeangle i is the knee joint angle of the i-th frame in the period. Connect all the points in ascending order of i to form a planar periodic cyclic curve, and the total number of frames in a period is n;

[0086] The calculation formula for the perimeter L of the planar periodic cyclic curve is as follows:

[0087]

[0088] The calculation formula for the envelope area S of the planar periodic cyclic curve is as follows:

[0089]

[0090] where hipangle i+1is the hip joint angle at the (i + 1)-th frame in the cycle, kneeangle i+1 is the knee joint angle at the (i + 1)-th frame in the cycle, |·| represents the calculation of the matrix determinant;

[0091] (1) Calculate the perimeter L_left of the left-plane periodic cyclic curve and the enveloping area S_left of the left-plane periodic cyclic curve using the list left_hipangle of the measured left hip joint angles and the list left_kneeangle of the measured left knee joint angles in a reverse cycle [back_s, back_e];

[0092] (2) Calculate the perimeter L_right of the right-plane periodic cyclic curve and the enveloping area S_right of the right-plane periodic cyclic curve using the list right_hipangle of the measured right hip joint angles and the list right_kneeangle of the measured right knee joint angles in a forward cycle [forward_s, forward_e];

[0093] Step S34: Calculate the left hip-knee coordination-related tightness coefficient and the right hip-knee coordination-related tightness coefficient; Define the coordination-related tightness coefficient π is the ratio of a circle's circumference to its diameter;

[0094] (1) Left hip-knee coordination-related tightness coefficient

[0095] (2) Right hip-knee coordination-related tightness coefficient

[0096] Preferably, single-item scoring is performed on some of the measured parameter values with reference to the standards of normal people and based on the probability density function of the Gaussian distribution. The specific standards are as follows in the table:

[0097]

[0098]

[0099] Among them, the unit of the angle is degree, and the symbol is °. Based on the probability density function of the Gaussian distribution, a single-item score single_score calculation model is constructed. The formula is as follows:

[0100]

[0101] Among them, test is the measured value of the single-item parameter, standard is the optimal value of the single-item parameter, range is the range within which the single-item parameter is allowed to fluctuate centered on standard, and exp(·) is the exponential function with the natural constant as the base.

[0102] Preferably, the gait conventional parameters and gait coordination indexes are standardized based on the training data and input into the trained MLP classifier, and the comprehensive score is calculated using the probability that the model classifies as normal; among them,

[0103] The training process of the MLP classifier is as follows. Measure all the above-mentioned gait conventional parameters and coordination indexes for 122 videos that meet the shooting requirements in the SAIL-TUG dataset. Take all the gait conventional parameters and coordination indexes as the feature columns, and take the qualitative judgment of whether the individual gait is normal in the SAIL-TUG dataset as the target column. After dividing the above 122 videos that meet the shooting requirements into a training set and a test set according to a ratio of 7:3, perform Z-score standardization on the feature column data of the training set and save the standardizer;

[0104] Construct an MLP classifier, including the following network structure. The first hidden layer: contains 32 neurons, uses the ReLU activation function, and uses L2 regularization to suppress overfitting; the first Dropout layer: randomly masks neurons with a probability of 30% during training to enhance the robustness of the model; the second hidden layer: contains 16 neurons, still uses the ReLU activation function, and uses L2 regularization to suppress overfitting; the second Dropout layer: randomly masks neurons with a probability of 50% during training; the output layer: has 1 neuron, uses the sigmoid activation function, and outputs the classification probability; then, use the training set to train the MLP classifier, and use the test set to evaluate the performance of the MLP classifier;

[0105] After standardizing the measured values based on the training data, that is, performing standardization using the standardizer, input them into the trained MLP classifier to obtain the predicted probability probability that the classifier classifies the gait of the prosthetic user as normal. The comprehensive score final_score uses 60 points as the passing line, and the calculation formula is as follows:

[0106] final_score = 80 × probability + 20.

[0107] Compared with the prior art, the present invention has the following beneficial effects:

[0108] The present invention can complete the scoring of the gait coordination of prosthetic users only by using the videos of prosthetic users completing the TUG test. Compared with the existing medical rehabilitation means, there is no need to wear sensors, and the test is convenient.

[0109] The present invention improves the YOLOv8-Pose model, improves the CBAM attention mechanism using residual connections, and introduces the improved CBAM attention mechanism in both the Backbone layer and the Neck layer, enabling the model to better focus on features and improving the detection accuracy of the model.

[0110] The present invention uses the GSConv module to replace all convolutional modules Conv in the Neck layer, reducing the computational complexity of the model while maintaining the accuracy.

[0111] During the training of the model, the MPDIoU loss function with a correction term is used, improving the accuracy of the model's detection boxes.

[0112] The present invention not only considers the measurement of conventional gait parameters but also introduces gait coordination indexes, including the DTW Manhattan distance of the vertical coordinates of the left and right knee joints after proportional normalization, the DTW Manhattan distance of the vertical coordinates of the left and right ankle joints after proportional normalization, the tightness coefficient related to the coordination of the left hip and knee, and the tightness coefficient related to the coordination of the right hip and knee. The above parameters evaluate the coordination ability of the same joints on the left and right sides and the coordination ability between the knee joint and hip joint on the same side, better reflecting the characteristics of gait coordination.

[0113] To better interpret the relationship between the measured values and the final results, the present invention trains an MLP classifier and uses this classifier to calculate the comprehensive coordination score of the prosthetic user based on the probability of classifying an individual as normal. Description of the Drawings

[0114] Figure 1 It is a flowchart of the method of the present invention. Detailed Embodiments

[0115] The technical solutions of the present invention will be specifically described below with reference to the accompanying drawings.

[0116] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs.

[0117] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0118] As Figure 1 shown, this embodiment provides an improved YOLOv8-Pose prosthetic user gait coordination scoring method, which specifically includes the following steps:

[0119] Step S1: Obtain the video of the prosthetic user completing the TUG test. Frame by frame, use the improved YOLOv8-Pose model to perform human key point detection on the video, obtain the coordinates and confidence levels of 17 human key points throughout the process, and obtain the diagonal vertex coordinates of the detection box;

[0120] Step S2: Use the change in the height of the detection box and the change in the shoulder width of the prosthetic user to divide the walking state of the prosthetic user, dividing it into the forward walking part and the reverse walking part. Then, use the change in the ankle joint coordinates to divide a complete gait cycle during forward walking and a complete gait cycle during reverse walking, and calculate the conventional gait parameters;

[0121] Step S3: Calculate the gait coordination index of the prosthetic user using the collected key point coordinates;

[0122] Step S4: Refer to the standards of normal people for some of the conventional gait parameters and all gait coordination indexes, and perform individual scoring for each item. Standardize the measured conventional gait parameters and gait coordination indexes, input them into the trained MLP classifier, and use the classification probability to calculate the comprehensive score.

[0123] In this embodiment, Step S1 specifically includes the following steps:

[0124] Step S11: Obtain the video of the prosthetic user completing the TUG test. The TUG test is the Timed Up and Go test, and the specific requirements are as follows: The subject sits on a chair with both feet flat on the ground. The tester gives the start command, and the subject stands up from the chair, walks forward 3 meters, turns around, and returns to the chair and sits down again. The video is taken from the front right side of the subject and ensures that the subject is within the frame throughout the process. In the video, the part from the subject standing up to preparing to turn around is defined as the forward walking part, and the part from the end of turning around to preparing to sit down in front of the chair is defined as the reverse walking part. The video shooting stops when the subject finishes the action and sits down completely. Input the video into the improved YOLOv8-Pose model, and the model processes each frame of the video;

[0125] Step S12: Add a CBAM introducing residual connections after the SPPF module in the Backbone layer. Among them, the Backbone layer is the backbone network of the YOLOv8-Pose model, mainly responsible for extracting features from the input image; the SPPF module is the Spatial Pyramid Pooling Fast module, which realizes efficient calculation and multi-scale feature fusion through optimized pooling operations; CBAM is the Convolutional Block Attention Module, which learns the importance of different channels and spatial positions in the input feature map through the cascaded structure of the channel attention module and the spatial attention module, improving the feature extraction ability of the model. The main calculation method of CBAM after introducing residual connections is as follows:

[0126]

[0127] Figure is the input feature map, where Figure ∈ R size_C×size_H×size_W , where R represents the set of real numbers, size_C is the number of channels of the input feature map, size_H is the height of the input feature map, and size_W is the width of the input feature map. is an element-wise multiplication operation combined with the broadcast mechanism, M c (·) is the operation of the channel attention module, M c (Figure) is the result of performing the operation of the channel attention module on Figure, Figure cout is the figure with residual connection introduced after channel attention enhancement, M s (·) is the operation of the spatial attention module, M s (Figure cout ) is the result of performing the operation of the spatial attention module on Figure cout , Figure sout is the figure with residual connection introduced after spatial attention enhancement;

[0128] Step S13: Optimize the Neck layer. The Neck layer is the neck network of the YOLOv8-Pose model. Add CBAM with residual connection introduced after the C2f module in the P3 / 8 branch, and the input channel number of CBAM is 256 to enhance feature representation; add CBAM with residual connection introduced after the C2f module in the P4 / 16 branch, and the input channel number of CBAM is 512 to further improve the quality of features; add CBAM with residual connection introduced after the C2f module in the P5 / 32 branch, and the input channel number of CBAM is 1024 to make the model better focus on key features. Among them, the P3 / 8 branch is the high-resolution branch of the Neck layer, the P4 / 16 branch is the medium-resolution branch of the Neck layer, P5 / 32 is the low-resolution branch of the Neck layer, and the C2f module is a key feature extraction and fusion module in the YOLOv8-Pose model.

[0129] Replace all the convolutional modules Conv in the Neck layer with GSConv to keep the model with high accuracy and reduce the computational amount. Among them, GSConv is a lightweight convolution method that combines standard convolution and depthwise separable convolution;

[0130] Step S14: Use the loss function Loss optimized based on MPDIoU during the model training phase. Among them, MPDIoU is the intersection over union based on the minimum point distance, and the optimization method is to add a correction term to the original MPDIoU calculation formula. The specific calculation method is as follows:

[0131]

[0132] Loss = 1-MPDIoU rev

[0133] MPDIoU rev is the MPDIoU with the correction term added, IoU is the traditional intersection-over-union ratio, distance1 is the distance between the upper left corner of the predicted box and the upper left corner of the real box, distance2 is the distance between the lower right corner of the predicted box and the lower right corner of the real box, and distance c is the distance between the center of the predicted box and the center of the true box, τ is the weight of the correction term and τ = 0.2, width is the width of the photo used for training, and height is the height of the photo used for training;

[0134] Step S15: Obtain the coordinates and confidence scores of 17 key points of the prosthetic limb user in each frame of the photo, and the coordinates of the upper left corner and lower right corner of the detection frame;

[0135] In this embodiment, step S2 specifically includes the following steps:

[0136] Step S21: Divide the forward walking part in the video. The specific division method is as follows:

[0137] (1) Calculate the starting frame frame_s1 of forward walking

[0138] box_h is a list of detection box heights. First, perform median filtering on box_h with a window size of 5, and then perform average filtering with a window size of 5 to make the box_h curve smoother. The first 1 / 3 of the box_h data is intercepted to form a judgment list box_h_choose. Analysis of the video shows that the start frame frame_s1 of the forward walking is in box_h_choose. The maximum value of the detection box height in the box_h_choose list is box_h_max, and the minimum value of the detection box height in the box_h_choose list is box_h_min. The calculation formula for frame_s1 is as follows:

[0139] box_h_choose frame_s1 ≥box_h_min+0.5×(box_h_max-box_h_min)

[0140] where box_h_choose frame_s1 Represents the frame_s1th element of box_h_choose. frame_s1 is an integer that increases from 1. When the above formula is satisfied for the first time, the value of frame_s1 is determined.

[0141] (2) Calculate the end frame frame_e1 of the forward walk

[0142] shoulderw is a list of shoulder widths. len_shoulderw is the length of the list shoulderw, that is, the number of collected shoulder width data. First, median filtering is performed on shoulderw with a window size of 0.05 times len_shoulderw, and then mean filtering is performed with a window size of 0.05 times len_shoulderw. After filtering, the maximum shoulder width is shoulder_w_max and the minimum shoulder width is shoulder_w_min. The calculation formulas are as follows:

[0143] frame_e1 > frame_s1

[0144] shoulderw frame_e1 ≥shoulder_w_min + 0.83×(shoulder_w_max - shoulder_w_min)

[0145] shoulderw frame_e1 is the frame_e1-th element in shoulderw. frame_e1 starts from 1 and increments as an integer. When both of the above two formulas are satisfied for the first time, the value of frame_e1 is determined.

[0146] Step S22: Divide the reverse walking part in the video. The specific division method is as follows:

[0147] (1) Calculate the start frame frame_s2 of reverse walking. The calculation formula is as follows:

[0148] frame_s2 > frame_e1

[0149] shoulderw frame_s2 ≤shoulder_w_min + 0.5×(shoulder_w_max - shoulder_w_min)

[0150] shoulderw frame_s2 is the frame_s2-th element in shoulderw. frame_s2 starts from 1 and increments as an integer. When both of the above two formulas are satisfied for the first time, the value of frame_s2 is determined.

[0151] (2) Calculate the end frame frame_e2 of reverse walking. The calculation formula is as follows:

[0152] frame_e2 > frame_s2

[0153] shoulderw frame_e2≥shoulder_w_min + 0.83×(shoulder_w_max - shoulder_w_min)

[0154] shoulderw frame_e2 It is the frame_e2-th element in shoulderw. frame_e2 starts from 1 and increments as an integer. When the above two formulas are satisfied simultaneously for the first time, the value of frame_e2 is determined.

[0155] The obtained interval for the forward walking part is [frame_s1, frame_e1], and the interval for the reverse walking part is [frame_s2, frame_e2];

[0156] Step S23: Use the ankle joint coordinate changes to respectively divide a complete gait cycle in forward walking and a complete gait cycle in reverse walking. left_anklex is the abscissa of the left ankle joint, right_anklex is the abscissa of the right ankle joint, and left_step is the left step length. The calculation method is as follows:

[0157] left_step = left_anklex - right_anklex

[0158] Since the length of a gait cycle is from the heel strike of the same side to the next heel strike, the left step lengths of each frame are respectively collected in the intervals of the forward walking part and the reverse walking part to form a left step length list. A complete gait cycle is determined by the adjacent peaks in the left step length list. A complete forward walking cycle is [forward_s, forward_e], and a complete reverse walking cycle is [back_s, back_e];

[0159] Step S24: Calculate the gait conventional parameters;

[0160] Use the key points in the interval [frame_s1, frame_e1] of the forward walking part to calculate the parameters on the right side of the prosthetic user, and use the key points in the interval [frame_s2, frame_e2] of the reverse walking part to calculate the parameters on the left side of the prosthetic user. The gait conventional parameters are as follows:

[0161] (1) The maximum left hip joint angle and the maximum right hip joint angle. The hip joint angle is defined as the angle between the line connecting the hip joint and the knee joint and the vertical axis. The angle is positive during flexion and negative during extension, where flexion is the forward movement of the thigh and extension is the backward movement of the thigh.

[0162] (2) The maximum left knee joint angle and the maximum right knee joint angle. The knee joint angle is defined as the included angle between the extension line of the connection line between the hip joint and the knee joint and the connection line between the knee joint and the ankle joint.

[0163] (3) The variance of the maximum hip joint angle. Collect the maximum values of multiple right hip joint angles in the forward walking part and the maximum values of multiple left hip joint angles in the backward walking part, and calculate the variance of these angles.

[0164] (4) The variance of the maximum knee joint angle. Collect the maximum values of multiple right knee joint angles in the forward walking part and the maximum values of multiple left knee joint angles in the backward walking part, and calculate the variance of these angles.

[0165] (5) The standard deviation of all left hip joint angles in the backward walking part and the standard deviation of all right hip joint angles in the forward walking part.

[0166] (6) The standard deviation of all left knee joint angles in the backward walking part and the standard deviation of all right knee joint angles in the forward walking part.

[0167] (7) The ratio of the step length to the leg length is ratio_step_leg. The step length step is defined as the maximum value of the horizontal distance between the left and right ankle joints during walking. The leg length leg is the sum of the thigh length and the calf length. The thigh length is defined as the length of the connection line between the hip joint and the knee joint, and the calf length is defined as the length of the connection line between the knee joint and the ankle joint. The calculation formula of ratio_step_leg is as follows:

[0168]

[0169] (8) The ratio of the walking speed to the leg length ratio_speed_leg. fps is the frame rate of the video, centerx is the list of the abscissas of the center points of the detection frames, dist is the distance of forward walking, and the calculation formula is as follows:

[0170] dist = centerx frame_e1 - centerx frame_s1

[0171] where centerx frame_e1 is the frame_e1-th element in centerx, and centerx frame_s1 is the frame_s1-th element in centerx. The calculation formula of ratio_speed_leg is as follows:

[0172]

[0173] (9) The ratio of the left and right thigh lengths.

[0174] (10) Ratio of the lengths of the left and right calves.

[0175] In this embodiment, step S3 specifically includes the following steps:

[0176] Step S31: Use the DTW algorithm to calculate the DTW Manhattan distance of the vertical coordinates of the left and right knee joints and the DTW Manhattan distance of the vertical coordinates of the left and right ankle joints during walking. The DTW algorithm is a dynamic time warping algorithm that calculates the distance between two time series by finding the optimal alignment path between them. Based on the scenario of evaluating the gait coordination of prosthetic users, the following optimizations are made to the calculation process of the DTW algorithm:

[0177] Define the left time series ListA and the right time series ListB. Totala is the total number of elements in ListA, and totalb is the total number of elements in ListB. The calculation process of the DTW Manhattan distance between ListA and ListB is as follows:

[0178] (1) Construct the Manhattan distance matrix Manhattan_DMatrix. Indexa is the order of the element in ListA, and indexb is the order of the element in ListB. The element M in this matrix indexa,indexb is the Manhattan distance corrected by the weight factor between the indexa-th element in ListA and the indexb-th element in ListB. The calculation formula is as follows:

[0179]

[0180] where a indexa is the indexa-th element of ListA, b indexb is the indexb-th element of ListB, is the weight factor of a indexa β indexb is the weight factor of b indexb , β indexb The calculation formula of is as follows:

[0181]

[0182] indexa, indexb ∈ [frame_s1, frame_e1] ∪ [frame_s2, frame_e2]

[0183]

[0184] (2) Construct the cumulative cost matrix Cost_Matrix. The element C in this matrix indexa,indexbis the minimum cumulative distance that aligns the first indexa elements of ListA with the first indexb elements of ListB, C indexa,indexb The calculation of C uses the method of dynamic programming in the DTW algorithm. And to avoid excessive global search and large drifts, a window constraint is added during the calculation. The formula for the window constraint is as follows:

[0185] -0.25 × max{totala, totalb} < indexa - indexb < 0.25 × max{totala, totalb}

[0186] where max{·} is the operation of taking the maximum value.

[0187] (3) The calculation formula for the DTW Manhattan distance dtw_manhasttan between the left time series ListA and the right time series ListB is as follows:

[0188] dtw_manhattan = C totala,totalb

[0189] Use the list left_kneey of the left knee joint ordinate and the list right_kneey of the right knee joint ordinate to calculate the DTW Manhattan distance dtw_knee of the left and right knee joint ordinates using the above method. Use the list left_ankley of the left ankle joint ordinate and the list right_ankley of the right ankle joint ordinate to calculate the DTW Manhattan distance dtw_ankle of the left and right ankle joint ordinates using the above method;

[0190] Step S32: Perform ratio normalization on dtw_knee and dtw_ankle. Due to differences in individual height and leg length, dtw_knee and dte_ankle need to be ratio-normalized. The calculation formula for the ratio-normalized DTW Manhattan distance dtw_knee_leg of the left and right knee joint ordinates is as follows:

[0191]

[0192] The calculation formula for the ratio-normalized DTW Manhattan distance dtw_ankle_leg of the left and right ankle joint ordinates is as follows:

[0193]

[0194] Step S33: Based on the spatio-temporal relationship of the hip-knee joint angles, calculate the perimeter and envelope area of the planar periodic cyclic curves of the left and right hips and knees. The specific calculation method is as follows:

[0195] The hip joint angle and the ipsilateral knee joint angle form a two-dimensional hip-knee joint angle spatio-temporal diagram. With the hip joint angle as the abscissa and the knee joint angle as the ordinate, the point coordinates on the two-dimensional hip-knee joint angle spatio-temporal diagram are (hipangle i , kneeangle i ), where i is the frame sequence number in a cycle, hipangle i is the hip joint angle of the i-th frame in the cycle, kneeangle i is the knee joint angle of the i-th frame in the cycle. Connect all the points in ascending order of i to form a hip-knee plane periodic loop curve. The total number of frames in one cycle is n.

[0196] The calculation formula for the perimeter L of the plane periodic loop curve is as follows:

[0197]

[0198] The calculation formula for the envelope area S of the plane periodic loop curve is as follows:

[0199]

[0200] where hipangle i+1 is the hip joint angle of the (i + 1)-th frame in the cycle, kneeangle i+1 is the knee joint angle of the (i + 1)-th frame in the cycle, and |·| represents the calculation of the matrix determinant.

[0201] (1) Use the list left_hipangle of the left hip joint angles and the list left_kneeangle of the left knee joint angles measured in a reverse cycle [back_s, back_e] to calculate the perimeter L_left of the left plane periodic loop curve and the envelope area S_left of the left plane periodic loop curve using the above formula.

[0202] (2) Use the list right_hipangle of the right hip joint angles and the list right_kneeangle of the right knee joint angles measured in a forward cycle [forward_s, forward_e] to calculate the perimeter L_right of the right plane periodic loop curve and the envelope area S_right of the right plane periodic loop curve using the above formula;

[0203] Step S34: Calculate the left hip-knee coordination-related tightness coefficient and the right hip-knee coordination-related tightness coefficient. Define the coordination-related tightness coefficient π is the pi. The smaller the value of the coordination-related tightness coefficient θ, the better the joint coordination.

[0204] (1) The left hip-knee coordination-related tightness coefficient

[0205] (2) Right hip-knee coordination related tightness coefficient

[0206] In this embodiment, step S4 specifically includes the following steps:

[0207] Step S41: Perform single-item scoring on the partially measured parameter values with reference to the standards of normal people and based on the probability density function of the Gaussian distribution. The specific standards are as follows:

[0208] Table 1 Parameters and Their Normal Ranges

[0209]

[0210] Among them, the unit of the angle is degree, and the symbol is °. Based on the probability density function of the Gaussian distribution, a single-item score single_score calculation model is constructed. The formula is as follows:

[0211]

[0212] Where test is the measured value of the single-item parameter, standard is the optimal value of the single-item parameter, range is the range within which the single-item parameter is allowed to fluctuate centered on standard, and exp(·) is the exponential function with the natural constant as the base;

[0213] Step S42: Standardize the gait conventional parameters and gait coordination indicators based on the training data, input them into the trained MLP classifier, and calculate the comprehensive score using the probability that the model classifies as normal.

[0214] MLP is a multi-layer perceptron, which is a supervised learning model based on artificial neural networks. It mainly learns the feature representation of the input data through the activation functions and weight adjustments of multiple layers of neurons. The training process of the MLP classifier is as follows: Measure all the above-mentioned gait conventional parameters and coordination indicators for 122 videos that meet the shooting requirements in the SAIL-TUG dataset (a new medical gait dataset focusing on human abnormal gait detection). Using all the gait conventional parameters and coordination indicators as feature columns and the qualitative judgment of whether the individual gait is normal in the SAIL-TUG dataset as the target column, after dividing the 122 videos that meet the shooting requirements into a training set and a test set according to a ratio of 7:3, perform Z-score standardization on the feature column data of the training set and save the standardizer.

[0215] Build an MLP classifier with the following network structure: The first hidden layer contains 32 neurons, uses the ReLU activation function, and uses L2 regularization to suppress overfitting. The first Dropout layer randomly masks neurons with a 30% probability during training to enhance the model's robustness. The second hidden layer contains 16 neurons, still uses the ReLU activation function, and uses L2 regularization to suppress overfitting. The second Dropout layer randomly masks neurons with a 50% probability during training. The output layer has 1 neuron, uses the sigmoid activation function, and outputs the classification probability. Then, use the training set to train the MLP classifier and use the test set to evaluate the performance of the MLP classifier.

[0216] Among them, Z-score standardization converts the data into a standard normal distribution with a mean of 0 and a standard deviation of 1. ReLU is the rectified linear unit, a non-linear activation function. L2 regularization adds an L2 norm penalty term to the loss function to constrain the magnitude of the weights and improve the generalization ability. Dropout randomly masks a part of the neurons during the training process to reduce the model's dependence on specific neurons and reduce the risk of overfitting. The sigmoid activation function can map the input of the neuron to a smooth output between (0, 1).

[0217] After standardizing the measured values based on the training data using the standardizer, input them into the trained MLP classifier to obtain the prediction probability probability that the classifier classifies the gait of the prosthetic user as normal. The comprehensive score final_score uses 60 points as the passing line, and the calculation formula is as follows:

[0218] final_score = 80 × probability + 20

[0219] Specifically, the present invention can complete the gait coordination scoring of prosthetic users only by using the videos of prosthetic users completing the TUG test. Compared with the existing medical rehabilitation methods, it does not require wearing sensors and is convenient for testing. The present invention improves the YOLOv8-Pose model, improves the CBAM attention mechanism by using residual connections, and introduces the improved CBAM attention mechanism in both the Backbone layer and the Neck layer, enabling the model to better focus on features and improve the detection accuracy of the model. The present invention uses the GSConv module to replace all the convolutional modules Conv in the Neck layer, reducing the computational amount of the model while maintaining the accuracy. The MPDIoU loss function with a correction term is used in the model training stage to improve the accuracy of the model detection frame. The present invention not only considers the measurement of conventional gait parameters but also introduces gait coordination indexes, including the DTW Manhattan distance of the left and right knee joint ordinates after proportional normalization, the DTW Manhattan distance of the left and right ankle joint ordinates after proportional normalization, the left hip-knee coordination-related tightness coefficient, and the right hip-knee coordination-related tightness coefficient. The above parameters evaluate the coordination ability of the same joints on the left and right sides and the coordination ability of the knee joint and hip joint on the same side, better reflecting the characteristics of gait coordination. To better interpret the relationship between the measured values and the final results, the present invention trains an MLP classifier and uses this classifier to calculate the comprehensive coordination score of prosthetic users based on the probability of classifying an individual as normal.

[0220] The above are only the preferred embodiments of the present invention, and all equivalent changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope covered by the present invention.

Claims

1. An improved YOLOv8-Pose-based gait coordination scoring method for prosthetic users, characterized in that: It includes the following steps: Step S1: Obtain the video of the prosthetic user completing the TUG test. Use the improved YOLOv8-Pose model to detect human key points frame by frame for the video, obtain the coordinates and confidence levels of 17 human key points throughout the process, and obtain the diagonal vertex coordinates of the detection frame; Step S2: Use the change in the height of the detection frame and the change in the shoulder width of the prosthetic user to divide the walking state of the prosthetic user, divide it into the forward walking part and the reverse walking part, and then use the change in the ankle joint coordinates to divide a complete gait cycle in forward walking and a complete gait cycle in reverse walking, and calculate the conventional gait parameters; Step S3: Calculate the gait coordination index of the prosthetic user using the collected key point coordinates; Step S4: Refer to the standards of normal people for some of the conventional gait parameters and all gait coordination indexes, and conduct individual scoring for each item; standardize the measured conventional gait parameters and gait coordination indexes, input them into the trained MLP classifier, and use the classification probability to calculate the comprehensive score.

2. The improved YOLOv8-Pose gait coordination scoring method for prosthetic limb users according to claim 1, characterized in that: The video of the TUG test is taken from the front right side of the subject and ensure that the subject is within the frame throughout the TUG test. In the video, the part from the subject standing up to preparing to turn around is defined as the forward walking part, and the part from the end of turning around to returning to in front of the chair and preparing to sit down is defined as the reverse walking part. The video shooting stops when the subject completes the action and sits down completely.

3. The improved YOLOv8-Pose gait coordination scoring method for prosthetic limb users according to claim 1, characterized in that: The improved YOLOv8-Pose model is specifically as follows: Add a CBAM introducing residual connection after the SPPF module in the Backbone layer of the YOLOv8-Pose model. CBAM is the convolutional block attention mechanism. Through the cascaded structure of the channel attention module and the spatial attention module, it learns the importance of different channels and spatial positions in the input feature map to improve the feature extraction ability of the model. The main calculation method of CBAM after introducing residual connection is as follows: Figure is the input feature map, where Figure ∈ R size_C×size_H×size_W , where R represents the set of real numbers, size_C is the number of channels of the input feature map, size_H is the height of the input feature map, and size_W is the width of the input feature map. is an element-wise multiplication operation combined with the broadcasting mechanism, M c (Figure) is the result of performing the operation of the channel attention module on Figure, Figure cout is the graph with residual connection introduced after channel attention enhancement, M s (Figure cout ) is the result of performing the operation of the spatial attention module on Figure cout , Figure sout is the graph with residual connection introduced after spatial attention enhancement; Optimize the Neck layer. Add a CBAM introducing residual connection after the C2f module in the P3 / 8 branch, and the input channel number of CBAM is 256 to enhance feature representation; add a CBAM introducing residual connection after the C2f module in the P4 / 16 branch, and the input channel number of CBAM is 512 to further improve the quality of the features; add a CBAM introducing residual connection after the C2f module in the P5 / 32 branch, and the input channel number of CBAM is 1024 to enable the model to better focus on key features; Replace all the convolutional modules Conv in the Neck layer with GSConv. GSConv is a lightweight convolution method combining standard convolution and depthwise separable convolution; Use the loss function Loss optimized based on MPDIoU during the model training stage. Among them, MPDIoU is the intersection over union based on the minimum point distance. The optimization method is to add a correction term to the original MPDIoU calculation formula. The specific calculation method is as follows: Loss=1-MPDIoU rev MPDIoU rev is the MPDIoU with a correction term added. IoU is the traditional intersection over union, distance1 is the distance between the upper left corner point of the predicted bounding box and the upper left corner point of the ground truth bounding box, distance2 is the distance between the lower right corner point of the predicted bounding box and the lower right corner point of the ground truth bounding box, distance c is the distance between the center point of the predicted bounding box and the center point of the ground truth bounding box, τ is the weight of the correction term, width is the width of the photo used for training, and height is the height of the photo used for training; The video is input into the improved YOLOv8-Pose model, which processes each frame of the video and obtains the coordinates and confidence levels of 17 key points of the prosthetic user in each frame, as well as the coordinates of the upper left and lower right corners of the detection box.

4. A gait coordination scoring method for prosthetic users based on the improved YOLOv8-Pose according to claim 1, characterized in that: Divide the forward walking part in the video. The specific division method is as follows: (1) Calculate the starting frame frame_s1 of forward walking box_h is a list of detection box heights. First, perform median filtering on box_h with a window size of 5, and then perform average filtering with a window size of 5 to smooth the box_h curve. The first 1 / 3 of the box_h data is intercepted to form a judgment list box_h_choose, where the start frame of forward walking, frame_s1, is in box_h_choose. The maximum value of the detection box height in the box_h_choose list is box_h_max, and the minimum value of the detection box height in the box_h_choose list is box_h_min. The calculation formula for frame_s1 is as follows: box_h_choose frame_s1 ≥box_h_min+0.5×(box_h_max-box_h_min) where box_h_choose frame_s1 Represents the frame_s1th element of box_h_choose. frame_s1 increases incrementally from 1 and is an integer. When the above formula is satisfied for the first time, the value of frame_s1 is determined. (2) Calculate the end frame frame_e1 of the forward walk shoulderw is a list of shoulder widths, len_shoulderw is the length of the shoulderw list, that is, the number of collected shoulder width data. First, perform median filtering on shoulderw with a window size of 0.05 times len_shoulderw, and then perform mean filtering with a window size of 0.05 times len_shoulderw. After filtering, the maximum shoulder width is shoulder_w_max, and the minimum shoulder width is shoulder_w_min. The calculation formula is as follows: frame_e1>frame_s1 shoulderw frame_e1 ≥shoulder_w_min+0.83×(shoulder_w_max-shoulder_w_min) shoulderw frame_e1 It is the frame_e1-th element in shoulderw, where frame_e1 starts from 1 and increases incrementally as an integer. When the above two formulas are satisfied simultaneously for the first time, the value of frame_e1 is determined.

5. A gait coordination scoring method for prosthetic users based on the improved YOLOv8-Pose according to claim 4, characterized in that: Divide the reverse walking part in the video. The specific division method is as follows: (1) Calculate the starting frame frame_s2 of the reverse walking. The calculation formula is as follows: frame_s2>frame_e1 shoulderw frame_s2 ≤shoulder_w_min+0.5×(shoulder_w_max-shoulder_w_min) shoulderw frame_s2 is the frame_s2th element in shoulderw. frame_s2 is an integer that increases from 1. The first time the above two formulas are satisfied at the same time, the value of frame_s2 is determined. (2) Calculate the end frame frame_e2 of the reverse walking. The calculation formula is as follows: frame_e2>frame_s2 shoulderw frame_e2 ≥shoulder_w_min + 0.83×(shoulder_w_max - shoulder_w_min) shoulderw frame_e2 It is the frame_e2-th element in shoulderw. frame_e2 starts from 1 and increments as an integer. When the above two formulas are satisfied simultaneously for the first time, the value of frame_e2 is determined; The interval of the forward walking part is [frame_s1, frame_e1], and the interval of the reverse walking part is [frame_s2, frame_e2].

6. A gait coordination scoring method for prosthetic users based on the improved YOLOv8-Pose according to claim 5, characterized in that: The method of dividing a complete gait cycle in forward walking and a complete gait cycle in reverse walking by using the change of ankle joint coordinates is specifically as follows: left_anklex is the horizontal coordinate of the left ankle joint, right_anklex is the horizontal coordinate of the right ankle joint, and left_step is the left step length, which is calculated as follows: left_step=left_anklex-right_anklex Collect the left step lengths of each frame in the intervals of the forward walking part and the reverse walking part respectively to form a list of left step lengths. Determine a complete gait cycle based on adjacent peaks in the list of left step lengths, and divide a complete forward walking cycle into [forward_s, forward_e], and a complete reverse walking cycle into [back_s, back_e].

7. The improved YOLOv8-Pose-based gait coordination scoring method for prosthetic limb users according to claim 6, characterized in that: The specific calculation of the gait conventional parameters is as follows: Use the key points in the interval [frame_s1, frame_e1] of the forward walking part to calculate the parameters on the right side of the prosthetic user, and use the key points in the interval [frame_s2, frame_e2] of the reverse walking part to calculate the parameters on the left side of the prosthetic user; the gait conventional parameters include: (1) The maximum left hip joint angle and the maximum right hip joint angle; the hip joint angle is defined as the angle between the line connecting the hip joint and the knee joint and the vertical axis, with the angle being positive during flexion and negative during extension, where flexion is the forward movement of the thigh and extension is the backward movement of the thigh; (2) The maximum left knee joint angle and the maximum right knee joint angle; the knee joint angle is defined as the angle between the extension line of the line connecting the hip joint and the knee joint and the line connecting the knee joint and the ankle joint; (3) The variance of the maximum hip joint angle; collect the maximum values of multiple right hip joint angles in the forward walking part and the maximum values of multiple left hip joint angles in the reverse walking part, and calculate the variance of the collected angles; (4) The variance of the maximum knee joint angle; collect the maximum values of multiple right knee joint angles in the forward walking part and the maximum values of multiple left knee joint angles in the reverse walking part, and calculate the variance of the collected angles; (5) The standard deviation of all left hip joint angles in the reverse walking part and the standard deviation of all right hip joint angles in the forward walking part; (6) The standard deviation of all left knee joint angles in the reverse walking part and the standard deviation of all right knee joint angles in the forward walking part; (7) The ratio of the step length to the leg length is ratio_step_leg; the step length step is defined as the maximum horizontal distance between the left and right ankle joints during walking, and the leg length leg is the sum of the thigh length and the calf length. The thigh length is defined as the length of the line connecting the hip joint and the knee joint, and the calf length is defined as the length of the line connecting the knee joint and the ankle joint; the calculation formula of ratio_step_leg is as follows: (8) The ratio of the walking speed to the leg length ratio_speed_leg; fps is the frame rate of the video, centerx is the list of the abscissas of the center points of the detection frames, and dist is the forward walking distance. The calculation formula is as follows: dist=centerx frame_e1 -centerx frame_s1 where centerx frame_e1 is the frame_e1-th element in centerx, and centerx frame_s1 is the frame_s1-th element in centerx; the calculation formula of ratio_speed_leg is as follows: (9) The ratio of the left and right thigh lengths; (10) The ratio of the left and right calf lengths.

8. The improved YOLOv8-Pose gait coordination scoring method for prosthetic limb users according to claim 1, characterized in that: The specific steps of step S3 are as follows: Step S31: Use the DTW algorithm to calculate the DTW Manhattan distance of the vertical coordinates of the left and right knee joints and the DTW Manhattan distance of the vertical coordinates of the left and right ankle joints during walking: Define the time series ListA on the left and the time series ListB on the right. totala is the total number of elements in ListA, and totalb is the total number of elements in ListB. The DTW Manhattan distance calculation process between ListA and ListB is as follows: (1)Construct the Manhattan distance matrix Manhattan_DMatrix, where indexa is the order of the element in ListA, and indexb is the order of the element in ListB. The element M in the Manhattan distance matrix indexa，indexb is the Manhattan distance between the indexa-th element in ListA and the indexb-th element in ListB, corrected by the weight factor. The calculation formula is as follows: where a indexa is the indexa-th element of ListA, and b indexb is the indexb-th element of ListB, is the weight factor of a indexa , and β indexb is the weight factor of b indexb . The calculation formula of β indexb is as follows: indexa, indexb∈[frame_s1, frame_e1]∪[frame_s2, frame_e2] (2) Construct the cumulative cost matrix Cost_Matrix, where the element C in the cumulative cost matrix indexa，indexb is the minimum cumulative distance obtained by aligning the first indexa elements of ListA with the first indexb elements of ListB. The calculation of C indexa，indexb adopts the method of dynamic programming in the DTW algorithm, and a window constraint is added during the calculation process. The formula for the window constraint is as follows: -0.25×max{total a, total b} <indexa-indexb<0.25×max{totala,totalb} Among them, max{·} is the operation of taking the maximum value; (3) The calculation formula for the DTW Manhattan distance dtw_manhattan between the left time series ListA and the right time series ListB is as follows: dtw_manhattan=C totala，totalb Use the left_kneey list of the left knee joint vertical coordinates and the rightkneey list of the right knee joint vertical coordinates to calculate the DTW Manhattan distance dtw_knee of the left and right knee joint vertical coordinates; use the left_ankley list of the left ankle joint vertical coordinates and the right_ankley list of the right ankle joint vertical coordinates to calculate the DTW Manhattan distance dtw_ankle of the left and right ankle joint vertical coordinates; Step S32: normalize the proportions of dtw_knee and dtw_ankle; The calculation formula of the proportionally normalized DTW Manhattan distance dtw_knee_leg of the left and right knee joint vertical coordinates is as follows: The calculation formula of the proportionally normalized DTW Manhattan distance dtw_ankle_leg of the left and right ankle joint vertical coordinates is as follows: Step S33: Based on the spatiotemporal relationship of the hip and knee joint angles, the perimeter and envelope area of the planar periodic cyclic curves of the left and right hip and knee joints are calculated. The specific calculation method is as follows: The hip joint angle and the ipsilateral knee joint angle form a two-dimensional hip-knee joint angle spatio-temporal diagram. With the hip joint angle as the abscissa and the knee joint angle as the ordinate, the point coordinates on the two-dimensional hip-knee joint angle spatio-temporal diagram are (hipangle i , kneeangle i ), where i is the frame sequence number in a cycle, hipangle i is the hip joint angle of the i-th frame in the cycle, and kneeangle i is the knee joint angle of the i-th frame in the cycle. Connect all the points in ascending order of i to form a cyclic curve of the hip-knee plane. The total number of frames in one cycle is n; The calculation formula for the perimeter L of a plane periodic cycle curve is as follows: The calculation formula of the envelope area S of the plane periodic cycle curve is as follows: where hipangle i+1 is the hip joint angle at the (i + 1)-th frame in the cycle, and kneeangle i+1 is the knee joint angle at the (i + 1)-th frame in the cycle; |·| represents the calculation of the matrix determinant; (1) Calculate the perimeter L_left of the left plane cycle curve and the envelope area S_left of the left plane cycle curve using the list of left hip joint angles left_hipangle and the list of left knee joint angles left_kneeangle measured in a reverse cycle [back_s, back_e]; (2) Using the list of right hip joint angles right_hipangle and the list of right knee joint angles right_kneeangle measured in a forward cycle [forward_s, forward_e], calculate the perimeter L_right of the right plane cycle curve and the envelope area S_right of the right plane cycle curve; Step S34: Calculate the left hip-knee coordination-related tightness coefficient and the right hip-knee coordination-related tightness coefficient; define the coordination-related tightness coefficient π is the ratio of a circle's circumference to its diameter; (1) Left hip-knee coordination correlation coefficient (2) Right hip-knee coordination related tightness coefficient 9. A gait coordination scoring method for prosthetic users based on the improved YOLOv8-Pose according to claim 1, characterized in that: Some measured parameter values are scored individually based on the normal standard and the probability density function of the Gaussian distribution. The specific standards are as follows: The unit of angle is degree, the symbol is °, and based on the probability density function of Gaussian distribution, the single score calculation model is constructed as follows: Where test is the measured value of a single parameter, standard is the optimal value of the single parameter, range is the range within which the single parameter is allowed to fluctuate centered around standard, and exp(·) is the exponential function with the natural constant as the base.

10. The improved YOLOv8-Pose based gait coordination scoring method for prosthetic limb users according to claim 9, characterized in that: Standardize the gait conventional parameters and gait coordination indexes based on the training data, input them into the trained MLP classifier, and calculate the comprehensive score using the probability of the model classifying as normal; among them, The training process of the MLP classifier is as follows. Measure all the above-mentioned gait conventional parameters and coordination indexes for 122 videos that meet the shooting requirements in the SAIL-TUG dataset. Use all the gait conventional parameters and coordination indexes as the feature columns, and the qualitative judgment of whether an individual's gait is normal in the SAIL-TUG dataset as the target column. After dividing the above 122 videos that meet the shooting requirements into a training set and a test set according to a ratio of 7:3, perform Z-score standardization on the feature column data of the training set and save the standardizer; Construct an MLP classifier, including the following network structure. The first hidden layer: contains 32 neurons, uses the ReLU activation function, and uses L2 regularization to suppress overfitting; the first Dropout layer: randomly masks neurons with a probability of 30% during training to enhance the robustness of the model; the second hidden layer: contains 16 neurons, still uses the ReLU activation function, and uses L2 regularization to suppress overfitting; the second Dropout layer: randomly masks neurons with a probability of 50% during training; the output layer: has 1 neuron, uses the sigmoid activation function to output the classification probability; then, use the training set to train the MLP classifier and use the test set to evaluate the performance of the MLP classifier; After standardizing the measured value based on the training data, that is, performing standardization using the standardizer, input it into the trained MLP classifier to obtain the predicted probability probability that the classifier classifies the gait of the prosthetic user as normal. The comprehensive score final_score uses 60 points as the passing line, and the calculation formula is as follows: final_score = 80×probability + 20.

Images