Training method and device of predictive and interpretive model and computer equipment

Abstract

The invention relates to a prediction and interpretation model training method and device and computer equipment. The method comprises the following steps: acquiring sample data of a plurality of users; ranking the feature importance of the plurality of detail action index parameters indicating the plurality of detail actions based on a feature screening algorithm, and screening out a plurality of important index parameters according to a ranking result; inputting the plurality of important index parameters into the trained prediction sub-model to output a prediction result indicating the health degree of the joint, and training by using the plurality of important index parameters obtained from the sample data of part of users, a plurality of important index parameters obtained from the sample data of the remaining users are used for testing to obtain a trained prediction sub-model; inputting the prediction result into an interpretation sub-model to interpret the prediction result so as to determine the contribution degree of the plurality of important index parameters to the prediction result; and obtaining a trained prediction and interpretation model based on the trained prediction sub-model and interpretation sub-model.

Description

Technical Field

[0001] The present disclosure relates to the field of machine learning prediction, and in particular, to training methods, apparatuses and computer devices for predictive and explanatory models for assessing the health of joints. Background Art

[0002] Traditionally, to evaluate the health of joint organs such as the knee and hip joints, the person being evaluated needs to complete the WOMAC scoring scale. If the total score of the scale is high, it means that the overall sub-health of the knee and hip joints is more likely. Then the person being evaluated needs to do several sets of knee and hip joint-related movements to evaluate whether some more detailed indicators are within the healthy range.

[0003] However, based on the performance of the experimental samples, the researchers found that the value ranges of various detailed movement indicators for people with sub-healthy and healthy knee and hip joints were highly overlapping, making it difficult to distinguish them by defining thresholds.

[0004] As one of the important branches of artificial intelligence, machine learning is widely used in data mining, image recognition, natural language processing and other fields. However, current machine learning algorithms cannot effectively assess the health of joint organs. Summary of the invention

[0005] In order to better evaluate the health of joints, the present disclosure provides a training method for a predictive and explanatory model for evaluating the health of joints. The trained predictive and explanatory model is used to perform an automated algorithm evaluation of the health of the joints, and an automatic explanation is given to the results predicted by the model.

[0006] According to one aspect of the present disclosure, a method for training a predictive and explanatory model for evaluating the health of joints is provided, the method comprising: obtaining sample data of multiple users, wherein the sample data of each user comprises multiple action video data of multiple groups of actions performed by each user on the joints, each action of the multiple groups of actions comprising multiple detail actions; ranking the feature importance of multiple detail action index parameters indicating the multiple detail actions based on a feature screening algorithm, and screening out a certain number of detail action index parameters from the multiple detail action index parameters as multiple important index parameters according to the ranking results; inputting the multiple important index parameters into a trained prediction sub-model to output an indication of the The method comprises the following steps: obtaining a probability-based prediction result of the health status of each user's joints, wherein a prediction sub-model is trained using a plurality of important indicator parameters obtained from sample data of some users among the plurality of users as training data, and testing the prediction sub-model using a plurality of important indicator parameters obtained from sample data of the remaining users among the plurality of users as verification data to obtain the trained prediction sub-model; inputting the prediction result into an explanation sub-model to perform feature contribution interpretation on the prediction result to determine the contribution of the plurality of important indicator parameters to the prediction result; and obtaining a trained prediction and explanatory model based on the trained prediction sub-model and the explanation sub-model.

[0007] According to an embodiment of the present disclosure, the method may further include performing action detection on each of the multiple action video data through a key point detection algorithm to obtain the multiple quantized detail action indicator parameters indicating different detail actions in the multiple groups of actions.

[0008] According to an embodiment of the present disclosure, the method may further include determining abnormal values ​​of the plurality of detail action index parameters, and removing the detail action index parameters determined to be abnormal values.

[0009] According to an embodiment of the present disclosure, outlier determination can be performed on the multiple detail action indicator parameters based on an IQR statistical measurement method, the IQR statistical measurement method comprising: calculating a first value of the 25th percentile minus 1.5IQR and a second value of the 75th percentile plus 1.5IQR, and determining values ​​less than the first value and values ​​greater than the second value as the outliers, wherein IQR refers to a value obtained by subtracting the 25th percentile value from the 75th percentile value.

[0010] According to an embodiment of the present disclosure, the feature screening algorithm may include a Boruta algorithm, and screening out the multiple important indicator parameters includes: constructing multiple shadow features corresponding to the multiple detail action indicator parameters; ranking the feature importance of the multiple detail action indicator parameters based on the feature importance of the multiple shadow features; and after multiple iterations, screening out the certain number of detail action indicator parameters from the multiple detail action indicator parameters as the multiple important indicator parameters according to the ranking results.

[0011] According to an embodiment of the present disclosure, the prediction sub-model may include a LightGBM prediction model, and the training of the LightGBM prediction model includes: constructing multiple value intervals for multiple sets of hyperparameters of the LightGBM prediction model to run a grid search strategy, and performing k-fold cross-validation on the LightGBM prediction model to determine the optimal hyperparameter combination of the LightGBM prediction model.

[0012] According to an embodiment of the present disclosure, the k-fold cross validation may include 5-fold cross validation and 10-fold cross validation.

[0013] According to an embodiment of the present disclosure, interpreting the feature contribution of a prediction result may include: calculating the feature contribution of each important indicator parameter among multiple important indicator parameters based on the SHAP algorithm; and interpreting the contribution of each important indicator parameter to the prediction result based on the feature contribution of each important indicator parameter.

[0014] According to an embodiment of the present disclosure, interpreting the feature contribution of the prediction result may further include: when the value of the feature contribution of an important indicator parameter is greater than 0, it indicates that the important indicator parameter has a positive impact on the prediction result; and when the value of the feature contribution of an important indicator parameter is less than 0, it indicates that the important indicator parameter has a negative impact on the prediction result, wherein the larger the absolute value of the feature contribution of the important indicator parameter is, the greater the impact of the important indicator parameter on the prediction result is.

[0015] According to an embodiment of the present disclosure, the joint may include at least one or more of a shoulder joint, an elbow joint, a hip joint, a wrist joint, a knee joint and an ankle joint.

[0016] According to an embodiment of the present disclosure, the multiple sets of actions may at least include: squatting, knee flexion angle in left and right directions, hip flexion and extension angle in left and right directions, raising the knee and patting the knee with left and right hands alternately, and standing on one leg.

[0017] According to an embodiment of the present disclosure, multiple detailed action indicator parameters may include at least: the average duration of standing and squatting, the average minimum angle that the knee joint can reach when squatting, the minimum angle that the knee joint can reach when flexing the right knee, the minimum angle that the knee joint can reach when flexing the left knee, the maximum angle that the hip joint can reach when flexing the right hip, the maximum angle that the hip joint can reach when flexing the left hip, the maximum angle that the hip joint can reach when extending the right hip, the maximum angle that the hip joint can reach when extending the left hip, the maximum angle that the right leg can reach with the vertical direction when the knee is raised and the left and right hands pat the knees alternately, the maximum angle that the left leg can reach with the vertical direction when the knee is raised and the left and right hands pat the knees alternately, the maximum angle that the left leg can reach with the vertical direction when standing on the right foot, the maximum angle that the right leg can reach with the vertical direction when standing on the left foot, the maintenance time when the right foot is standing on one foot, and the maintenance time when the left foot is standing on one foot.

[0018] According to an embodiment of the present disclosure, multiple important indicator parameters may include at least: the average duration of standing and squatting, the average minimum angle that the knee joint can reach when squatting, the minimum angle that the knee joint can reach when bending the right knee, the minimum angle that the knee joint can reach when bending the left knee, the maximum angle that the right leg can reach with the vertical direction when the knee is raised and the left and right hands pat the knees alternately, the maximum angle that the left leg can reach with the vertical direction when the knee is raised and the left and right hands pat the knees alternately, the duration of standing on one foot with the right foot, and the duration of standing on one foot with the left foot.

[0019] According to another aspect of the present disclosure, a computer-implemented method for assessing the health of joints is provided, the method may include: acquiring multiple action video data of multiple groups of actions performed by a user on the joints; inputting the multiple action video data into a predictive and explanatory model; predicting the health of the user's joints based on the predictive and explanatory model to output a probability-based prediction result indicating the health of the user's joints; and performing feature contribution interpretation on the prediction result to output a feature contribution ranking of detailed action indicator parameters corresponding to each action in the multiple groups of actions, wherein the predictive and explanatory model is obtained based on the above-mentioned training method of the predictive and explanatory model for assessing the health of joints.

[0020] According to an embodiment of the present disclosure, the method may further include: determining the impact of each action in the multiple groups of actions on the health of the user's joints based on the prediction results and the feature contribution rankings.

[0021] According to an embodiment of the present disclosure, the method may further include: providing customized exercise or training suggestions to the user based on the prediction results and the feature contribution rankings.

[0022] According to another aspect of the present disclosure, there is provided a training device for a predictive and explanatory model for evaluating the health of joints, the device comprising: a memory storing computer executable instructions; and a processor coupled to the memory, the processor being configured to: obtain sample data of multiple users, wherein the sample data of each user comprises multiple action video data of multiple groups of actions performed by each user on a joint, each action of the multiple groups of actions comprising multiple detail actions; rank the feature importance of multiple detail action indicator parameters indicating the multiple detail actions based on a feature screening algorithm, and screen out a certain number of detail action indicator parameters from the multiple detail action indicator parameters as multiple important indicator parameters according to the ranking result; and select the multiple important indicator parameters as the feature importance of the multiple detail action indicator parameters. The important indicator parameters are input into a trained prediction sub-model to output a probability-based prediction result indicating the health of the joints of each user, wherein the prediction sub-model is trained using a plurality of important indicator parameters obtained from sample data of some users among the multiple users as training data, and the prediction sub-model is tested using a plurality of important indicator parameters obtained from sample data of the remaining users among the multiple users as verification data to obtain the trained prediction sub-model; the prediction result is input into an explanation sub-model to perform feature contribution interpretation on the prediction result to determine the contribution of the plurality of important indicator parameters to the prediction result; and a trained prediction and explanatory model is obtained based on the trained prediction sub-model and the explanation sub-model.

[0023] According to another aspect of the present disclosure, a device for evaluating the health of joints is provided, the device comprising: an acquisition unit, configured to acquire multiple action video data of multiple groups of actions performed by a user on the joints; an input unit, configured to input the multiple action video data into a prediction and explanatory model, wherein the prediction and explanatory model is obtained based on the above-mentioned training method of the prediction and explanatory model for evaluating the health of joints; a processing unit, configured to predict the health of the user's joints based on the prediction and explanatory model to output a probability-based prediction result indicating the health of the user's joints, the processing unit being further configured to perform feature contribution interpretation on the prediction result to output a feature contribution ranking of a detail action indicator parameter corresponding to each action in the multiple groups of actions; and a display unit, configured to display the prediction result and the feature contribution ranking.

[0024] According to another aspect of the present disclosure, a computer program product is provided, on which computer readable instructions are stored, and when the computer readable instructions are executed by a processor, they can implement the above-mentioned training method of the predictive and explanatory model for assessing the health of joints and the above-mentioned computer-implemented method for assessing the health of joints.

[0025] Therefore, according to the training method, device and computer equipment of the predictive and explanatory model for evaluating the health of joints in accordance with the embodiments of the present disclosure, feature screening and explanatory machine learning techniques are used to realize automatic evaluation of the health of joints, thereby solving the problem of high overlap of detailed movement indicator parameters of joint health and sub-health groups that could not be solved by the original threshold method. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some exemplary embodiments of the present disclosure, and a person of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

[0027] Figure 1 A flowchart of a method for training a predictive and explanatory model for assessing the health of a joint according to an embodiment of the present disclosure is shown;

[0028] Figure 2 A schematic diagram showing multiple groups of video actions on knee and hip joints according to an embodiment of the present disclosure;

[0029] Figure 3 shows a feature distribution diagram based on the Boruta algorithm according to an embodiment of the present disclosure;

[0030] Figure 4 A schematic diagram of a LightGBM prediction model according to an embodiment of the present disclosure is shown;

[0031] Figure 5 A schematic diagram showing a prediction result indicating the health of a user's joints and a characteristic contribution of each important indicator parameter according to an embodiment of the present disclosure;

[0032] Figure 6 A flow chart showing a method for assessing the health of a joint according to an embodiment of the present disclosure;

[0033] Figure 7 A block diagram showing a training device for a predictive and explanatory model for assessing joint health according to an embodiment of the present disclosure; and

[0034] Figure 8A schematic diagram of a device for assessing the health of a joint according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0036] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits the detailed description of some known functions and known components.

[0037] Flowcharts are used in the present disclosure to illustrate the steps of the method according to the embodiments of the present disclosure. It should be understood that the preceding or following steps are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more steps may be removed from these processes.

[0038] In the specification and drawings of the present disclosure, elements are described in singular or plural form, depending on the embodiment. However, the singular and plural forms are appropriately selected for the proposed situation only for the convenience of explanation and are not intended to limit the present disclosure thereto. Therefore, the singular form may include the plural form, and the plural form may also include the singular form, unless the context clearly indicates otherwise.

[0039] The training method, apparatus and computer device of the predictive and explanatory model for assessing the health of joints provided by the present disclosure will be described in detail below with reference to the accompanying drawings.

[0040] Figure 1A flowchart S100 of a method for training a prediction and explanatory model for evaluating the health of a joint according to an embodiment of the present disclosure is shown. Figure 1 The various steps of the training method of the predictive and explanatory model for assessing the health of joints according to an embodiment of the present disclosure are described in detail.

[0041] First, if Figure 1 As shown, the flowchart S100 of the training method of the predictive and explanatory model for evaluating the health of joints according to the present disclosure may include the various steps shown according to processes S1020, S1040, S1060, S1080 and S1100.

[0042] In process S1020, sample data of multiple users may be acquired, wherein the sample data of each user includes multiple action video data of multiple groups of actions performed by each user on a joint, and each action of the multiple groups of actions includes multiple detailed actions.

[0043] According to an embodiment of the present disclosure, the joint may include at least one or more of a shoulder joint, an elbow joint, a hip joint, a wrist joint, a knee joint, and an ankle joint. In an embodiment of the present disclosure, the knee and hip joints will be used as examples for explanation.

[0044] For example, users can perform multiple sets of actions to test the health of the knee and hip joints, such as squatting, sitting, hip flexion and extension, etc. Taking squatting as an example, users need to complete a series of video actions when performing squatting, including three detailed actions of standing-squatting-standing. Taking sitting as an example, users also need to complete a series of video actions when performing sitting, including three detailed actions of sitting-standing-sitting.

[0045] According to an embodiment of the present disclosure, action detection may be performed on each of a plurality of action video data using a key point detection algorithm to obtain a plurality of quantized detail action index parameters indicating different detail actions in a plurality of action groups.

[0046] Key point detection technology is one of the basic research directions in computer vision. Its main goal is to identify the key points of objects in images or videos, such as the joint positions of the human body, facial feature points, etc. Key point detection technology plays an important role in many applications, such as human posture estimation, facial feature detection, hand movement recognition, etc. In human posture estimation, key points usually refer to the joint positions of the human body, such as shoulders, elbows, knees, hips, etc.

[0047] In the embodiment of the present disclosure, key point detection technology is used to Figure 2The five groups of knee and hip joint video movements shown in are used for key point detection, and are quantified into 14 detail movement index parameters. It should be noted that in this embodiment, 14 detail movements are taken as an example for explanation, but in the implementation process, the number of detail movements is not limited to 14, and more than 14 or less than 14 detail movements can be used.

[0048] like Figure 2 As shown, Figure 2 (a)-(e) in the figure show schematic diagrams of five sets of video movements of the knee and hip joints completed by the user.

[0049] Specifically, Figure 2 (a) in the figure is a schematic diagram of a video action of a user completing a squat (stand-squat-stand) action. For example, through this video action, the average duration of the user's stand-squat-stand cycle within 20 seconds and the average minimum angle that the knee joint can reach when squatting when completing multiple consecutive actions within 20 seconds can be tested. The shorter the squatting time, the smaller the knee joint angle when squatting, indicating that the knee joint function is relatively better.

[0050] Figure 2 (b) in the figure is a schematic diagram of a video action of a user completing a knee flexion action in the left and right directions. For example, by completing the video action once in the left and right directions respectively, the minimum angle that the knee joint can reach when the user completes the right knee flexion action and the minimum angle that the knee joint can reach when the user completes the left knee flexion action can be tested. The smaller the angle, the better the function of the knee joint. In addition, the user can also try the action multiple times to test the angle of the calf change, and take the best performance as the test result.

[0051] Figure 2 (c) in the figure is a schematic diagram of a video action of a user completing a hip flexion and extension angle action in the left and right directions. For example, by completing the video action once in the left and right directions respectively, the maximum angle that the hip joint can reach when the user completes the right hip flexion action, the maximum angle that the hip joint can reach when the user completes the left hip flexion action, the maximum angle that the hip joint can reach when the user completes the right hip extension action, and the maximum angle that the hip joint can reach when the user completes the left hip extension action. The larger the angle, the better the function of the hip joint.

[0052] Figure 2 (d) in the figure is a schematic diagram of a video action of a user completing the knee-raising and left-right hand patting knee action. For example, by trying to complete the knee-raising and left-right hand patting knee action multiple times and taking the best performance, the maximum angle that the user's right leg can reach with the vertical direction when the knee is raised and the left and right hands are patting the knee alternately, and the maximum angle that the user's left leg can reach with the vertical direction when the knee is raised and the left and right hands are patting the knee alternately can be tested. The larger the angle, the better the hip joint flexibility function.

[0053] In addition, the user can also test the number of times the user lifts the left and right legs according to the above-mentioned video action of raising the knee and patting the knee with the left and right hands alternately. The more times the user lifts the legs, the better the hip joint flexibility function is.

[0054] Figure 2 (e) in the figure is a schematic diagram of the video action of the user completing the single-leg standing action. For example, by trying to complete the video action of standing on one leg and lifting the contralateral leg for many times and taking the longest section of the action, the maximum angle that the left leg can reach with the vertical direction when the user stands on the right foot, the maximum angle that the right leg can reach with the vertical direction when the left foot stands on the left foot, the maintenance time when the right foot stands on the right foot, and the maintenance time when the left foot stands on the left foot can be tested. The larger the angle of the contralateral leg when standing on one leg, the better the flexibility function of the hip joint. The longer the maintenance time when standing on one leg, the better the stability function of the lower limbs.

[0055] It should be noted that the above five groups of video movements about the knee and hip joints and the detailed movement index parameters corresponding to the video movements are only examples. The technology disclosed in the present invention is not limited to the above video movements and detailed movement index parameters, but can adopt any other video movements and corresponding detailed movement index parameters that can be used to evaluate the health of the knee and hip joints.

[0056] According to an embodiment of the present disclosure, outlier determination can also be performed on the above-mentioned multiple detail action indicator parameters, and the detail action indicator parameters determined as outliers can be removed. For example, outliers in multiple detail action indicator parameters can be determined based on an IQR (interquartile range) statistical measurement method.

[0057] IQR is a commonly used outlier detection method. Specifically, IQR is defined as Q3-Q1, where Q1 is the lower quartile (i.e., the 25th percentile value) and Q3 is the upper quartile (i.e., the 75th percentile value). If the value of a data point is less than (Q1-1.5×IQR) or greater than (Q3+1.5×IQR), it is determined to be an outlier. In addition, if the value of a data point is less than 0, it is replaced by a value of 0, thereby completing the filtering of data noise.

[0058] In process S1040, the feature importance of multiple detail action index parameters indicating multiple detail actions can be ranked based on a feature screening algorithm, and a certain number of detail action index parameters can be screened out from the multiple detail action index parameters as multiple important index parameters according to the ranking results.

[0059] According to an embodiment of the present disclosure, the feature screening algorithm may include a Boruta algorithm, and screening out multiple important indicator parameters includes: constructing multiple shadow features corresponding to multiple detail action indicator parameters; ranking the feature importance of multiple detail action indicator parameters based on the feature importance of the multiple shadow features; and after multiple iterations, screening out a certain number of detail action indicator parameters from the multiple detail action indicator parameters as multiple important indicator parameters according to the ranking results.

[0060] In one example, multiple detail action indicator parameters are used as original features, the original features are randomized to construct shadow features corresponding to the original features, and then the shadow features and the original features are merged to construct a new extended information data set. A random forest is established using the new data set, and the importance of each feature (including original features and shadow features) is determined by the random forest. For each original feature, its importance is compared with the corresponding shadow feature, where the original feature with a significantly higher importance than the shadow feature is defined as important, and the original feature with a significantly lower importance than the shadow feature is defined as unimportant. Then, the unimportant original features and all shadow features are deleted, and a new round of iterations is carried out again: construct shadow features-combine extended data sets-random forest modeling-select important features, until all original features are assigned importance, or the preset number of iterations is reached.

[0061] Figure 3 A feature distribution diagram based on the Boruta algorithm of an embodiment of the present disclosure is shown.

[0062] The core of the Boruta algorithm is based on two ideas: Shadow Features and Binomial Distribution. In practice, assuming that the original feature is X, each feature in X is randomly reordered (shuffle) based on the Boruta algorithm. These randomly reordered features are called shadow features. Then, the shadow data frame is appended to the original data frame to obtain a new data frame (for example, X_boruta), which has twice the number of feature columns as the original X. Then a model is used to train it to obtain the feature importance of each feature.

[0063] If a feature is useful, then its feature importance must be better than its random version. At this time, we only need to retain the original features whose feature importance is higher than the highest feature importance score in the random version. But there may be a problem. The model may not be trained very well, and the feature importance is not very accurate, so the second strategy is needed, namely the binomial distribution.

[0064] The core of binomial distribution is continuous iteration. If a result is lucky, then iterate multiple times, such as 20 times or 100 times, and then take the average of the feature importance and compare and screen. In the Boruta algorithm, the maximum uncertainty level of the feature is expressed as a 50% probability. Since each independent experiment can give a binary result (hit or miss), a series of n trials follow the binomial distribution. The probability mass function based on the binomial distribution can be obtained as follows Figure 3 The characteristic distribution diagram is shown.

[0065] exist Figure 3 In the feature distribution diagram shown: the left area is the rejection area, the features in this area are considered to be noise, so they can be discarded directly; the middle area is the hesitation area, the Boruta algorithm has difficulty in making a decision on the features in this area; and the right area is the acceptable area, the features in this area are considered to be predictive and can generally be retained.

[0066] In one example, samples of 200 users were collected according to the user collection specification. The 14 detail action indicator parameters of the 200 samples were ranked in importance based on the Boruta algorithm, and the number of iterations of the Boruta algorithm was set to 100 times, thereby generating a binomial distribution of whether each indicator parameter met the standard. Thus, the indicator parameters located in the rejection area of ​​the data distribution were excluded through the statistical T-Test, and the top 8 indicators were selected as important indicator parameters.

[0067] In an example, the eight important indicator parameters may include: the average duration of standing and squatting, the average minimum angle that the knee joint can reach when squatting, the minimum angle that the knee joint can reach when bending the right knee, the minimum angle that the knee joint can reach when bending the left knee, the maximum angle that the right leg can reach with the vertical direction when raising the knee and patting the knees alternately with the left and right hands, the maximum angle that the left leg can reach with the vertical direction when raising the knee and patting the knees alternately with the left and right hands, the duration of standing on one foot with the right foot, and the duration of standing on one foot with the left foot.

[0068] Based on the 8 important indicator parameters selected above, it can be determined that the corresponding video actions about the knee and hip joints are: squatting, knee flexion in the left and right directions, raising the knee and patting the knee alternately with the left and right hands, and standing on one leg.

[0069] By screening out multiple important indicator parameters from multiple detailed action indicator parameters based on a feature screening algorithm, the actions that users need to complete when assessing the health of their knee and hip joints can be reduced, further improving user convenience while ensuring the accuracy of the assessment.

[0070] In process S1060, multiple important indicator parameters can be input into a trained prediction sub-model to output a probability-based prediction result indicating the health of each user's joints, wherein the prediction sub-model is trained using multiple important indicator parameters obtained from sample data of some of the multiple users as training data, and the prediction sub-model is tested using multiple important indicator parameters obtained from sample data of the remaining users among the multiple users as verification data to obtain the trained prediction sub-model.

[0071] According to an embodiment of the present disclosure, the prediction sub-model may include a LightGBM prediction model, and the training of the LightGBM prediction model includes: constructing multiple value intervals for multiple sets of hyperparameters of the LightGBM prediction model to run a grid search strategy, and performing k-fold cross-validation on the LightGBM prediction model to determine the optimal hyperparameter combination of the LightGBM prediction model.

[0072] According to an embodiment of the present disclosure, the k-fold cross validation may include 5-fold cross validation and 10-fold cross validation.

[0073] Figure 4 A schematic diagram of a LightGBM prediction model according to an embodiment of the present disclosure is shown.

[0074] like Figure 4 As shown in the figure, the LightGBM prediction model adopts an integrated machine learning method based on decision trees, and uses gradient boosting technology to combine multiple weak learners (usually decision trees) into a powerful model. The principles of the LightGBM prediction model are as follows: Gradient boosting, the LightGBM prediction model uses gradient boosting technology to continuously fit the residual of the model (that is, the difference between the actual value and the predicted value) in an iterative manner, thereby gradually improving the prediction ability of the model; Based on the decision tree model, the LightGBM prediction model uses a decision tree-based model as a weak learner, where each tree consists of multiple nodes, each node contains a feature and a threshold, and by comparing the feature value of the sample with the size of the threshold, the sample is assigned to the left subtree or the right subtree, thereby achieving classification or regression of the sample; Efficient feature splitting strategy, the LightGBM prediction model adopts a feature splitting strategy called "Leaf-wise" (leaf node splitting), which is different from the traditional "Level-wise" (level splitting). It can select the features and thresholds that contribute the most to the current sample set at each split, thereby reducing the depth of the tree and improving the training speed.

[0075] In practice, the original data set is first preprocessed and the data set is divided into a training set and a test set for model training and evaluation. In the model training phase, LightGBM is used on the training set for model training. The parameters of the model need to be specified, such as the learning rate, the number of trees, the depth of the tree, the feature splitting strategy, etc. Parameters can be tuned by cross-validation and other methods to obtain better model performance. In the model evaluation phase, the trained model is evaluated on the test set, and common evaluation indicators such as accuracy, precision, recall, F1-score, etc. can be used to evaluate the performance of the model. In the model prediction phase, the trained model is used to predict new data to obtain the output of the model.

[0076] In one example, a LightGBM prediction model is constructed for the 8 important indicator parameters selected above. For example, for the sample data of 200 users collected, the sample data of 75% of the users (ie, 150 users) are used as training data to determine the optimal model hyperparameter structure. Six value intervals are constructed for each of the 7 sets of hyperparameters of the model to run the grid search strategy, and the model is cross-validated 5-fold to determine the final optimal hyperparameter combination of the model. Then, the sample data of the remaining 25% of the users (ie, 50 users) are used as verification data for testing, and the predicted probability of sub-health level of these 50 users is output.

[0077] By constructing the optimal prediction sub-model based on multiple important indicator parameters that have been screened out, it is possible to better assess whether the user's knee and hip joints are sub-healthy or healthy, so as to more accurately predict the health of the user's knee and hip joints.

[0078] In process S1080, the prediction results may be input into the explanation sub-model to perform feature contribution interpretation on the prediction results to determine the contribution of multiple important indicator parameters to the prediction results.

[0079] According to an embodiment of the present disclosure, interpreting the feature contribution of a prediction result may include: calculating the feature contribution of each important indicator parameter among multiple important indicator parameters based on the SHAP algorithm; and interpreting the contribution of each important indicator parameter to the prediction result based on the feature contribution of each important indicator parameter.

[0080] According to an embodiment of the present disclosure, interpreting the feature contribution of the prediction result may also include: when the value of the feature contribution of an important indicator parameter is greater than 0, it indicates that the important indicator parameter has a positive impact on the prediction result; and when the value of the feature contribution of an important indicator parameter is less than 0, it indicates that the important indicator parameter has a negative impact on the prediction result, wherein the larger the absolute value of the feature contribution of the important indicator parameter is, the greater the impact of the important indicator parameter on the prediction result is.

[0081] Figure 5 A schematic diagram showing a prediction result indicating the health of a user's joints and a characteristic contribution of each important indicator parameter according to an embodiment of the present disclosure.

[0082] Specifically, from Figure 5 It can be seen that the prediction result of the health level of the user's knee and hip joints is f(x)=0.999, indicating that the probability of the user's knee and hip joints being sub-healthy is 99.9%. Figure 5 The characteristic contribution of the eight selected important indicator parameters to the prediction results is also shown. Figure 5 The 8 important indicator parameters are: the average duration of standing squats (aver_dur_crouch), the average minimum angle that the knee joint can reach when squatting (min_angel_crouch), the minimum angle that the knee joint can reach when bending the right knee (min_angel_bend_knees_right), the minimum angle that the knee joint can reach when bending the left knee (min_angel_bend_knees_left), the maximum angle that the right leg can reach with the vertical direction when raising the knee and tapping the knees alternately with the left and right hands (max_angel_leg_lift_right), the maximum angle that the left leg can reach with the vertical direction when raising the knee and tapping the knees alternately with the left and right hands (max_angel_leg_lift_left), the duration of standing on one foot with the right foot (dur_one_foot_right), and the duration of standing on one foot with the left foot (dur_one_foot_left).

[0083] refer to Figure 5 , taking the min_angel_bend_knees_right indicator parameter as an example, the min_angel_bend_knees_right indicator parameter indicates the minimum angle that the knee joint can reach in the right knee bending action, and its corresponding feature contribution value is +0.3, which indicates that the indicator parameter has a positive impact on the prediction result of the sub-health level of the knee and hip joints, or it can be understood that the indicator parameter has caused the sub-health of the user's knee and hip joints to a large extent. In addition, taking the min_angel_crouch indicator parameter as an example, the min_angel_crouch indicator parameter indicates the average minimum angle that the knee joint can reach when squatting, and its corresponding feature contribution value is -0.02, which indicates that the parameter has a reverse impact on the prediction result of the sub-health level of the knee and hip joints, or it can be understood that the indicator parameter does not cause the sub-health of the user's knee and hip joints.

[0084] Reference again Figure 5, min_angel_bend_knees_right and min_angel_bend_knees_left indicator parameters indicate the minimum angles that the knee joint can reach in the right knee flexion movement and the minimum angles that the knee joint can reach in the left knee flexion movement, respectively, and their corresponding feature contribution values ​​are +0.3 and +0.03, respectively. That is, both min_angel_bend_knees_right and min_angel_bend_knees_left indicator parameters have a positive impact on the sub-health level of the user's knee and hip joints, or can be understood as causing the sub-health of the user's knee and hip joints to a certain extent. Combined with Figure 2 In (b), the two indicator parameters min_angel_bend_knees_right and min_angel_bend_knees_left correspond to Figure 2 The video action of the user completing the knee flexion action in the left and right directions shown in (b) of FIG. Figure 2 and Figure 5 It can be concluded that the user's left and right knee flexion movements reflect the sub-health of the user's knee joints to a certain extent, reminding the user to strengthen training in this area.

[0085] Through the explanation sub-model in the present disclosure, it is possible to further explain to users which detailed action indicators and their corresponding knee and hip joint actions have led to sub-health of the user's knee and hip joints, thereby providing users with corresponding improvement suggestions and solutions.

[0086] In process S1100, a trained prediction and explanatory model may be obtained based on the trained prediction sub-model and explanatory sub-model.

[0087] According to the prediction and explanatory model disclosed in the present invention, it is possible to help adjust the designed actions for evaluating the health of the user's joints, so as to more accurately evaluate the sub-health level of the user's joints, and improve the convenience of user operation. In addition, the prediction and explanatory model disclosed in the present invention includes prediction and explanation functions, which can not only output probability-based prediction results for evaluating the health of the user's joints, but also can intuitively explain to the user which joints of the body need to be focused on for improvement.

[0088] Figure 6 A flowchart S600 of a method for evaluating the health of a joint according to an embodiment of the present disclosure is shown.

[0089] like Figure 6 As shown, a flowchart S600 of a method for evaluating the health of a joint according to an embodiment of the present disclosure may include various steps shown according to processes S6020, S6040, S6060 and S6080.

[0090] In process S6020, multiple action video data of multiple groups of actions performed by the user on the joints can be obtained. Figure 2 The five sets of action videos shown in (a)-(e) are used to complete the corresponding actions.

[0091] In process S6040, a plurality of action video data may be input into a Figure 1 The predictive and explanatory model obtained by the training method of the predictive and explanatory model for assessing the health of joints is shown.

[0092] In one example, based on the predictive and explanatory model of an embodiment of the present disclosure, 14 detailed action index parameters for the user's knee and hip joints can be calculated through the five sets of action videos obtained, which may include: the average duration of standing and squatting, the average minimum angle that the knee joint can reach when squatting, the minimum angle that the knee joint can reach when right knee flexion action, the minimum angle that the knee joint can reach when left knee flexion action, the maximum angle that the hip joint can reach when right hip flexion action, the maximum angle that the hip joint can reach when left hip flexion action, the maximum angle that the hip joint can reach when right hip extension action, the maximum angle that the hip joint can reach when left hip extension action, the maximum angle that the right leg can reach with the vertical direction when the knee is raised and the left and right hands alternately pat the knees, the maximum angle that the left leg can reach with the vertical direction when the knee is raised and the left and right hands alternately pat the knees, the maximum angle that the left leg can reach with the vertical direction when standing on the right foot, the maximum angle that the right leg can reach with the vertical direction when standing on the left foot, the maintenance time when the right foot stands on one foot, and the maintenance time when the left foot stands on one foot.

[0093] Then, based on the feature screening function in the prediction and explanatory model of the embodiment of the present disclosure, the above 14 detailed action index parameters are feature screened to extract 8 important index parameters. In one example, the 8 important index parameters may include: the average duration of standing and squatting, the average minimum angle that the knee joint can reach when squatting, the minimum angle that the knee joint can reach when the right knee is bent, the minimum angle that the knee joint can reach when the left knee is bent, the maximum angle that the right leg can reach with the vertical direction when the knee is raised and the left and right hands alternately pat the knee, the maximum angle that the left leg can reach with the vertical direction when the knee is raised and the left and right hands alternately pat the knee, the duration of standing on one foot with the right foot, and the duration of standing on one foot with the left foot.

[0094] In process S6060, the health of the user's joints may be predicted based on the prediction and explanatory model to output a probability-based prediction result indicating the health of the user's joints.

[0095] In process S6080, feature contribution interpretation may be performed on the prediction results to output feature contribution rankings of detail action index parameters corresponding to each action in the plurality of action groups.

[0096] Combination Figure 5 According to the prediction and explanatory model of the embodiment of the present disclosure, the prediction results of the sub-health level of the user's knee and hip joints can be intuitively seen, as well as the characteristic contribution of the selected 8 important indicator parameters to the prediction results.

[0097] According to an embodiment of the present disclosure, based on the above prediction results and feature contribution rankings, the impact of each action in multiple groups of actions on the health of the user's joints can be determined, and customized exercise or training suggestions can be further provided to the user.

[0098] Figure 7 A block diagram of a training device 700 for evaluating a predictive and explanatory model for the health of a joint according to an embodiment of the present disclosure is shown.

[0099] like Figure 7 As shown, according to an embodiment of the present disclosure, a training device 700 for a predictive and explanatory model for evaluating the health of a joint includes: a memory 702 storing computer executable instructions; and a processor 704 coupled to the memory 702, configured to: obtain sample data of multiple users, wherein the sample data of each user includes multiple action video data of multiple groups of actions performed by each user on a joint, and each action of the multiple groups of actions includes multiple detail actions; rank the feature importance of multiple detail action index parameters indicating the multiple detail actions based on a feature screening algorithm, and screen out a certain number of detail action index parameters from the multiple detail action index parameters as multiple important index parameters according to the ranking result; and select the multiple important index parameters Input into a trained prediction sub-model to output a probability-based prediction result indicating the health of the joints of each user, wherein the prediction sub-model is trained using a plurality of important indicator parameters obtained from sample data of some of the multiple users as training data, and the prediction sub-model is tested using a plurality of important indicator parameters obtained from sample data of the remaining users of the multiple users as verification data to obtain the trained prediction sub-model; the prediction result is input into an explanation sub-model to perform feature contribution interpretation on the prediction result to determine the contribution of the plurality of important indicator parameters to the prediction result; and based on the trained prediction sub-model and the explanation sub-model, a trained prediction and explanatory model is obtained. Figure 7 The functions performed by the processor 704 in Figure 1 The functions performed by processes S1020 to S1100 are the same or similar and will not be described in detail here.

[0100] The memory 702 stores computer executable instructions, and when the computer executable instructions are executed by the processor 704, the training method of the prediction and explanatory model for assessing the health of the joint and the method for assessing the health of the joint implemented by the computer are implemented. The memory 702 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memory. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous connection dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM). It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0101] The processor 704 can perform various actions and processes according to the program stored in the memory 702. Specifically, the processor 704 can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present disclosure can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can be an X86 architecture or an ARM architecture.

[0102] Figure 8 An apparatus 800 for assessing the health of a joint according to an embodiment of the present disclosure is shown.

[0103] like Figure 8 As shown, the device 800 for evaluating the health of a joint according to an embodiment of the present disclosure includes: an acquisition unit 802 (not shown), configured to acquire a plurality of action video data of a plurality of groups of actions performed by a user on a joint; an input unit 804 (not shown), configured to input the plurality of action video data into a prediction and interpretation model, wherein the prediction and interpretation model is based on Figure 1The example shown is obtained by the training method of the predictive and explanatory model for evaluating the health of joints according to an embodiment of the present disclosure; a processing unit 806 (not shown) is configured to predict the health of the user's joints based on the predictive and explanatory model to output a probability-based prediction result indicating the health of the user's joints, and the processing unit 806 is further configured to perform feature contribution interpretation on the prediction result to output a feature contribution ranking of the detailed action indicator parameters corresponding to each action in the multiple groups of actions; and a display unit 808 is configured to display the prediction result and the feature contribution ranking.

[0104] The above-mentioned acquisition unit involved in the present disclosure can acquire in real time a plurality of action video data of a plurality of groups of actions that a user is performing on a joint. For example, the acquisition unit of the device can be a camera, which acquires a plurality of action video data for a plurality of groups of actions by shooting a video of a plurality of groups of actions that a user is performing on a joint in real time. Optionally, a user can locally upload a video of a plurality of groups of actions that have been previously completed for a joint to a device for evaluating the health of a joint according to an embodiment of the present disclosure, and then acquire a plurality of action video data from the plurality of groups of actions through the acquisition unit in the device. In addition, the device for evaluating the health of a joint according to an embodiment of the present disclosure can communicate with a device such as a server to acquire a video of a plurality of groups of actions that a user is performing on a joint from the server through the acquisition unit of the device.

[0105] The device for evaluating the health of joints according to the present disclosure can be any type of portable device (such as a smart phone, a tablet computer, etc.), or any type of fixed device (such as a desktop computer, etc.). Taking a smart phone as an example, by outputting the predicted results indicating the health of the user's joints and the impact of each of the multiple sets of actions performed by the user on the health of the user's joints to the display interface of the user's mobile phone, the user can intuitively see whether the joints are in a sub-healthy state, and can understand which aspects of exercise should be focused on.

[0106] In addition, the training method of the predictive and explanatory model for evaluating the health of joints and the computer-implemented method for evaluating the health of joints according to the present disclosure can be stored in a computer program product. Specifically, according to the present disclosure, a computer program product storing computer-readable instructions can be provided, and when the computer-readable instructions are executed by a processor, the processor can execute the training method of the predictive and explanatory model for evaluating the health of joints and the computer-implemented method for evaluating the health of joints as described above.

[0107] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains at least one executable instruction for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0108] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0109] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and should not be interpreted in an idealized or highly formal sense, unless explicitly defined as such herein.

[0110] The above is an explanation of the present disclosure and should not be considered as a limitation thereof. Although several exemplary embodiments of the present disclosure are described, it will be readily understood by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the present disclosure as defined in the claims. It should be understood that the above is an explanation of the present disclosure and should not be considered to be limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present disclosure is defined by the claims and their equivalents.

Claims

1. A method for training a predictive and explanatory model for assessing joint health, the method comprising: Acquire sample data of multiple users, wherein the sample data of each user includes multiple action video data of multiple groups of actions performed by each user on a joint, and each action of the multiple groups of actions includes multiple detailed actions; Ranking the feature importance of multiple detail action index parameters indicating the multiple detail actions based on a feature screening algorithm, and screening a certain number of detail action index parameters from the multiple detail action index parameters as multiple important index parameters according to the ranking result; Inputting the plurality of important indicator parameters into a trained prediction sub-model to output a probability-based prediction result indicating the health of the joints of each user, wherein the prediction sub-model is trained using the plurality of important indicator parameters obtained from sample data of some of the plurality of users as training data, and the prediction sub-model is tested using the plurality of important indicator parameters obtained from sample data of the remaining users of the plurality of users as verification data to obtain the trained prediction sub-model; Inputting the prediction results into the explanation sub-model to perform feature contribution interpretation on the prediction results to determine the contribution of the multiple important indicator parameters to the prediction results; as well as Based on the trained prediction sub-model and the explanation sub-model, a trained prediction and explanation model is obtained.

2. The method according to claim 1 further includes performing action detection on each of the multiple action video data through a key point detection algorithm to obtain the multiple quantized detail action indicator parameters indicating different detail actions in the multiple groups of actions. 3 . The method according to claim 1 , further comprising determining abnormal values ​​of the plurality of detail action index parameters, and removing the detail action index parameters determined to be abnormal values.

4. The method according to claim 3, wherein: An outlier determination is performed on the plurality of detail action index parameters based on an IQR statistical measurement method, wherein the IQR statistical measurement method comprises: A first value of the 25th percentile value minus 1.5 IQR and a second value of the 75th percentile value plus 1.5 IQR are calculated, and values ​​less than the first value and values ​​greater than the second value are determined as the abnormal values, wherein IQR refers to a value obtained by subtracting the 25th percentile value from the 75th percentile value.

5. The method according to claim 1, wherein: The feature screening algorithm includes the Boruta algorithm, and the plurality of important indicator parameters screened out include: constructing a plurality of shadow features corresponding to the plurality of detail action indicator parameters; Ranking the feature importance of the plurality of detail action indicator parameters based on the feature importance of the plurality of shadow features as a reference basis; and After multiple iterations, the certain number of detail action index parameters are screened out from the multiple detail action index parameters as the multiple important index parameters according to the ranking results.

6. The method according to claim 1, wherein: The prediction sub-model includes a LightGBM prediction model, and the training of the LightGBM prediction model includes: The optimal hyperparameter combination of the LightGBM prediction model is determined by constructing multiple value intervals for multiple sets of hyperparameters of the LightGBM prediction model to run a grid search strategy, and performing k-fold cross validation on the LightGBM prediction model.

7. The method according to claim 6, wherein: The k-fold cross validation includes 5-fold cross validation and 10-fold cross validation.

8. The method according to claim 1, wherein: Explaining the feature contribution of the prediction result includes: Calculating the characteristic contribution of each of the plurality of important indicator parameters based on the SHAP algorithm; and The contribution of each important indicator parameter to the prediction result is explained according to the feature contribution of each important indicator parameter.

9. The method according to claim 8, wherein: Explaining the feature contribution of the prediction result further includes: When the value of the characteristic contribution of the important indicator parameter is greater than 0, it means that the important indicator parameter has a positive impact on the prediction result; and When the value of the characteristic contribution of an important indicator parameter is less than 0, it means that the important indicator parameter has a negative impact on the prediction result. The larger the absolute value of the characteristic contribution of the important indicator parameter is, the greater the influence of the important indicator parameter on the prediction result is.

10. The method according to claim 1, wherein: The joints include at least one or more of a shoulder joint, an elbow joint, a hip joint, a wrist joint, a knee joint and an ankle joint.

11. The method according to any one of claims 1 to 10, wherein: The multiple sets of movements at least include: squatting, knee flexion angle in left and right directions, hip flexion and extension angle in left and right directions, knee lifting and alternating knee tapping with left and right hands, and standing on one leg.

12. The method according to claim 11, wherein: The multiple detailed action indicator parameters include at least: the average duration of standing and squatting, the average minimum angle that the knee joint can reach when squatting, the minimum angle that the knee joint can reach when bending the right knee, the minimum angle that the knee joint can reach when bending the left knee, the maximum angle that the hip joint can reach when bending the right hip, the maximum angle that the hip joint can reach when bending the left hip, the maximum angle that the hip joint can reach when extending the right hip, the maximum angle that the hip joint can reach when extending the left hip, the maximum angle that the right leg can reach with the vertical direction when raising the knee and patting the knee with the left and right hands alternately, the maximum angle that the left leg can reach with the vertical direction when raising the knee and patting the knee with the left and right hands alternately, the maximum angle that the left leg can reach with the vertical direction when standing on the right foot, the maximum angle that the right leg can reach with the vertical direction when standing on the left foot, the maintenance time when standing on the right foot, and the maintenance time when standing on the left foot.

13. The method according to claim 12, wherein: The multiple important indicator parameters include at least: the average duration of standing and squatting, the average minimum angle that the knee joint can reach when squatting, the minimum angle that the knee joint can reach when bending the right knee, the minimum angle that the knee joint can reach when bending the left knee, the maximum angle that the right leg can reach with the vertical direction when raising the knee and patting the knees alternately with the left and right hands, the maximum angle that the left leg can reach with the vertical direction when raising the knee and patting the knees alternately with the left and right hands, the duration of standing on one foot with the right foot, and the duration of standing on one foot with the left foot.

14. A computer-implemented method for assessing the health of a joint, the method comprising: Acquire multiple action video data of multiple sets of actions performed by the user on the joints; Inputting the plurality of action video data into a predictive and explanatory model; Predicting the health of the user's joints based on the prediction and explanatory model to output a probability-based prediction result indicating the health of the user's joints; as well as Performing feature contribution interpretation on the prediction results to output a feature contribution ranking of a detail action index parameter corresponding to each action in the plurality of action groups, Wherein, the predictive and explanatory model is obtained based on the method described in any one of claims 1 to 13.

15. The method according to claim 14, further comprising: Based on the prediction results and the feature contribution rankings, the influence of each action in the plurality of groups of actions on the health of the user's joints is determined.

16. The method according to claim 14 or 15, further comprising: Based on the prediction results and the feature contribution rankings, customized exercise or training suggestions are provided to the user.

17. A training device for a predictive and explanatory model for assessing joint health, the device comprising: a memory storing computer executable instructions; as well as a processor, coupled to the memory, the processor being configured to: Acquire sample data of multiple users, wherein the sample data of each user includes multiple action video data of multiple groups of actions performed by each user on a joint, and each action of the multiple groups of actions includes multiple detailed actions; Ranking the feature importance of multiple detail action index parameters indicating the multiple detail actions based on a feature screening algorithm, and screening a certain number of detail action index parameters from the multiple detail action index parameters as multiple important index parameters according to the ranking result; Inputting the plurality of important indicator parameters into a trained prediction sub-model to output a probability-based prediction result indicating the health of the joints of each user, wherein the prediction sub-model is trained using the plurality of important indicator parameters obtained from sample data of some of the plurality of users as training data, and the prediction sub-model is tested using the plurality of important indicator parameters obtained from sample data of the remaining users of the plurality of users as verification data to obtain the trained prediction sub-model; Inputting the prediction results into the explanation sub-model to perform feature contribution interpretation on the prediction results to determine the contribution of the multiple important indicator parameters to the prediction results; as well as Based on the trained prediction sub-model and the explanation sub-model, a trained prediction and explanation model is obtained.

18. A device for assessing the health of a joint, the device comprising: An acquisition unit, the acquisition unit being configured to acquire a plurality of action video data of a plurality of groups of actions performed by a user on a joint; An input unit configured to input the plurality of action video data into a prediction and interpretation model, wherein the prediction and interpretation model is obtained based on the method according to any one of claims 1 to 13; a processing unit configured to predict the health of the user's joints based on the prediction and explanatory model to output a probability-based prediction result indicating the health of the user's joints, The processing unit is further configured to perform feature contribution interpretation on the prediction result to output a feature contribution ranking of a detail action index parameter corresponding to each action in the plurality of groups of actions; as well as A display unit is configured to display the prediction result and the feature contribution ranking.

19. A computer program product, wherein: The computer program product has computer readable instructions stored thereon, which implement the method according to any one of claims 1 to 16 when executed by a processor.

Images