Action quality assessment method and system based on probabilistic uncertainty guidance
By introducing an uncertainty estimation module based on conditional variational automatic encoder into the action quality evaluation model, the existing models ignore the problem of ambiguity and high cost of multi-head networks is solved, and more flexible and accurate action quality evaluation is achieved.
Patent Information
- Application Number
- CN202510102097.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-22
- Publication Date
- 2025-02-28
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing action quality evaluation model ignores the ambiguity in the action quality evaluation task, and the multi-head network is costly and not flexible enough, making it difficult to effectively simulate the diversity of multiple referee scores.
An uncertainty estimation module based on conditional variational automatic encoder is introduced to simulate the generation of diversified predictions of different referee scoring processes by capturing the ambiguity in the action quality assessment task and sampling multiple times from the latent space.
The information on all referee scores is effectively utilized, which improves the flexibility and accuracy of the action quality assessment task, and allows for more flexible and accurate prediction of the final score.
Smart Images

Figure CN119541057A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of action quality assessment, and in particular to an action quality assessment method and system based on probability uncertainty guidance. Background Art
[0002] The task of action quality assessment aims to score the actions performed, such as in sports competitions, surgical skills assessment, piano skill level assessment, and skills scoring in daily life. Multiple judges are usually invited to score the same action to ensure fairness and comprehensiveness. The research on automatic action quality assessment scoring systems has attracted increasing interest due to its objectivity and labor-saving. Unlike action recognition tasks involving significantly different action categories, the videos used in action quality assessment tasks usually contain hundreds of frames, in which the actions and environments show subtle changes. This inherent similarity between actions and environments makes it more challenging to distinguish different scores, and it is necessary to capture the subtle changes that lead to score differences between videos.
[0003] In order to extract distinguishable features from action videos, some methods directly learn video-level features through 3D convolution, while others first obtain frame-level representations through 2D convolution and then use 1D convolution to introduce temporal information. Some studies also introduce posture-based features, which first identify sub-actions in the video and then extract relevant features. After generating features about the action, most existing methods use three learning strategies to evaluate the quality of the action: classification-based, regression-based, and contrastive learning-based.
[0004] All the methods mentioned above are deterministic models, that is, they only predict the final score of one action. However, such deterministic models ignore the inherent ambiguity of the action quality assessment task. In order to obtain fair results, multiple judges are usually invited to score the actions, so it becomes crucial for the automatic action quality assessment model to have the ability to simulate multiple judges' scores. However, most current methods directly ignore this effective information of multiple judges' scores, or simply predict each judge's score by building a multi-head model. Although these methods have achieved good performance, the multi-head network is costly and not flexible enough when dealing with an inconsistent number of judges scoring each action. Summary of the invention
[0005] In order to solve the above problems in the prior art, the present invention provides a method and system for action quality assessment based on probability uncertainty guidance. The invention introduces an uncertainty estimation module based on conditional variational autoencoder, which effectively captures the inherent ambiguity in the action quality assessment task, and can sample multiple times from the latent space to simulate and generate diversified predictions of different referee scoring processes, thereby enhancing the effective use of all referee scores and helping to predict the final score more flexibly and accurately. To achieve the above purpose, the technical solution is as follows:
[0006] On the one hand, a method for evaluating action quality based on probability uncertainty guidance is provided, the method comprising:
[0007] S1. Obtain a training video, and segment the training video to obtain multiple training video segments;
[0008] S2, inputting the plurality of training video clips into a video-level feature extraction module for training, to obtain a trained video-level feature extraction module;
[0009] S3, inputting the training video into an uncertainty estimation module based on a conditional variational autoencoder for training, to obtain a trained uncertainty estimation module based on a conditional variational autoencoder;
[0010] S4, obtaining a video to be evaluated, and segmenting the video to be evaluated to obtain a plurality of video segments to be evaluated;
[0011] S5, processing the plurality of video clips to be evaluated by the trained video-level feature extraction module to obtain video-level features ;
[0012] S6: Process the video to be evaluated by the uncertainty estimation module based on the trained conditional variational autoencoder to obtain uncertainty-related features. ;
[0013] S7. The uncertainty-related features With the video level features Perform a merge join to get the final feature representation ;
[0014] S8, representing the features Send it to a multi-layer perceptron regressor to get the prediction score .
[0015] Optionally, in S2, the plurality of training video clips are input into a video-level feature extraction module for training to obtain a trained video-level feature extraction module, comprising:
[0016] S21, input the training video clip into the Inflated 3D ConvNets backbone network to obtain clip-level features ;
[0017] S22, the segment-level features Input to the weighted attention module to obtain the segment-level features Video-level features Contribution weight ;
[0018] S23, the contribution weight Use the softmax function to normalize along the fragment dimension to obtain the normalized contribution weight ;
[0019] S24, according to the segment-level features and the normalized contribution weight , through formula (1), we get the video-level feature ,
[0020] (1)
[0021] In the formula, is a video-level feature, is the segment-level feature of the kth segment, It is The normalized contribution weight of each fragment, ⊙ represents element-by-element multiplication.
[0022] Optionally, the weighted attention module includes four fully connected layers, and the first three fully connected layers of the four fully connected layers have a rectified linear unit activation function.
[0023] Optionally, in S3, the training video is input into an uncertainty estimation module based on a conditional variational autoencoder for training to obtain a trained uncertainty estimation module based on a conditional variational autoencoder, comprising:
[0024] S31, extracting 3D features from the training video through a feature extractor to obtain input data for an uncertainty estimation module based on a conditional variational autoencoder;
[0025] S32, inputting the input data into the prior network to obtain the mean, logarithmic variance and latent variable z of the prior network latent space 1 ;
[0026] S33, inputting the input data and the sampled label scores of the training video into the posterior network to obtain the mean, logarithmic variance and latent variable z of the posterior network latent space 2 ;
[0027] S34, according to the mean and logarithmic variance of the posterior network latent space and the mean and logarithmic variance of the test network latent space, using the Kullback–Leibler divergence calculation method, obtain the distance between the prior network and the posterior network distribution ;
[0028] S35, according to the potential variable z of the prior network potential space 1 , mapped to a scalar through two fully connected layers, obtaining the uncertainty of the prior network estimate ;
[0029] S36, according to the latent variable z of the posterior network latent space 2 , mapped to a scalar through two fully connected layers, obtaining the uncertainty of the posterior network estimate ;
[0030] S37, uncertainty estimated according to the prior network and the uncertainty of the posterior network estimate According to formula (2), the uncertainty of the prior network estimate is obtained and the uncertainty of the posterior network estimate The distance between ;
[0031] (2)
[0032] In the formula, is the uncertainty of the prior network estimate and the uncertainty of the posterior network estimate The distance between is the uncertainty of the prior network estimate, is the uncertainty of the posterior network estimate;
[0033] S38: Prediction score based on the training video and the sampled label scores to obtain the reweighted automatic action quality assessment loss ;
[0034] S39, evaluating the loss according to the re-weighted automatic action quality , the uncertainty of the prior network estimate and the uncertainty of the posterior network estimate The distance between The distance between the prior network and the posterior network distribution , through formula (3), we get the loss function ,
[0035] (3)
[0036] In the formula, is the loss function, is the uncertainty of the prior network estimate and the uncertainty of the posterior network estimate The distance between is the reweighted automatic action quality assessment loss, is the distance between the prior network and the posterior network distribution, yes The weight coefficient of yes The weight coefficient of
[0037] S310, according to the loss function , assigning weights to the training videos in the training learning so that the mean of the prior network latent space is consistent with the mean of the posterior network latent space and the logarithmic variance of the prior network latent space is consistent with the logarithmic variance of the posterior network latent space, thereby obtaining a trained uncertainty estimation module based on conditional variational autoencoder.
[0038] Optionally, the feature extractor includes four 3D convolutional layers, the input channels of the 3D convolutional layers are 3, 16, 32 and 64 respectively, and the output channels are 16, 32, 64 and 128 respectively.
[0039] Optionally, the prior network is a learnable Gaussian distribution associated with the input variables, and the posterior network is a Gaussian distribution associated with the input variables and conditions.
[0040] Optionally, the prior network includes three fully connected layers, two ReLU activation function layers and two dropout layers.
[0041] Optionally, in S6, the video to be evaluated is processed by the uncertainty estimation module based on the trained conditional variational autoencoder to obtain uncertainty-related features ,include:
[0042] (4)
[0043] In the formula, is the uncertainty-related feature, is the mean of the prior network latent space, is random noise sampled from a normal distribution, is the log-variance of the prior network latent space.
[0044] Optionally, in S38, the predicted score of the training video is and the sampled label scores to obtain the reweighted automatic action quality assessment loss ,include:
[0045] (5)
[0046] In the formula, is the reweighted automatic action quality assessment loss, is the prediction score of the training video, y is the sampled label score of the training video, is the uncertainty of the posterior network estimate.
[0047] On the other hand, a motion quality assessment system based on probability uncertainty guidance is provided, the system is applied to a motion quality assessment method based on probability uncertainty guidance, the system comprising:
[0048] A training video module is used to obtain a training video and segment the training video to obtain multiple training video segments;
[0049] A first training module, used for inputting the plurality of training video clips into a video-level feature extraction module for training to obtain a trained video-level feature extraction module;
[0050] A second training module is used to input the training video into an uncertainty estimation module based on a conditional variational autoencoder for training, so as to obtain a trained uncertainty estimation module based on a conditional variational autoencoder;
[0051] The video module to be evaluated is used to obtain the video to be evaluated, and segment the video to be evaluated to obtain multiple video segments to be evaluated;
[0052] The first feature module is used to process the plurality of video clips to be evaluated through the trained video-level feature extraction module to obtain video-level features. ;
[0053] The second feature module is used to process the video to be evaluated through the uncertainty estimation module based on the trained conditional variational autoencoder to obtain uncertainty-related features ;
[0054] Feature joining module, used to connect the uncertainty related features With the video level features Perform a merge join to get the final feature representation ;
[0055] Prediction score module, used to represent the feature Send it to a multi-layer perceptron regressor to get the prediction score .
[0056] Compared with the prior art, the technical solution of the present invention has at least the following beneficial effects:
[0057] The above scheme introduces an uncertainty estimation module based on conditional variational autoencoder, which effectively captures the inherent ambiguity in the motion quality assessment task, and can sample multiple times from the latent space to simulate and generate diversified predictions of different referee scoring processes, thereby enhancing the effective utilization of all referee scores and helping to predict the final score more flexibly and accurately. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0059] Figure 1 is a flow chart of an embodiment of an action quality assessment method based on probability uncertainty guidance of the present invention;
[0060] Figure 2 It is a flow chart of video-level feature extraction module training of an embodiment of the motion quality assessment method based on probability uncertainty guidance of the present invention;
[0061] Figure 3 It is a flow chart of uncertainty estimation module training based on conditional variational autoencoder of an embodiment of the method for action quality assessment based on probability uncertainty guidance of the present invention;
[0062] Figure 4 It is a schematic diagram of an embodiment of the method for evaluating action quality based on probability uncertainty guidance of the present invention;
[0063] Figure 5 A system block diagram is provided for an embodiment of the action quality assessment system guided by probability uncertainty of the present invention. DETAILED DESCRIPTION
[0064] The technical solution of the present invention is described below in conjunction with the accompanying drawings.
[0065] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.
[0066] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.
[0067] like Figures 1 to 4 As shown, an embodiment of the present invention provides an action quality assessment method based on probability uncertainty guidance, which can be implemented by an action quality assessment system based on probability uncertainty guidance, such as Figure 1 The flowchart provided by the embodiment of the present invention shown in the figure comprises the following steps:
[0068] S1. Obtain a training video, and segment the training video to obtain multiple training video segments;
[0069] S2, inputting the plurality of training video clips into a video-level feature extraction module for training, to obtain a trained video-level feature extraction module;
[0070] Specifically, Figure 2 As shown, in S2, the plurality of training video clips are input into a video-level feature extraction module for training to obtain a trained video-level feature extraction module, including:
[0071] S21, input the training video clip into the Inflated 3D ConvNets (I3D) backbone network to obtain clip-level features ;
[0072] S22, the segment-level features Input to the weighted attention (WA) module to obtain the segment-level features Video-level features Contribution weight ;
[0073] S23, the contribution weight Use the softmax function to normalize along the fragment dimension to obtain the normalized contribution weight ;
[0074] S24, according to the segment-level features and the normalized contribution weight , through formula (1), we get the video-level feature ,
[0075] (1)
[0076] In the formula, is a video-level feature, is the segment-level feature of the kth segment, is the normalized contribution weight of the kth fragment, and ⊙ represents element-by-element multiplication.
[0077] The weighted attention module includes four fully connected layers, and the first three fully connected layers of the four fully connected layers have rectified linear unit activation functions.
[0078] S3, inputting the training video into an uncertainty estimation module based on a conditional variational auto-encoder (CVAE) for training, to obtain a trained uncertainty estimation module based on a conditional variational auto-encoder;
[0079] Specifically, Figure 3 As shown, in S3, the training video is input into the uncertainty estimation module based on the conditional variational autoencoder for training, and the trained uncertainty estimation module based on the conditional variational autoencoder is obtained, including:
[0080] S31, extracting 3D features from the training video through a feature extractor to obtain input data for an uncertainty estimation module based on a conditional variational autoencoder;
[0081] S32, inputting the input data into the prior network to obtain the mean, logarithmic variance and latent variable z of the prior network latent space 1 ;
[0082] S33, inputting the input data and the sampled label scores of the training video into the posterior network to obtain the mean, logarithmic variance and latent variable z of the posterior network latent space 2 ;
[0083] S34, according to the mean and logarithmic variance of the posterior network latent space and the mean and logarithmic variance of the test network latent space, using the Kullback–Leibler divergence calculation method, obtain the distance between the prior network and the posterior network distribution ;
[0084] S35, according to the potential variable z of the prior network potential space 1, mapped to a scalar through two fully connected layers, obtaining the uncertainty of the prior network estimate ;
[0085] S36, according to the latent variable z of the posterior network latent space 2 , mapped to a scalar through two fully connected layers, obtaining the uncertainty of the posterior network estimate ;
[0086] S37, uncertainty estimated according to the prior network and the uncertainty of the posterior network estimate According to formula (2), the uncertainty of the prior network estimate is obtained and the uncertainty of the posterior network estimate The distance between ;
[0087] (2)
[0088] In the formula, is the uncertainty of the prior network estimate and the uncertainty of the posterior network estimate The distance between is the uncertainty of the prior network estimate, is the uncertainty of the posterior network estimate;
[0089] S38: Prediction score based on the training video and the sampled label scores to obtain the reweighted automatic action quality assessment loss ;
[0090] S39, evaluating the loss according to the re-weighted automatic action quality , the uncertainty of the prior network estimate and the uncertainty of the posterior network estimate The distance between The distance between the prior network and the posterior network distribution , through formula (3), we get the loss function ,
[0091] (3)
[0092] In the formula, is the loss function, is the uncertainty of the prior network estimate and the uncertainty of the posterior network estimate The distance between is the reweighted automatic action quality assessment loss, is the distance between the prior network and the posterior network distribution, yes The weight coefficient of yes The weight coefficient of
[0093] S310, according to the loss function , assigning weights to the training videos in the training learning so that the mean of the prior network latent space is consistent with the mean of the posterior network latent space and the logarithmic variance of the prior network latent space is consistent with the logarithmic variance of the posterior network latent space, thereby obtaining a trained uncertainty estimation module based on conditional variational autoencoder.
[0094] The feature extractor includes four 3D convolutional layers, the input channels of the 3D convolutional layers are 3, 16, 32 and 64 respectively, and the output channels are 16, 32, 64 and 128 respectively.
[0095] The prior network is a learnable Gaussian distribution associated with the input variables, and the posterior network is a Gaussian distribution associated with the input variables and the conditions.
[0096] The prior network includes three fully connected layers, two ReLU activation function layers and two dropout layers.
[0097] The predicted score of the training video in S38 and the sampled label scores to obtain the reweighted automatic action quality assessment loss ,include:
[0098] (5)
[0099] In the formula, is the reweighted automatic action quality assessment loss, is the prediction score of the training video, y is the sampled label score of the training video, is the uncertainty of the posterior network estimate.
[0100] S4, obtaining a video to be evaluated, and segmenting the video to be evaluated to obtain a plurality of video segments to be evaluated;
[0101] S5, processing the plurality of video clips to be evaluated by the trained video-level feature extraction module to obtain video-level features ;
[0102] S6: Process the video to be evaluated by the uncertainty estimation module based on the trained conditional variational autoencoder to obtain uncertainty-related features. ;
[0103] Specifically, in S6, the video to be evaluated is processed by the uncertainty estimation module based on the trained conditional variational autoencoder to obtain uncertainty-related features. ,include:
[0104] (4)
[0105] In the formula, is the uncertainty-related feature, is the mean of the prior network latent space, is random noise sampled from a normal distribution, is the log-variance of the prior network latent space.
[0106] S7. The uncertainty-related features With the video level features Perform a merge join to get the final feature representation ;
[0107] S8, representing the features Send it to a multi-layer perceptron regressor to get the prediction score .
[0108] Figure 5 A motion quality assessment system based on probability uncertainty guidance is shown according to an exemplary embodiment. The system is used for a motion quality assessment method based on probability uncertainty guidance. Figure 5 The system includes a training video module 510, a first training module 520, a second training module 530, a to-be-evaluated video module 540, a first feature module 550, a second feature module 560, a feature connection module 570, and a prediction score module 580, wherein
[0109] The training video module 510 is used to obtain a training video and segment the training video to obtain a plurality of training video segments;
[0110] The first training module 520 is used to input the plurality of training video clips into a video-level feature extraction module for training to obtain a trained video-level feature extraction module;
[0111] The second training module 530 is used to input the training video into the uncertainty estimation module based on the conditional variational autoencoder for training, so as to obtain the trained uncertainty estimation module based on the conditional variational autoencoder;
[0112] The to-be-evaluated video module 540 is used to obtain the to-be-evaluated video and segment the to-be-evaluated video to obtain a plurality of to-be-evaluated video segments;
[0113] The first feature module 550 is used to process the plurality of video clips to be evaluated through the trained video-level feature extraction module to obtain video-level features. ;
[0114] The second feature module 560 is used to process the video to be evaluated through the uncertainty estimation module based on the trained conditional variational autoencoder to obtain uncertainty-related features ;
[0115] The feature linking module 570 is used to link the uncertainty-related features With the video level features Perform a merge join to get the final feature representation ;
[0116] The prediction score module 580 is used to represent the feature Send it to a multi-layer perceptron regressor to get the prediction score .
[0117] The present invention provides a motion quality assessment method and system based on probabilistic uncertainty guidance. The present invention introduces an uncertainty estimation module based on a conditional variational autoencoder, which effectively captures the inherent ambiguity in the motion quality assessment task, and can sample multiple times from the latent space to simulate and generate diversified predictions of different referee scoring processes, thereby enhancing the effective use of all referee scores and helping to predict the final score more flexibly and accurately.
[0118] It is to be understood that the present invention is described by the above embodiments and should not be construed as limiting the embodiments of the present invention and the scope of the present invention. It is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of the present application are within the scope of protection of the present invention.
Claims
1. A motion quality assessment method based on probability uncertainty guidance, characterized in that: The method comprises: S1. Obtain a training video, and segment the training video to obtain multiple training video segments; S2, inputting the plurality of training video clips into a video-level feature extraction module for training, to obtain a trained video-level feature extraction module; S3, inputting the training video into an uncertainty estimation module based on a conditional variational autoencoder for training, to obtain a trained uncertainty estimation module based on a conditional variational autoencoder; S4, obtaining a video to be evaluated, and segmenting the video to be evaluated to obtain a plurality of video segments to be evaluated; S5, processing the plurality of video clips to be evaluated by the trained video-level feature extraction module to obtain video-level features ; S6: Process the video to be evaluated by the uncertainty estimation module based on the trained conditional variational autoencoder to obtain uncertainty-related features. ; S7. The uncertainty-related features With the video level features Perform a merge join to get the final feature representation ; S8, representing the features Send it to a multi-layer perceptron regressor to get the prediction score .
2. The motion quality assessment method based on probability uncertainty guidance according to claim 1 is characterized in that: In S2, the plurality of training video clips are input into a video-level feature extraction module for training to obtain a trained video-level feature extraction module, including: S21, input the training video clip into the Inflated 3D ConvNets backbone network to obtain clip-level features ; S22, the segment-level features Input to the weighted attention module to obtain the segment-level features Video-level features Contribution weight ; S23, the contribution weight Use the softmax function to normalize along the fragment dimension to obtain the normalized contribution weight ; S24, according to the segment-level features and the normalized contribution weight , through formula (1), we can get the video-level feature , (1) In the formula, is a video-level feature, is the segment-level feature of the kth segment, It is The normalized contribution weight of each fragment, ⊙ represents element-by-element multiplication.
3. The motion quality assessment method based on probability uncertainty guidance according to claim 2 is characterized in that: The weighted attention module includes four fully connected layers, and the first three fully connected layers of the four fully connected layers have rectified linear unit activation functions.
4. The motion quality assessment method based on probability uncertainty guidance according to claim 1 is characterized in that: In S3, the training video is input into an uncertainty estimation module based on a conditional variational autoencoder for training, so as to obtain a trained uncertainty estimation module based on a conditional variational autoencoder, including: S31, extracting 3D features from the training video through a feature extractor to obtain input data for an uncertainty estimation module based on a conditional variational autoencoder; S32, inputting the input data into the prior network to obtain the mean, logarithmic variance and latent variable z1 of the prior network latent space; S33, inputting the input data and the sampled label scores of the training video into the posterior network to obtain the mean, logarithmic variance and latent variable z2 of the latent space of the posterior network; S34, according to the mean and logarithmic variance of the latent space of the posterior network and the mean and logarithmic variance of the latent space of the test network, using the Kullback–Leibler divergence calculation method, obtain the distance between the distribution of the prior network and the posterior network ; S35, according to the potential variable z1 of the prior network latent space, mapping to a scalar through two fully connected layers to obtain the uncertainty of the prior network estimate ; S36, according to the latent variable z2 of the posterior network latent space, mapping to a scalar through two fully connected layers to obtain the uncertainty of the posterior network estimate ; S37, uncertainty estimated according to the prior network and the uncertainty of the posterior network estimate According to formula (2), the uncertainty of the prior network estimate is obtained and the uncertainty of the posterior network estimate The distance between ; (2) In the formula, is the uncertainty of the prior network estimate and the uncertainty of the posterior network estimate The distance between is the uncertainty of the prior network estimate, is the uncertainty of the posterior network estimate; S38: Prediction score based on the training video and the sampled label scores to obtain the reweighted automatic action quality assessment loss ; S39, evaluating the loss according to the re-weighted automatic action quality , the uncertainty of the prior network estimate and the uncertainty of the posterior network estimate The distance between The distance between the prior network and the posterior network distribution , through formula (3), we get the loss function , (3) In the formula, is the loss function, is the uncertainty of the prior network estimate and the uncertainty of the posterior network estimate The distance between is the reweighted automatic action quality assessment loss, is the distance between the prior network and the posterior network distribution, yes The weight coefficient of yes The weight coefficient of S310, according to the loss function , assigning weights to the training videos in the training learning so that the mean of the prior network latent space is consistent with the mean of the posterior network latent space and the logarithmic variance of the prior network latent space is consistent with the logarithmic variance of the posterior network latent space, thereby obtaining a trained uncertainty estimation module based on conditional variational autoencoder.
5. The method for evaluating motion quality based on probability uncertainty guidance according to claim 4, characterized in that: The feature extractor includes four 3D convolutional layers, the input channels of the 3D convolutional layers are 3, 16, 32 and 64 respectively, and the output channels are 16, 32, 64 and 128 respectively.
6. The method for evaluating motion quality based on probability uncertainty guidance according to claim 4, characterized in that: The prior network is a learnable Gaussian distribution associated with the input variables, and the posterior network is a Gaussian distribution associated with the input variables and the conditions.
7. The motion quality assessment method based on probability uncertainty guidance according to claim 4 is characterized in that: The prior network includes three fully connected layers, two ReLU activation function layers and two dropout layers.
8. The method for evaluating motion quality based on probability uncertainty guidance according to claim 4, characterized in that: In S6, the video to be evaluated is processed by the uncertainty estimation module based on the trained conditional variational autoencoder to obtain uncertainty-related features. ,include: (4) In the formula, is the uncertainty-related feature, is the mean of the prior network latent space, is random noise sampled from a normal distribution, is the log-variance of the prior network latent space.
9. The method for evaluating motion quality based on probability uncertainty guidance according to claim 4, characterized in that: S38 predicts the score based on the training video and the sampled label scores to obtain the reweighted automatic action quality assessment loss ,include: (5) In the formula, is the reweighted automatic action quality assessment loss, is the prediction score of the training video, y is the sampled label score of the training video, is the uncertainty of the posterior network estimate.
10. An action quality assessment system based on probability uncertainty guidance, used to implement the action quality assessment method based on probability uncertainty guidance as described in any one of claims 1 to 9, characterized in that: include: A training video module is used to obtain a training video and segment the training video to obtain multiple training video segments; A first training module, used for inputting the plurality of training video clips into a video-level feature extraction module for training to obtain a trained video-level feature extraction module; A second training module is used to input the training video into an uncertainty estimation module based on a conditional variational autoencoder for training, so as to obtain a trained uncertainty estimation module based on a conditional variational autoencoder; The video module to be evaluated is used to obtain the video to be evaluated, and segment the video to be evaluated to obtain multiple video segments to be evaluated; The first feature module is used to process the plurality of video clips to be evaluated through the trained video-level feature extraction module to obtain video-level features. ; The second feature module is used to process the video to be evaluated through the uncertainty estimation module based on the trained conditional variational autoencoder to obtain uncertainty-related features ; Feature joining module, used to connect the uncertainty related features With the video level features Perform a merge join to get the final feature representation ; Prediction score module, used to represent the feature Send it to a multi-layer perceptron regressor to get the prediction score .
Citation Information
Patent Citations
Semi-supervised reference-free image quality evaluation method based on uncertainty estimation
CN117541562A
Medical image data sample uncertainty evaluation method and system
CN118968232A