A motion function evaluation system and method based on multi-view video capture

Abstract

The application provides a motion function evaluation system and method based on multi-view video capture, and relates to the technical field of human posture recognition. The system comprises a data acquisition module for acquiring industrial camera parameters and a synchronous frame sequence, a model construction module for constructing a motion posture estimation model, a 2D posture identification module for inputting the synchronous frame sequence into the motion posture estimation model to obtain a synchronous frame sequence labeled with key point labels, a 3D model reconstruction module for calculating a bone frame sequence based on the labeled synchronous frame sequence and the industrial camera parameters, a motion parameter extraction module for extracting kinematic indexes of a tester, and a motion function evaluation module for evaluating the motion function grade of the tester. Through multi-view acquisition, occlusion perception posture estimation and three-dimensional reconstruction, the application realizes accurate quantification of actions and evaluation of motion functions, and improves the comprehensiveness of human motion function evaluation.

Description

Technical Field

[0001] This application relates to the field of human posture recognition technology, and in particular to a motion function evaluation system and method based on multi-view video capture. Background Technology

[0002] With increasing societal focus on the health and physical fitness of young adults, the importance of scientific and systematic motor function assessment is becoming increasingly prominent in fields such as sports training, military training, rehabilitation medicine, and physical fitness screening for high-risk occupations. Human motor function encompasses not only joint flexibility, muscle strength, and endurance, but also multi-dimensional indicators including motor coordination, rotation and anti-rotation capabilities, core stability, and abnormal posture recognition. High-level athletes and professional physical trainers need to use scientific assessment methods to quantitatively analyze movement quality in order to optimize training effectiveness and prevent sports injuries.

[0003] Currently, there are mature motor function assessment systems internationally, such as the Functional Movement Screening (FMS) in the United States and the Physical Movement Function Assessment (EMPF) in China. Although these motor function assessment systems provide a reference for human motor function assessment, they still have certain limitations in practical applications, including: insufficient movement dimensions: lack of comprehensive evaluation of human rotation, anti-rotation ability and core stability, and inability to systematically detect abnormal postures such as scoliosis and elbow hyperextension; limited assessment accuracy and efficiency: existing systems usually use single-view or two-dimensional measurement, which is easily affected by occlusion and angle limitations, and cannot comprehensively capture human movement data. Summary of the Invention

[0004] To improve the comprehensiveness and reliability of human motion function assessment, this application provides a motion function assessment system and method based on multi-view video capture.

[0005] Firstly, this application provides a motion function evaluation system based on multi-view video capture, employing the following technical solution: A motion function assessment system based on multi-view video capture, the system comprising: The data acquisition module is used to acquire industrial camera parameters and simultaneously acquire video data of the tester performing preset test actions using multiple industrial cameras. It also extracts images of each industrial camera at the same moment from the acquired video data and aggregates them to obtain each synchronous frame group. By aggregating all synchronous frame groups, a synchronous frame sequence is obtained. The model building module is used to build motion attitude estimation models based on the RTMPose model. The 2D pose labeling module is used to input the obtained synchronous frame sequence into the motion pose estimation model to obtain a synchronous frame sequence labeled with keypoints; the keypoint labels include keypoint name, keypoint 2D coordinates, keypoint confidence and visibility label; The 3D model reconstruction module is used to calculate the three-dimensional coordinates of the test subject's key points using triangulation based on the labeled synchronous frame sequence and industrial camera parameters, and to collect the three-dimensional coordinates of the test subject's key points to obtain the test subject's skeletal frame sequence. The motion parameter extraction module is used to extract the kinematic parameters of the test subject based on the test subject's skeletal frame sequence and the labeled synchronous frame sequence. The motor function assessment module is used to build a motor function assessment model. It inputs the extracted kinematic indicators of the test subject into the motor function assessment model and outputs the test subject's motor function level.

[0006] Optionally, the key points include the nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, right ankle, top of head, neck, center of hip, left big toe, right big toe, left little toe, right little toe, left heel, and right heel.

[0007] Optionally, the data acquisition module may also include a layout constraint submodule; The layout constraint submodule is used to constrain the installation position of industrial cameras, including spatial constraint units, field-of-view constraint units, and temporal constraint units; Spatial constraint unit, used to constrain the arrangement of each industrial camera in different positions relative to the tester; Field of view constraint unit, used to constrain each key point of the tester to be within the field of view intersection area of ​​at least two industrial cameras; The time constraint unit is used to ensure that the acquisition time synchronization error of each industrial camera is less than a preset error threshold.

[0008] Optionally, the data acquisition module further includes a device parameter acquisition submodule, which includes an intrinsic parameter acquisition unit and an extrinsic parameter acquisition unit; the industrial camera parameters include a camera intrinsic parameter matrix and a camera extrinsic parameter matrix; The intrinsic parameter acquisition unit is used to capture several chessboard images using each industrial camera, obtain the pixel position of the chessboard corner point in each chessboard image, and iteratively fit the pinhole camera model based on the pixel position of the chessboard corner point in each chessboard image and the preset world coordinate origin to obtain the camera intrinsic parameter matrix of each industrial camera. The extrinsic parameter acquisition unit is used to select any industrial camera as the new world coordinate origin to construct a new world coordinate system. Based on the pixel position of the corner point of the checkerboard in each checkerboard image and the new world coordinate system, it calculates the rotation matrix and translation vector of other industrial cameras relative to the new world coordinate origin to obtain the camera extrinsic parameter matrix of each industrial camera.

[0009] Optionally, a model training module is also included after executing the model building module and before executing the 2D pose identification module; The model training module includes a training set construction submodule and a training submodule; A training set submodule is constructed to set up a training video dataset when multiple industrial cameras synchronously acquire training videos of the test subject performing training actions. Training images of each industrial camera at the same moment are extracted from the acquired training video dataset and collected separately to obtain each training synchronous frame group. All training synchronous frame groups are collected to obtain a training synchronous frame sequence. The test subject's key point labels are labeled on each training image in the training synchronous frame sequence. The labeled training synchronous frame sequence is recorded as the training dataset. The training submodule is used to define the loss function, input the training dataset into the motion pose estimation model for iterative training, calculate the model loss based on the loss function during the iterative training process, update the model gradient based on the model loss, and use the gradient descent algorithm to iteratively optimize the model parameters to obtain the trained motion pose estimation model. The trained motion pose estimation model is recorded as the new motion pose estimation model.

[0010] Optionally, the motion pose estimation model includes an input layer, a feature extraction layer, an occlusion feature perception layer, and an output layer; The training submodule specifically includes: inputting the training dataset into the motion pose estimation model from the input layer; the feature extraction layer extracting features from the training dataset, extracting feature maps for each training image in the training dataset; inputting the obtained feature maps into the occlusion feature perception layer for global average pooling to generate channel weights; weighting the channel features of each feature map; performing channel max pooling and average pooling on the channel-weighted feature maps to generate spatial weights; weighting the feature values ​​of the feature maps of each training image at each spatial coordinate position to obtain spatially weighted feature maps; convolving and fusing the channel-weighted feature maps and spatially weighted feature maps; denoting the fused feature map as the occlusion feature perception map; inputting the occlusion feature perception map into the output layer to output the model prediction result; calculating the model loss according to the defined loss function; updating the model gradient based on the model loss; and iteratively optimizing the model parameters using the gradient descent algorithm to obtain the trained motion pose estimation model; and denoting the trained motion pose estimation model as the new motion pose estimation model.

[0011] Optionally, visibility labels include fully visible labels, location-estimated labels, and location-unestimated labels; the 3D model reconstruction module also includes an image filtering submodule; The image filtering submodule is used to traverse the key points of images in each synchronous frame group in the labeled synchronous frame sequence, delete the images containing key points in the current synchronous frame group whose key point visibility label is a label that cannot be estimated and whose key point confidence is less than a preset confidence threshold, take the synchronous frame group after deleting the images as the new synchronous frame group, update the synchronous frame sequence based on the new synchronous frame group, and denoted as the new synchronous frame sequence.

[0012] Optionally, the kinematic parameters of the test subject include heel-off-ground distance, lower limb joint angle, difference in left and right knee coordinates, trunk upright angle, difference in left and right shoulder height, touch height, difference in spinal stride, shoulder-hip rotation angle, spinal plane rotation angle, spinal lateral displacement angle, and elbow joint movement distance. The motion parameter extraction module includes: a first extraction submodule, a second extraction submodule, and a third extraction submodule; The first extraction submodule is used to extract the distance of the heel off the ground, the angle of the lower limb joints, the difference in coordinates between the left and right knees, the upright angle of the torso, and the difference in height between the left and right shoulders. The second extraction submodule is used to extract the difference between touch height and spine span. The third extraction submodule is used to extract shoulder and hip rotation angles, spinal plane rotation angles, spinal lateral displacement angles, and elbow joint movement distances.

[0013] Secondly, this application provides a motion function evaluation method based on multi-view video capture, wherein the control method is applicable to the control system described in any one of the first aspects above, and the method includes: A method for evaluating motion function based on multi-view video capture, the method comprising: Data acquisition: Obtain industrial camera parameters, and use multiple industrial cameras to synchronously acquire video data when the tester performs preset test actions. Extract the images of each industrial camera at the same moment from the acquired video data and collect them separately to obtain each synchronous frame group. Collect all synchronous frame groups to obtain a synchronous frame sequence. Model Construction: Based on the RTMPose model, a motion attitude estimation model is constructed; 2D pose labeling: The obtained synchronous frame sequence is input into the motion pose estimation model to obtain a synchronous frame sequence labeled with keypoints; the keypoint labels include keypoint name, keypoint 2D coordinates, keypoint confidence and visibility label; 3D model reconstruction: Based on the labeled synchronous frame sequence and industrial camera parameters, the three-dimensional coordinates of the test subject's key points are calculated using triangulation. The three-dimensional coordinates of the test subject's key points are then collected to obtain the test subject's skeletal frame sequence. Motion parameter extraction: Based on the test subject's skeletal frame sequence and labeled synchronous frame sequence, the test subject's kinematic parameters are extracted; Motor function assessment: Construct a motor function assessment model, input the extracted kinematic indicators of the test subjects into the motor function assessment model, and output the test subjects' motor function level.

[0014] In summary, this application includes at least one of the following beneficial technical effects: 1. This application utilizes multiple industrial cameras to simultaneously acquire video data of the test subject performing preset test actions. It combines the constructed motion posture estimation model to perform key point annotation and two-dimensional posture estimation. Based on the industrial camera parameters, it uses triangulation to reconstruct the three-dimensional coordinates of key points, generating a skeletal frame sequence of the test subject. Simultaneously, it extracts multidimensional kinematic indicators from the skeletal frame sequence and inputs them into the evaluation model, outputting the motor function level. This achieves a scientific quantitative assessment of the test subject's motor ability, providing a reliable basis for motor function analysis, rehabilitation training, and health management.

[0015] 2. This application also adds visibility labels to key points, which helps guide the model to accurately predict key point positions even in cases of partial occlusion or missing key points. This improves the robustness and training effect of the pose estimation model, enhances the model's adaptability to complex actions and occluded scenes, and thus improves the accuracy and reliability of kinematic index extraction and motion function assessment.

[0016] 3. This application combines the visibility labels and confidence levels of key points to filter images containing key points whose locations cannot be estimated and whose confidence levels are low. This helps reduce the possibility of errors in 3D reconstruction using unreliable data, solves the problems of occlusion limitations, incomplete data, and low reconstruction accuracy in traditional single-view or 2D measurements, improves the accuracy of skeletal frame sequences and kinematic indices, and provides a reliable basis for comprehensive and accurate assessment of motor function.

[0017] 4. This application sets up an occlusion perception layer. By using the channel weights and spatial weights generated by the occlusion perception layer as intermediate supervision features, these features participate in the backpropagation process of the keypoint confidence loss function and the visibility classification loss function. This allows the confidence loss function and the visibility classification loss function to be weighted according to the spatial weights and channel weights of the region corresponding to the keypoint. This makes the occlusion region exert a stronger constraint on the model parameter update, which helps the model learn the correspondence between the channel weights and spatial weights and the occlusion state of the keypoint during training. This enhances the model's adaptability to complex actions and occlusion scenarios, thereby improving the accuracy and reliability of kinematic index extraction and motion function evaluation. Attached Figure Description

[0018] Figure 1 This is a schematic block diagram of Embodiment 1 of this application; Figure 2 This is a schematic diagram of the structure of Embodiment 1 of this application. Detailed Implementation

[0019] The following combination Figures 1 to 2 This application will be described in further detail.

[0020] Example 1: This example discloses a motion function evaluation system based on multi-view video capture. The system is deployed on an integrated edge computing device 100, such as... Figure 1 As shown, the system includes: a data acquisition module 110 for acquiring industrial camera parameters and synchronous frame sequences; a model building module 120 for constructing a motion posture estimation model; a 2D posture labeling module 140 for inputting the synchronous frame sequences into the motion posture estimation model to obtain synchronous frame sequences labeled with key points; a 3D model reconstruction module 150 for calculating the test subject's skeletal frame sequence based on the labeled synchronous frame sequences and industrial camera parameters; a motion parameter extraction module 160 for extracting the test subject's kinematic indicators; and a motion function evaluation module 170 for evaluating the test subject's motion function level. The specific steps in this embodiment are as follows: The data acquisition module 110 includes a layout constraint submodule 111, a device parameter acquisition submodule 112, and an image acquisition submodule 113.

[0021] The arrangement constraint submodule 111 is used to constrain the installation position of the industrial camera, including a spatial constraint unit, a field of view constraint unit, and a temporal constraint unit.

[0022] Spatial constraint units are used to constrain the placement and angle of each industrial camera, ensuring that the camera's position covers different orientations of the tester. Specifically, the constraints... The distribution of industrial cameras in three-dimensional space makes the angle between the optical axes of any two adjacent industrial cameras... It satisfies formula (1) and contains at least one pitch angle. Industrial cameras.

[0023] The field-of-view constraint unit is used to ensure that each key point of the tester is within the intersection area of ​​the fields of view of at least two industrial cameras, thus guaranteeing the quality of multi-view fusion. Specifically, it constrains any key point of the tester. It must fall within the intersection of the fields of view of at least two industrial cameras, satisfying formula (2).

[0024] The time constraint unit is used to ensure that the acquisition time synchronization error of each industrial camera is less than a preset error threshold. Specifically, the acquisition time of each device is constrained through a network clock synchronization protocol. It must satisfy formula (3).

[0025]

[0026]

[0027]

[0028] in, Industrial cameras and industrial cameras The angle between them , , Indicates the number of industrial cameras. Industrial cameras Field of view, Represents the characteristic function, Industrial cameras The collection time, Industrial cameras The collection time, In this embodiment, a preset error threshold is represented between industrial cameras. , Indicates key points.

[0029] The submodule 112 for acquiring equipment parameters includes an intrinsic parameter acquisition unit and an extrinsic parameter acquisition unit. The intrinsic parameter acquisition unit is used to acquire several checkerboard images from different angles and distances using various industrial cameras. It acquires the pixel positions of the corner points of the checkerboard in each image, uses a feature extraction algorithm to detect the pixel coordinates of the corner points, acquires pre-set 3D world coordinate points corresponding to the checkerboard structure, and uses these as preset world coordinate points. It establishes a mapping relationship between the 2D pixel coordinates of the corner points of each image and the preset world coordinate points. Based on the pinhole camera model, it calibrates the imaging characteristics, uses a nonlinear least squares method to iteratively fit the pinhole camera model, obtains the focal length, principal point offset, and distortion coefficients of each industrial camera, and outputs the camera intrinsic parameter matrix for each industrial camera.

[0030] Specifically, the calculation formula (4) for mapping the pixel coordinates of the chessboard corner points to the preset world coordinate points is as follows:

[0031] in, Indicates the scale factor. Represents the pixel coordinates of the corner points of the chessboard. Represents the intrinsic parameter matrix, Represents the preset world coordinates. and Indicates the focal length of an industrial camera. This indicates the principal point offset.

[0032] Considering lens distortion, the calculation formula (5) of the distortion model is optimized by nonlinear least squares method to obtain high-precision distortion coefficients.

[0033]

[0034] in, For normalized, distortion-free coordinates, The radial distortion coefficient is... The tangential distortion coefficient is... , Represents the distorted x-coordinate. Represents the ordinate after distortion. This indicates the distance from the corner points of the checkerboard grid to the center of the industrial camera lens. Those skilled in the art can set this distance according to the specific industrial camera model. and Values.

[0035] The extrinsic parameter acquisition unit is used to arbitrarily select one industrial camera from all industrial cameras as the new world coordinate origin, construct the new world coordinates using the pixel coordinates of the checkerboard corner points acquired by the industrial camera, and calculate the rotation matrix and translation vector of the other industrial cameras relative to the new world coordinate origin by minimizing the reprojection error based on the pixel coordinates of the checkerboard corner points in the checkerboard images of other industrial cameras. This yields the camera extrinsic parameter matrix of each industrial camera.

[0036] Specifically, the objective function for optimizing the rotation matrix and translation vector is solved by minimizing the reprojection error, as shown in Equation (6):

[0037] in, This represents the objective function to be optimized. Indicates the number of industrial cameras. Indicates the number of corner points on the chessboard. Indicates the corner points of the chessboard. , Industrial cameras checkerboard corners in the image pixel coordinates, Represents the projection function. Industrial cameras The intrinsic parameter matrix, Industrial cameras The rotation matrix, Industrial cameras The translation vector, For the corner points of the chessboard The coordinates in the new world coordinate system. By iteratively solving this nonlinear optimization problem, a globally unified camera extrinsic parameter matrix is ​​finally output.

[0038] The image acquisition submodule 113, after setting up the positions of the industrial cameras and acquiring the parameters of each camera, controls multiple industrial cameras to trigger simultaneously and record video data of the tester performing preset test actions. Each industrial camera's recorded video data has a timestamp. Based on the timestamps, images with the same timestamp are extracted from the video data recorded by different industrial cameras and aggregated to obtain various synchronized frame groups. Multiple synchronized frame groups are then aggregated in chronological order to obtain a synchronized frame sequence. Each image is labeled with the name and location of the corresponding industrial camera. The preset test actions include a full squat with elbows clasped, upper limb flexibility exercises, and a push-up exercise. The full squat with elbows clasped involves the tester standing with elbows bent at 90 degrees, hands clasped around elbows, shoulders bent and raised behind the head, feet together, and squatting to the lowest point while maintaining an upright upper body. This is repeated three times. Upper limb flexibility exercises are performed with the subject standing, alternating between both arms. The first movement involves the palm facing outwards, reaching upwards from behind to touch the highest point of the opposite shoulder blade; the second movement involves the palm facing inwards, reaching downwards from behind to touch the highest point of the same-side shoulder blade. The push-up exercise involves the subject in a push-up position with hands shoulder-width apart, elbows straight, palms on the ground, feet shoulder-width apart, spine, hips, and knees straight, alternately raising arms and legs while maintaining body stability. Industrial camera parameters include the camera intrinsic parameter matrix and the camera extrinsic parameter matrix.

[0039] Specifically, such as Figure 2 As shown, in this embodiment, three industrial cameras with a resolution of 1080P and a frame rate of 60 are selected, including industrial camera 101, industrial camera 102 and industrial camera 103. The three cameras form an equilateral triangle around the test area where the tester performs the preset test action, and the optical axes of the three cameras intersect at the center of the test area. At least one camera is positioned in front of the tester, one camera is positioned to the side and rear of the tester, and one camera is positioned at a downward angle of 20°-45° diagonally forward and above the tester. The field of view overlap of two adjacent cameras is greater than 60%. The height of the cameras is adjusted according to the height range of the tester's specified action, so that each key point of the tester is within the imaging area of ​​at least two cameras. Synchronization triggers are used to connect the cameras or a unified clock source is used for timing synchronization, so that each camera exposes instantly for each trigger signal, ensuring that the time deviation between the same frame is less than 1ms.

[0040] A standard checkerboard calibration board is placed in the test area. Several checkerboard images are acquired from different angles and distances using three cameras. A feature extraction algorithm is then used to detect the pixel coordinates of the checkerboard corner points. Based on a preset world coordinate origin, the homography matrix of each checkerboard image from the checkerboard calibration board plane to the checkerboard image plane is calculated. Initial values ​​of the intrinsic parameter matrices are decomposed from multiple homography matrices using orthogonal constraints. The distance between the projected checkerboard corner points and the actual checkerboard corner points is calculated and minimized. A reprojection error minimization objective function based on all checkerboard corner points is constructed. A nonlinear least squares optimization method is used to optimize the camera's intrinsic parameter matrix, minimizing the distance between the pixel coordinates of the projected checkerboard corner points and the pixel coordinates of the actually detected checkerboard corner points, thus obtaining the intrinsic parameter matrix that best matches the real imaging model. Alternatively, any camera is selected and used as the new world coordinate origin. A new world coordinate system is established using the pixel coordinates of the checkerboard corner points acquired by this camera. The rotation matrices and translation vectors of the other two cameras relative to the current camera are calculated using the minimized reprojection error, resulting in the extrinsic parameter matrices of the three cameras.

[0041] A trigger signal is sent, and the three cameras receive the trigger signal through a synchronization trigger to achieve synchronous acquisition. After recording is completed, images at the same moment are extracted from the video data recorded by different industrial cameras according to the timestamp and then collected to obtain a synchronization frame group. Multiple synchronization frame groups are collected in chronological order to obtain a synchronization frame sequence.

[0042] In this city example, to facilitate the adjustment of the height and angle of the industrial camera, a support assembly can be provided for each industrial camera. The support assembly can be a telescopic structure.

[0043] The model building module 120 is used to construct a motion pose estimation model based on the RTMPose model. In this embodiment, the motion pose estimation model adopts an improved RTMPose architecture. The backbone of the motion pose estimation model uses a CSPNEXt structure, and the resolution of the input image is normalized to 288x384 pixels to balance inference speed and feature extraction precision. To enhance the model's ability to capture key human features, a channel attention mechanism is integrated into the backbone network. This mechanism is configured to adaptively recalibrate the channel weights of the feature map, assigning higher response weights to effective features and suppressing background noise, thereby improving the model's feature extraction efficiency in complex backgrounds. The prediction head of the motion pose estimation model is configured to output 26 keypoints conforming to the Halpe-26 protocol. These keypoints cover common human joints and foot keypoints including the left and right big toes, left and right little toes, and left and right heels. Keypoint localization employs the SimCC (Simulate Classification for Coordinate Regression) algorithm. This algorithm transforms the keypoint coordinate regression problem into a horizontal and vertical classification problem, and optimizes it using the KL divergence loss function. Compared to traditional heatmap regression, the SimCC algorithm eliminates the quantization error caused by low-resolution feature maps, achieving sub-pixel level localization accuracy.

[0044] In this embodiment, the training process uses a hybrid dataset and is configured for 300 epochs. The optimizer chosen is AdamW, and the base learning rate is set to 3.10. -4 The training process employed a learning rate adjustment strategy combining warmup and cosine annealing. To improve the model's robustness to limb occlusion in practical applications, a two-stage data augmentation strategy was introduced during training: in the regular training phase and the second phase (the last 30 epochs), augmentation techniques including random flipping, random half-body transformation, and coarse-grained dropout were applied. Coarse-grained dropout simulated the visual effect of occluded limbs by randomly generating occluded regions in the training images (e.g., setting max_holes=1, max_height=0.4), forcing the model to utilize attention mechanisms to focus on unoccluded contextual features, thereby inferring the location of occluded keypoints.

[0045] To ensure the accuracy of the motion pose estimation model, this embodiment also includes a model validation module to evaluate the model's performance during training. The validation process uses keypoint accuracy (PCK) and area under the curve (AUC) as core evaluation metrics. The PCK metric calculates the proportion of predicted keypoints whose normalized distance to the ground truth keypoints is less than a preset threshold (e.g., 0.1), while the AUC metric comprehensively reflects the model's robustness across different accuracy thresholds. Furthermore, to improve the generalization ability and stability of the final deployed model, an exponential moving average (EMA) mechanism is introduced during training to smooth the model weights. When the validated PCK and AUC metrics meet preset requirements, the current EMA model weights are exported as the final motion pose estimation model.

[0046] Specifically, the motion pose estimation model includes an input layer, a feature extraction layer, an occlusion feature perception layer, a visual feature fusion layer, and an output layer.

[0047] The model training module 130 includes a training set construction submodule 131 and a training submodule 132. Details are as follows: The training set submodule 131 is used to synchronously collect a training video dataset with timestamps from multiple testers performing training actions. Based on the timestamp, from Training images at the same moment are extracted from training video data recorded by different industrial cameras and aggregated to obtain a training synchronization frame group. ,in, express Time Industrial Camera The acquired training images are aggregated into multiple training synchronization frame groups in chronological order to obtain a training synchronization frame sequence. Keypoint labels for the test subjects are annotated on the training images in the training synchronization frame sequence, and the annotated training synchronization frame sequence is recorded as the training dataset. Keypoint labels include keypoint name, keypoint 2D coordinates, keypoint confidence, and visibility label. Keypoint names include nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, right ankle, top of head, neck, center of hip, left big toe, right big toe, left little toe, right little toe, left heel, and right heel. Visibility labels include fully visible, location estimable, and location unestimateable labels. A fully visible label indicates that the keypoint is fully visible in the acquired image; a location estimable label indicates that the keypoint is not fully visible but its location can be inferred; and a location unestimateable label indicates that the keypoint is completely invisible and its location cannot be accurately inferred.

[0048] In this embodiment, the key points are specific location points used to describe the structure of the human body. Among these key points... The key point label vector is , ,in, Indicate key points Two-dimensional coordinates in the image coordinate system; Indicate key points Confidence level; Indicate key points Visibility tags Indicate key points For fully visible tags, Indicate key points For location-predictable tags, Indicate key points The location cannot be used to estimate the label. Indicates the current key point The time of image acquisition.

[0049] The training submodule 132 is used to define the loss function, input the training dataset into the motion pose estimation model for iterative training, calculate the model loss based on the loss function during iterative training, update the model gradient based on the model loss, and iteratively optimize the model parameters using the gradient descent algorithm to obtain the trained motion pose estimation model. The trained motion pose estimation model is then recorded as the new motion pose estimation model. The specific steps are as follows: The training dataset is input into the motion pose estimation model from the input layer, and the feature extraction layer extracts features from the training dataset, extracting feature maps for each training image. The obtained feature map The input occlusion feature perception layer undergoes global average pooling to generate channel weights. Based on channel weight The channel features of each feature map are weighted to obtain the channel feature weighted feature map. The calculation formula is: ,in, This indicates element-wise multiplication. It is the Sigmoid activation function. The weight matrix corresponding to the channel features, and the feature map after weighting the channel features. Channel max pooling and average pooling are performed to generate spatial weights. Based on spatial weights The feature map is obtained by weighting the feature values ​​at each spatial coordinate position. The calculation formula is: The feature map after channel weighting Feature map with spatial weighting Perform convolutional fusion, and denote the fused feature map as the occlusion feature perception map. .

[0050] Occlusion feature perception map Input visual feature fusion layer, and perceive occlusion features according to acquisition time. Grouping them together yields a set of data from the same time point. The occlusion feature perception maps of several industrial cameras are denoted as view feature map groups. Based on the camera extrinsic parameter matrices of each industrial camera, each pixel in each view feature map group is mapped to the same 3D coordinate system. Based on the spatial positional relationship of features under each viewpoint, feature vectors corresponding to the same spatial position in each view feature map group are aligned to obtain an aligned feature set. Using the confidence score of each viewpoint as a weight, features in the aligned feature set are fused to obtain a fused feature map, which is called a multi-view feature map. Specifically, let a point in the current coordinate system be... , obtain the Intrinsic parameter matrix of an industrial camera With extrinsic matrix Establish the projection mapping relationship between 3D spatial points and pixel coordinates of 2D feature maps. The calculation formula is as follows: The formula for feature fusion calculation based on the confidence weights of each perspective for each feature in the aligned feature set is as follows: .in, express Dot projection on industrial camera Pixel coordinates on the image Industrial cameras The weight of the confidence level, Industrial cameras The features of the feature map.

[0051] The multi-view feature maps are input into the output layer, and the model prediction results are output. Specifically, the SimCC algorithm is used to input the multi-view feature maps into the prediction head of the motion pose estimation model, predicting the key points in the horizontal direction. (axis) and vertical direction ( The classification log odds vector (axis) and Finally, key points are determined by finding the maximum response value of the classification vector. Two-dimensional coordinates The calculation formula is: , The model loss is calculated based on the defined loss function, the model gradient is updated based on the model loss, and the model parameters are iteratively optimized using the gradient descent algorithm to obtain a trained motion pose estimation model. The trained motion pose estimation model is then denoted as the new motion pose estimation model.

[0052] In this embodiment, the occlusion feature perception layer helps the model reduce the predicted confidence value of keypoints in low-weight regions or with low-weight channel responses when predicting keypoint confidence and visibility labels, based on the distribution characteristics of channel weights and spatial weights in the occlusion feature perception map. It also tends to classify keypoints in low-weight regions or with low-weight channel responses as either having an estimable location label or an unestimable location label. Furthermore, this embodiment effectively reduces the impact of occlusion and viewpoint differences on pose estimation accuracy by mapping multi-view features to a unified space and combining them with confidence scores for weighted fusion.

[0053] The loss function formula is as follows: B ; ; ; ; .

[0054] in, This represents the loss function for keypoint coordinates. Represents the confidence loss function. This represents the visibility classification loss function. Represents the consistency loss function. express The weight, express The weight, express The weight, Indicate key points In this embodiment, the weights are... , , , , Indicate key points The visibility tag, in this embodiment, when hour, ; Key points of images in the training dataset that the model predicts Two-dimensional coordinates Key points of images in the training dataset Actual two-dimensional coordinates Indicates the first One key point, Key points representing model predictions Confidence level Indicate key points The target confidence level, Key points representing model predictions Visibility tags, Representing key points of images in the training dataset Actual visibility tag, Indicates the current perspective At the key point Occlusion feature perception vector at location, Key points after multi-view fusion eigenvectors.

[0055] In this embodiment, the channel weights and spatial weights generated by the occlusion perception layer serve as intermediate supervisory features, participating in the backpropagation process of the keypoint confidence loss function and the visibility classification loss function. This allows the confidence loss function and the visibility classification loss function to be weighted according to the spatial weights and channel weights of the region corresponding to the keypoint, thus imposing stronger constraints on the model parameter updates due to the occlusion region. This helps the model learn the correspondence between channel weights and spatial weights and the occlusion state of the keypoint during training. Simultaneously, by combining dynamic weighting of visibility labels and multi-view consistency constraints, the model can effectively identify occluded keypoints during training, improving the accuracy of keypoint 2D coordinate prediction, confidence estimation accuracy, and visibility determination reliability, while ensuring the consistency and robustness of keypoint features across multiple views.

[0056] The 2D attitude identification module 140 is used to input the obtained synchronization frame sequence into the motion attitude estimation model to obtain a synchronization frame sequence labeled with key point tags.

[0057] The 3D model reconstruction module 150 includes an image filtering submodule 151 and a model reconstruction submodule 152. The image filtering submodule 151 is used to traverse the key points of the images in each synchronous frame group in the labeled synchronous frame sequence, delete the images containing key points in the current synchronous frame group whose key point visibility label is "location cannot be estimated" and whose key point confidence is less than a preset confidence threshold, and use the synchronous frame group after deleting the images as the new synchronous frame group. The synchronous frame sequence is updated based on the new synchronous frame group and denoted as the new synchronous frame sequence.

[0058] The model reconstruction submodule 152 is used to calculate the three-dimensional coordinates of each key point of the test subject using triangulation based on the labeled synchronous frame sequence and industrial camera parameters, collect the calculated three-dimensional coordinates of each key point in the current frame group, obtain the test subject's skeletal frame group in the current frame group, and collect all the obtained skeletal frame groups to obtain the test subject's skeletal frame sequence.

[0059] The motion parameter extraction module 160 is used to extract the kinematic parameters of the test subject based on the test subject's skeletal frame sequence and the labeled synchronous frame sequence. The test subject's kinematic parameters include heel-to-ground distance, lower limb joint angles, left and right knee coordinate differences, trunk upright angle, left and right shoulder height differences, touch height, spinal stride differences, shoulder-hip rotation angles, spinal plane rotation angles, spinal lateral displacement angles, and elbow joint movement distances. The motion parameter extraction module 160 includes a first extraction submodule 161, a second extraction submodule 162, and a third extraction submodule 163.

[0060] The first extraction submodule 161 is used to extract the distance from the heel to the ground. Lower limb joint angles Difference between left and right knee coordinates , torso upright angle and the difference in height between the left and right shoulders The specific extraction process includes: Based on the parameters of various industrial cameras, images captured by industrial cameras with an optical axis angle between 30° and 150° from the test subject's frontal direction are recorded as side images. The two-dimensional coordinates of the key points, the left and right ankles, are obtained from each side image. Using the left and right ankles as center points, image regions of preset sizes are cropped and recorded as the left ankle region image and the right ankle region image, respectively. A foot binary classification model is constructed. The left and right ankle region images are input into the foot binary classification model to determine whether the heel is not off the ground in either the left or right ankle region image. If so, the result is output as the distance of the heel off the ground. Conversely, the output result is the distance of the heel off the ground. The output result is used as the distance of the heel off the ground. .

[0061] Obtain the three-dimensional coordinates of the left hip, right hip, left knee, right knee, left ankle, and right ankle. For each time point in the skeletal frame group, construct the left hip-left knee vector and the left knee-left ankle vector based on the left hip, left knee, and left ankle, respectively. Use the vector dot product to solve for the angle between the left hip-left knee vector and the left knee-left ankle vector, denoted as the left lower limb angle. Similarly, obtain the angle between the right hip-right knee vector and the right knee-right ankle vector, denoted as the right lower limb angle. Calculate the average of the left and right lower limb angles at each time point. From the average values ​​of the left and right lower limb angles at each time point, select the smallest average value and denot it as the lower limb joint angle of the skeletal frame group. .

[0062] Calculating the lower limb joint angles of the current skeletal frame group At any given time, obtain the three-dimensional coordinates of the left and right knees in the current skeletal frame group, calculate the height difference between the left and right knees, obtain the height difference between the left and right knees at each time point, and select the smallest height difference from the height differences between the left and right knees at each time point as the left and right knee coordinate difference value of the skeletal frame group. .

[0063] Obtain the 3D coordinates of the left shoulder, right shoulder, left hip, and right hip in the skeletal frame group at each time point. Calculate the angle difference between the left shoulder and left hip, denoted as the left torso upright angle. Calculate the angle difference between the right shoulder and right hip, denoted as the right torso upright angle. Calculate the average of the left and right torso upright angles at each time point. From the average values ​​of the left and right torso upright angles at each time point, select the smallest average value and denot it as the trunk upright angle. .

[0064] Obtain the 3D coordinates of the left and right shoulders in the skeletal frame group at each time point, calculate the height difference between the left and right shoulders at each time point, and select the smallest height difference from the height differences between the left and right shoulders as the left and right shoulder height difference. .

[0065] It should be noted that all the above kinematic indices are calculated frame by frame at multiple time points during all movements, and the index value corresponding to the extreme value of the movement amplitude is selected from the calculation results.

[0066] The second extraction sub-model 162 is used to extract the difference between touch height and spine span, specifically including: Obtain the 3D coordinates of the center of the neck and hips, and construct a spine vector based on these coordinates. Based on the skeletal frame group at each time point, obtain the 3D coordinates of the left and right shoulders. Based on the spatial positions of the left and right shoulders, extract a pre-defined scapular region centered on each shoulder. Input the extracted scapular regions at each time point into the pre-constructed scapular coordinate prediction model, and output the coordinates of four virtual keypoints at each time point: the superior angle of the left scapula, the inferior angle of the left scapula, the superior angle of the right scapula, and the inferior angle of the right scapula.

[0067] Specifically, the lightweight object detection network YOLO11n is used as the basic network structure of the scapular coordinate prediction model. Multiple images of the scapular region of a preset size are collected, and the coordinates of the upper left scapula, lower left scapula, upper right scapula, and lower right scapula are labeled on the collected scapular region images. The labeled scapular region images are used as the training set and input into the scapular coordinate prediction model for training, resulting in a trained scapular coordinate prediction model. The trained scapular coordinate prediction model is then used as the constructed scapular coordinate prediction model.

[0068] A Hand21 hand model was constructed, and the fingertip coordinates of the left and right middle fingers were extracted from it. The superior angle of the left scapula was designated as the first target reference point, and the superior angle of the right scapula as the second target reference point. When the test subject performed the first movement of the upper limb flexibility exercise, the displacement difference between the fingertip coordinates of the left middle finger and the second target reference point was calculated. From the displacement differences between the fingertip coordinates of the left middle finger and the second target reference point at various time points, the smallest displacement difference was selected and recorded as the left finger displacement difference. Calculate the displacement difference between the right middle fingertip coordinate and the first target reference point. From the displacement differences between the right middle fingertip coordinate and the first target reference point at each time point, select the smallest displacement difference and record it as the right finger displacement difference. The difference in left finger displacement and the difference in right finger displacement Record as touch height The formula for calculating the displacement difference is as follows: , This represents the y-axis coordinate of the tip of the middle finger, including the y-axis coordinate of the tip of the left middle finger. The y-axis coordinate of the tip of the right middle finger , This represents the y-axis coordinate of the target reference point, including the y-axis coordinate of the first target reference point. y-axis coordinates of the second target reference point .

[0069] Obtain the coordinates of the spine's center position in the horizontal direction. Calculate the difference between the coordinates of the left middle fingertip and the center position of the spine in the horizontal direction at each time point. From these differences, select the smallest difference and record it as the first spine difference. Calculate the coordinate difference between the fingertip of the right middle finger and the horizontal center position of the spine at each time point. From the coordinate differences between the fingertip of the right middle finger and the horizontal center position of the spine at each time point, select the smallest coordinate difference and record it as the second spinal difference. ,Will and This is denoted as the spinal stride difference, where the formula for calculating the coordinate difference is: , Indicates the coordinate difference. This represents the x-axis coordinate of the middle fingertip. The x-axis coordinate represents the center position of the spine in the horizontal direction. This represents the fingertip direction coefficient, with the left middle finger having a fingertip direction coefficient of -1 and the right middle finger having a fingertip direction coefficient of 1.

[0070] The third extraction submodule 163 is used to extract shoulder and hip rotation angles, spinal plane rotation angles, spinal lateral displacement angles, and elbow joint movement distances. Specifically, it includes: The three-dimensional coordinates of the left and right hips at each time point were obtained. The center points of the left and right hips were selected as pelvic coordinates. The three-dimensional coordinates of the neck at the corresponding time points were obtained. Based on the three-dimensional coordinates of the neck and pelvis, trunk vectors were constructed for each time point. The three-dimensional coordinates of the left shoulder, right shoulder, left elbow, and right elbow at each time point were obtained. Based on the three-dimensional coordinates of the left shoulder and left elbow at each time point, left limb vectors were constructed. Based on the three-dimensional coordinates of the right shoulder and right elbow at each time point, right limb vectors were constructed. The left and right limb vectors were denoted as upper limb vectors. Using the vector dot product formula, the angle between the trunk vector and the upper limb vector at each time point was calculated. From the angles between the trunk vector and the upper limb vector at each time point, the angle with the smallest value was selected and recorded as the shoulder-hip rotation angle. The calculation formula is: ,in, Represents the torso vector. This represents the vector of the upper limb.

[0071] Based on the 3D coordinates of the left and right shoulders at each time point, left and right shoulder vectors are constructed to obtain the left and right shoulder vectors of the initial time point skeletal frame group. Calculate the left and right shoulder vectors of the skeletal frame groups at each time point. and the left and right shoulder vectors of the skeletal frame group at the initial time point The angle between the left and right shoulder vectors of the skeletal frame group at each time point. and the left and right shoulder vectors of the initial time point skeletal frame group Among the angles, the angle with the smallest value is selected and recorded as the spinal plane rotation angle. The calculation formula is: .

[0072] Obtain the two-dimensional coordinates of the neck and hip centers at each time point. Calculate the two-dimensional vector of the spine based on these coordinates. Then, calculate the two-dimensional vector of the spine and its absolute perpendicular vector. The angle between them, from the two-dimensional vector of the spine and the absolute perpendicular vector at each time point. Among the angles between them, the angle with the smallest value is selected and recorded as the spinal lateral displacement angle. The calculation formula is: , The x-axis coordinate of the neck is represented. This represents the y-axis coordinate of the neck. The x-axis coordinate represents the center of the buttocks. The y-axis coordinate represents the center of the buttocks.

[0073] Obtain the three-dimensional coordinates of the left and right elbows at each time point, denoted as elbow coordinates. Calculate the positional dispersion of the elbow coordinates within a preset movement time using the standard deviation algorithm from statistics. Select the positional dispersion with the smallest value from the positional dispersion values ​​within each preset movement time and record it as the elbow joint movement distance. The calculation formula is: ,in This indicates the average position of the elbow joint within a preset movement time. This indicates the number of frames within the preset movement time. Indicates the first The position of the elbow joint.

[0074] The motor function assessment module 170 is used to construct a motor function assessment model. It inputs the extracted kinematic indices of the test subject's joints into the model and outputs the test subject's motor function level. Specifically, the motor function assessment model is constructed based on preset assessment rules. These preset rules include rules for elbow-hugging squats, upper limb flexibility movements, and push-ups. The test subject's scores for each item are summed according to the preset rules to obtain their motor function score. Based on the pre-set motor function level and the test subject's motor function score, the module outputs the motor function level corresponding to the test subject's motor function score.

[0075] Elbow-hugging full squat assessment criteria: When performing the elbow-hugging full squat, if the test subject meets the following criteria... , , , and If the result is 5 points, then it is counted as 5 points; the characteristics of the elbow-holding squatting dysfunction include , , and 4 points are awarded if the test subject meets any one of the above-mentioned characteristics of elbow-hugging squatting dysfunction; 3 points are awarded if the test subject meets any two of the above-mentioned characteristics of elbow-hugging squatting dysfunction; 2 points are awarded if the test subject meets any three of the above-mentioned characteristics of elbow-hugging squatting dysfunction; 1 point is awarded if the test subject meets all four of the above-mentioned characteristics of elbow-hugging squatting dysfunction; If the test taker fails to meet all of the above scoring criteria while performing the action, a score of 0 will be awarded. This indicates the acceptable value for the torso's upright angle. This indicates the preset acceptable value for the difference in height between the left and right shoulders. This indicates the acceptable value for the difference in coordinates between the left and right knees.

[0076] Upper limb flexibility assessment rules: When performing upper limb flexibility exercises, if... , , and If the condition of 5 points is not met, then Boolean decision is made based on the following four secondary geometric constraints: (1) (2) (3) (4) If the test taker meets one of the above constraints, 4 points are awarded; if the test taker meets two of the above constraints, 3 points are awarded; if the test taker meets three of the above constraints, 2 points are awarded; if the test taker meets all of the above constraints, 1 point is awarded; if the test taker does not meet any of the above scoring conditions when performing the action, 0 points are awarded.

[0077] Push-up performance evaluation rules: The high constraint range for the pass / fail value of the push-up performance is: , The constraint range for the pass / fail performance in the push-up motion is: , The lower constraint range for the pass value of the push-up motion is: , If the tester meets the requirements , If the test taker meets the high constraint range of the push-up movement, 5 points will be awarded; if the test taker meets the requirements... If the test taker meets the acceptable range for the push-up motion, 4 points are awarded; or if the test taker meets the acceptable range for the push-up motion, 4 points are awarded. However, if the test taker meets the acceptable range for the push-up movement, 4 points will be awarded; if the test taker meets the acceptable range... If the test taker meets the minimum acceptable value for the push-up movement and is within the lower constraint range, then 3 points are awarded; if the test taker meets the minimum acceptable value for the push-up movement... If the test taker meets the minimum acceptable value for the push-up movement and is within the lower constraint range, then 3 points are awarded; if the test taker meets the minimum acceptable value for the push-up movement... If the test taker meets the acceptable range for the push-up motion, 2 points are awarded; if the test taker meets the acceptable range for the push-up motion, 2 points are awarded. If the test taker meets the minimum acceptable value for the push-up movement, 1 point is awarded; if the test taker does not meet any of the above scoring conditions when performing the movement, 0 points are awarded.

[0078] Example 2: This example discloses a motion function assessment method based on multi-view video capture. The method is implemented based on the motion function assessment system based on multi-view video capture described in Example 1. The method specifically includes: Data Acquisition: Several checkerboard images are captured using various industrial cameras. The pixel positions of the checkerboard corners in each image are obtained. Based on the pixel positions of the checkerboard corners and a preset world coordinate origin, a pinhole camera model is iteratively fitted to obtain the intrinsic parameter matrix of each industrial camera. Then, any industrial camera is selected as the new world coordinate origin to construct a new world coordinate system. Based on the pixel positions of the checkerboard corners and the new world coordinate system, the rotation matrices and translation vectors of the other industrial cameras relative to the new world coordinate origin are calculated to obtain the extrinsic parameter matrix of each industrial camera. The intrinsic and extrinsic parameter matrices of each industrial camera are used as the industrial camera parameters. Video data is simultaneously acquired using three or more industrial cameras while the tester performs a preset test action. Images from each industrial camera at the same moment are extracted from the acquired video data and aggregated to obtain synchronized frame groups. These synchronized frame groups are then aggregated to obtain a synchronized frame sequence.

[0079] Model Construction: Based on the RTMPose model, a motion pose estimation model is constructed, which includes an input layer, a feature extraction layer, an occlusion feature perception layer, a visual feature fusion layer, and an output layer.

[0080] 2D pose labeling: The obtained synchronous frame sequence is input into the motion pose estimation model to obtain a synchronous frame sequence labeled with keypoints; the keypoint labels include keypoint name, keypoint 2D coordinates, confidence score and visibility label.

[0081] 3D Model Reconstruction: The keypoints of images in each synchronized frame group within the labeled synchronized frame sequence are traversed. Images containing keypoints with visibility labels indicating unpredictable locations and keypoint confidence levels below a preset confidence threshold are deleted. The synchronized frame group containing these deleted images becomes the new synchronized frame group. The synchronized frame sequence is updated based on this new synchronized frame group and denoted as the new synchronized frame sequence. Based on the labeled synchronized frame sequence and industrial camera parameters, triangulation is used to calculate the 3D coordinates of each keypoint of the test subject. The calculated 3D coordinates of each keypoint in the current frame group are aggregated to obtain the test subject's skeletal frame group. All the obtained skeletal frame groups are then aggregated to obtain the test subject's skeletal frame sequence.

[0082] Motion parameter extraction: Based on the tester's skeletal frame sequence and the labeled synchronous frame sequence, the tester's kinematic indicators are extracted. The tester's kinematic indicators include heel-to-ground distance, lower limb joint angle, left and right knee coordinate difference, trunk upright angle, left and right shoulder height difference, touch height, spinal stride difference, shoulder and hip rotation angle, spinal plane rotation angle, spinal lateral displacement angle, and elbow joint movement distance.

[0083] Motor function assessment: A motor function assessment model is constructed based on preset assessment rules, including rules for elbow-tuck squats, upper limb flexibility movements, and push-ups. The extracted kinematic indices of the test subject's joints are input into the motor function assessment model. The model sums the test subject's scores according to the preset rules to obtain the test subject's motor function score. Based on the pre-set motor function levels and the test subject's motor function score, the model outputs the motor function level corresponding to the test subject's motor function score.

[0084] This application utilizes multiple industrial cameras to simultaneously acquire video data of test subjects performing preset test actions. It combines the constructed motion posture estimation model to perform key point annotation and two-dimensional posture estimation. Based on the parameters of the industrial cameras, the three-dimensional coordinates of the key points are reconstructed using triangulation, generating a skeletal frame sequence of the test subject. At the same time, multi-dimensional kinematic indicators are extracted from the skeletal frame sequence and input into the evaluation model to output the motor function level. This achieves a scientific quantitative assessment of the test subject's motor ability and provides a reliable basis for motor function analysis, rehabilitation training, and health management.

[0085] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A motion function evaluation system based on multi-view video capture, characterized in that, The system includes: The data acquisition module is used to acquire industrial camera parameters and simultaneously acquire video data of the tester performing preset test actions using multiple industrial cameras. It also extracts images of each industrial camera at the same moment from the acquired video data and aggregates them to obtain each synchronous frame group. By aggregating all synchronous frame groups, a synchronous frame sequence is obtained. The model building module is used to build motion attitude estimation models based on the RTMPose model. The 2D pose labeling module is used to input the obtained synchronous frame sequence into the motion pose estimation model to obtain a synchronous frame sequence labeled with keypoints; the keypoint labels include keypoint name, keypoint 2D coordinates, keypoint confidence and visibility label; The 3D model reconstruction module is used to calculate the three-dimensional coordinates of the test subject's key points using triangulation based on the labeled synchronous frame sequence and industrial camera parameters, and to collect the three-dimensional coordinates of the test subject's key points to obtain the test subject's skeletal frame sequence. The motion parameter extraction module is used to extract the kinematic parameters of the test subject based on the test subject's skeletal frame sequence and the labeled synchronous frame sequence. The motor function assessment module is used to build a motor function assessment model. It inputs the extracted kinematic indicators of the test subject into the motor function assessment model and outputs the test subject's motor function level.

2. The motion function evaluation method based on multi-view video capture according to claim 1, characterized in that, The key points include the nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, right ankle, top of head, neck, center of hip, left big toe, right big toe, left little toe, right little toe, left heel, and right heel.

3. The motion function evaluation method based on multi-view video capture according to claim 1, characterized in that, The data acquisition module also includes a constraint arrangement submodule; The layout constraint submodule is used to constrain the installation position of industrial cameras, including spatial constraint units, field-of-view constraint units, and temporal constraint units; Spatial constraint unit, used to constrain the arrangement of each industrial camera in different positions relative to the tester; Field of view constraint unit, used to constrain each key point of the tester to be within the field of view intersection area of ​​at least two industrial cameras; The time constraint unit is used to ensure that the acquisition time synchronization error of each industrial camera is less than a preset error threshold.

4. The motion function evaluation method based on multi-view video capture according to claim 1, characterized in that, The data acquisition module also includes a device parameter acquisition submodule, which includes an intrinsic parameter acquisition unit and an extrinsic parameter acquisition unit; the industrial camera parameters include a camera intrinsic parameter matrix and a camera extrinsic parameter matrix; The intrinsic parameter acquisition unit is used to capture several chessboard images using each industrial camera, obtain the pixel position of the chessboard corner point in each chessboard image, and iteratively fit the pinhole camera model based on the pixel position of the chessboard corner point in each chessboard image and the preset world coordinate origin to obtain the camera intrinsic parameter matrix of each industrial camera. The extrinsic parameter acquisition unit is used to select any industrial camera as the new world coordinate origin to construct a new world coordinate system. Based on the pixel position of the corner point of the checkerboard in each checkerboard image and the new world coordinate system, it calculates the rotation matrix and translation vector of other industrial cameras relative to the new world coordinate origin to obtain the camera extrinsic parameter matrix of each industrial camera.

5. The motion function evaluation method based on multi-view video capture according to claim 1, characterized in that, The model training module is included after the model building module and before the 2D pose labeling module. The model training module includes a training set construction submodule and a training submodule; A training set submodule is constructed to set up a training video dataset when multiple industrial cameras synchronously acquire training videos of the test subject performing training actions. Training images of each industrial camera at the same moment are extracted from the acquired training video dataset and collected separately to obtain each training synchronous frame group. All training synchronous frame groups are collected to obtain a training synchronous frame sequence. The test subject's key point labels are labeled on each training image in the training synchronous frame sequence. The labeled training synchronous frame sequence is recorded as the training dataset. The training submodule is used to define the loss function, input the training dataset into the motion pose estimation model for iterative training, calculate the model loss based on the loss function during the iterative training process, update the model gradient based on the model loss, and use the gradient descent algorithm to iteratively optimize the model parameters to obtain the trained motion pose estimation model. The trained motion pose estimation model is recorded as the new motion pose estimation model.

6. The motion function evaluation method based on multi-view video capture according to claim 5, characterized in that, The motion pose estimation model includes an input layer, a feature extraction layer, an occlusion feature perception layer, and an output layer; The training submodule specifically includes: inputting the training dataset into the motion pose estimation model from the input layer; the feature extraction layer extracting features from the training dataset, extracting feature maps for each training image in the training dataset; inputting the obtained feature maps into the occlusion feature perception layer for global average pooling to generate channel weights; weighting the channel features of each feature map; performing channel max pooling and average pooling on the channel-weighted feature maps to generate spatial weights; weighting the feature values ​​of the feature maps of each training image at each spatial coordinate position to obtain spatially weighted feature maps; convolving and fusing the channel-weighted feature maps and spatially weighted feature maps; denoting the fused feature map as the occlusion feature perception map; inputting the occlusion feature perception map into the output layer to output the model prediction result; calculating the model loss according to the defined loss function; updating the model gradient based on the model loss; and iteratively optimizing the model parameters using the gradient descent algorithm to obtain the trained motion pose estimation model; and denoting the trained motion pose estimation model as the new motion pose estimation model.

7. The motion function evaluation method based on multi-view video capture according to claim 1, characterized in that, Visibility labels include fully visible labels, location-estimated labels, and location-unestimated labels; the 3D model reconstruction module also includes an image filtering submodule; The image filtering submodule is used to traverse the key points of images in each synchronous frame group in the labeled synchronous frame sequence, delete the images containing key points in the current synchronous frame group whose key point visibility label is a label that cannot be estimated and whose key point confidence is less than a preset confidence threshold, take the synchronous frame group after deleting the images as the new synchronous frame group, update the synchronous frame sequence based on the new synchronous frame group, and denoted as the new synchronous frame sequence.

8. The motion function evaluation method based on multi-view video capture according to claim 1, characterized in that, The kinematic parameters of the test subjects include heel-off distance, lower limb joint angle, difference in left and right knee coordinates, trunk upright angle, difference in left and right shoulder height, touch height, difference in spinal stride, shoulder and hip rotation angle, spinal plane rotation angle, spinal lateral displacement angle, and elbow joint movement distance. The motion parameter extraction module includes: a first extraction submodule, a second extraction submodule, and a third extraction submodule; The first extraction submodule is used to extract the distance of the heel off the ground, the angle of the lower limb joints, the difference in coordinates between the left and right knees, the upright angle of the torso, and the difference in height between the left and right shoulders. The second extraction submodule is used to extract the difference between touch height and spine span. The third extraction submodule is used to extract shoulder and hip rotation angles, spinal plane rotation angles, spinal lateral displacement angles, and elbow joint movement distances.

9. A method for evaluating motion function based on multi-view video capture, characterized in that, The method is applicable to the system as described in any one of claims 1-8, and the method includes: Data acquisition: Obtain industrial camera parameters, and use multiple industrial cameras to synchronously acquire video data when the tester performs preset test actions. Extract the images of each industrial camera at the same moment from the acquired video data and collect them separately to obtain each synchronous frame group. Collect all synchronous frame groups to obtain a synchronous frame sequence. Model Construction: Based on the RTMPose model, a motion attitude estimation model is constructed; 2D pose labeling: The obtained synchronous frame sequence is input into the motion pose estimation model to obtain a synchronous frame sequence labeled with keypoints; the keypoint labels include keypoint name, keypoint 2D coordinates, keypoint confidence and visibility label; 3D model reconstruction: Based on the labeled synchronous frame sequence and industrial camera parameters, the three-dimensional coordinates of the test subject's key points are calculated using triangulation. The three-dimensional coordinates of the test subject's key points are then collected to obtain the test subject's skeletal frame sequence. Motion parameter extraction: Based on the test subject's skeletal frame sequence and labeled synchronous frame sequence, the test subject's kinematic parameters are extracted; Motor function assessment: Construct a motor function assessment model, input the extracted kinematic indicators of the test subjects into the motor function assessment model, and output the test subjects' motor function level.

Images