A method and device for functional assessment of the spine-pelvis-lower limb kinetic chain

Abstract

This application provides a method and device for functional assessment of the spine-pelvis-lower limb kinetic chain, effectively addressing the lack of technical solutions in the field of spinal function, pelvic, and lower limb coordinated function assessment that rely solely on monocular video from a mobile phone and are suitable for outpatient and home settings. The method includes: acquiring video sequences and performing quality screening to obtain target video sequences; performing multi-dimensional processing on the target video sequences to reconstruct the user's three-dimensional reconstruction results and establish dynamic multi-viewpoint attention features; fusing the dynamic multi-viewpoint attention features with the positional features in the three-dimensional reconstruction results to obtain motion analysis features, and performing multi-step motion analysis on preset functional movements to obtain motion analysis results; calculating functional parameters and pain expression scores based on the three-dimensional reconstruction results, obtaining functional analysis results through a functional analysis model, and generating training suggestions for the next stage.

Description

Technical Field

[0001] This application relates to the field of machine vision assessment technology, and more specifically, to a method and apparatus for functional assessment of the spine-pelvis-lower limb kinetic chain. Background Technology

[0002] In the field of spinal function, pelvic and lower limb coordination function assessment, current methods still mainly rely on visual observation of patients' movements by doctors and rehabilitation therapists, questionnaires, and a small number of manual goniometer measurements. These methods are highly subjective, lack repeatability, make it difficult to quantify the dynamic compensatory chain of "lumbar spine-pelvis-hip-knee-ankle," and cannot be continuously tracked outside of outpatient clinics, which seriously restricts the objective assessment of patients' home rehabilitation.

[0003] While existing 3D motion capture systems can provide accurate kinematic parameters, they generally rely on multiple cameras, infrared markers, depth cameras, or laboratory-grade equipment. These systems are complex to deploy, costly, and unsuitable for the high-frequency use required in outpatient settings or for patients to record themselves in their homes. Using only ordinary 2D video, however, lacks reliable depth information, making it impossible to accurately reconstruct key 3D indicators such as pelvic rotation, trunk compensation, and hand-to-foot and hand-to-hip target area proximity distances.

[0004] Therefore, there is an urgent need for a complete technical solution that relies solely on monocular video from a mobile phone, is suitable for outpatient and home settings, and can achieve 3D reconstruction, movement stage understanding, pain expression recognition, spinal-pelvic-lower limb functional assessment, and rehabilitation training recommendations. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a method and device for functional assessment of the spine-pelvis-lower limb kinetic chain, which effectively solves the problem of the lack of technical solutions in the field of assessment of spinal function, pelvic and lower limb coordinated function that rely solely on monocular video from a mobile phone and are suitable for outpatient and home scenarios.

[0006] In a first aspect, embodiments of this application provide a method for functional assessment of the spine-pelvis-lower limb kinetic chain, the method comprising: The system acquires video sequences of users performing preset functional actions, captured and uploaded by a monocular camera on a mobile terminal, and performs quality screening on the video sequences to obtain target video sequences; the preset functional actions are actions that characterize the coordinated function of the spine-pelvis-lower limbs. The target video sequence is processed in multiple dimensions to reconstruct the user's three-dimensional reconstruction result, and dynamic multi-view attention features are established based on the target video sequence and the three-dimensional reconstruction result; The motion analysis features are obtained by fusing the dynamic multi-viewpoint attention features with the positional features in the 3D reconstruction results, and the motion analysis results are obtained by performing multi-step motion analysis on the preset functional motion based on the motion analysis features. Based on the three-dimensional reconstruction results, the functional parameters of the spine-pelvis-lower limb kinetic chain and the pain expression score of the facial region are calculated. The functional parameters, pain expression scores, motion analysis results and preset information are analyzed by the functional analysis model to obtain the functional analysis results and generate training suggestions for the next stage.

[0007] In conjunction with the first aspect, this application provides a first possible implementation of the first aspect, wherein multi-dimensional processing is performed on the target video sequence to reconstruct the user's three-dimensional reconstruction result, including: The target video sequence is subjected to human detection, human segmentation, face detection, target region detection and initial localization of two-dimensional human key points to obtain positional features including head and face, trunk, pelvis, upper limbs, lower limbs and action target region; Based on the location features and the depth map and camera relative pose obtained from the target video sequence, a 3D reconstruction is performed to obtain the user's 3D reconstruction result.

[0008] In conjunction with the first aspect, this application provides a second possible implementation of the first aspect, wherein three-dimensional reconstruction based on the location features and the depth map and camera relative pose obtained based on the target video sequence includes: Based on the target frame and adjacent source frames, monocular depth estimation and camera relative pose estimation are performed to obtain a depth map and camera relative pose. The depth map and the relative pose of the camera are reprojected onto the target frame according to the camera model to optimize the depth map and the relative pose of the camera.

[0009] In conjunction with the first aspect, this application provides a third possible implementation of the first aspect, wherein functional parameters of the spine-pelvis-lower limb kinetic chain and pain expression scores for the facial region are calculated based on the three-dimensional reconstruction results, including: Calculate the lumbar flexion / extension surrogate angle, lumbar lateral flexion surrogate angle, and lumbar rotation surrogate angle based on the relative rotation of the thoracic or trunk coordinate system with respect to the pelvic coordinate system. The pelvic anteroposterior tilt angle, pelvic elevation tilt angle, and pelvic rotation angle are calculated based on the orientation of the pelvic coordinate system relative to the world coordinate system. The activity range is calculated based on the difference between the maximum and minimum values ​​of the corresponding angle variable within the action window; Calculate left-right symmetry based on the difference in the range of motion on the left and right sides; Terminal reachability is calculated based on the distance between the three-dimensional points of the hand and the three-dimensional points of the target area of ​​the preset functional action. The degree of compensation is calculated based on the ratio of the weighted range of motion of the compensating segment to the weighted range of motion of the main exerting segment, and the stability is calculated based on the dispersion of the whole body center of mass trajectory, the pelvic center trajectory, or the projection trajectory of the supporting base.

[0010] In conjunction with the first aspect, this application provides a fourth possible implementation of the first aspect, wherein calculating the functional parameters of the spine-pelvis-lower limb kinetic chain and generating pain expression scores for the facial region based on the three-dimensional reconstruction results, further comprising: Pain recognition is performed on the facial region of the 3D reconstruction results to obtain the initial pain probability of the user when performing preset function actions; A pain expression score is obtained by integrating the proximity between the preset functional action and the target area of ​​the task, the current stage of the action, the protective behavior of the action, and the initial pain probability.

[0011] In conjunction with the first aspect, this application provides a fifth possible implementation of the first aspect, wherein obtaining functional analysis results and generating training suggestions for the next stage includes: The user status is constructed based on the user's functional analysis results and historical training records, and various training schemes including multiple training actions are matched based on the user status. The various training movements in each training program are evaluated to obtain the final recommended program, which will serve as a training suggestion for the next stage.

[0012] In conjunction with the first aspect, this application provides a sixth possible implementation of the first aspect, wherein, based on the motion analysis features, a multi-step motion analysis of the preset functional action is performed to obtain the motion analysis result, including: The stage memory state is updated based on the intermediate results of the current action stage; different preset function actions correspond to different action stages; The stage probability is output based on the temporal state and the stage memory state, and the optimal stage path that satisfies the stage sequence constraint is determined by the preset stage transition constraint. Based on the optimal stage path, the preset functional actions are analyzed to obtain action analysis results including the action stage sequence and the start and end times, duration, stage completion degree, and stage constraint information of each stage.

[0013] Secondly, embodiments of this application provide a functional assessment device for the spine-pelvis-lower limb kinetic chain, the device comprising: The acquisition module is used to acquire video sequences of users performing preset functional actions captured and uploaded by the monocular camera of the mobile terminal, and to perform quality screening on the video sequences to obtain target video sequences; the preset functional actions are actions that characterize the coordinated function of the spine-pelvis-lower limbs. The processing module is used to perform multi-dimensional processing on the target video sequence to reconstruct the user's three-dimensional reconstruction result, and to establish dynamic multi-view attention features based on the target video sequence and the three-dimensional reconstruction result; The fusion module is used to fuse the dynamic multi-viewpoint attention features with the position features in the 3D reconstruction results to obtain motion analysis features, and to perform multi-step motion parsing on the preset functional motion based on the motion analysis features to obtain motion analysis results; The analysis module is used to calculate the functional parameters of the spine-pelvis-lower limb kinetic chain and generate the pain expression score of the facial region based on the three-dimensional reconstruction results. It then analyzes the functional parameters, pain expression score, motion analysis results, and preset information through a functional analysis model to obtain the functional analysis results and generate training suggestions for the next stage.

[0014] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of any of the functional assessment methods for the spine-pelvis-lower limb kinetic chain described in the present application.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of any of the methods for functional assessment of the spine-pelvis-lower limb kinetic chain described in the present application.

[0016] This application provides a method for functional assessment of the spine-pelvis-lower limb kinetic chain. The method first acquires a video sequence of a user performing preset functional actions, captured and uploaded by a monocular camera on a mobile terminal. The video sequence is then quality-screened to obtain a target video sequence. The preset functional actions represent the coordinated function of the spine-pelvis-lower limbs. Next, the target video sequence undergoes multi-dimensional processing to reconstruct the user's three-dimensional reconstruction. Dynamic multi-viewpoint attention features are then established based on the target video sequence and the three-dimensional reconstruction results. Next, the dynamic multi-viewpoint attention features are fused with the positional features in the three-dimensional reconstruction results to obtain action analysis features. Based on these action analysis features, the preset functional actions are analyzed using a multi-step action analysis method to obtain action analysis results. Finally, functional parameters of the spine-pelvis-lower limb kinetic chain and a pain expression score for the facial region are calculated based on the three-dimensional reconstruction results. The functional parameters, pain expression score, action analysis results, and preset information are analyzed using a functional analysis model to obtain functional analysis results and generate training suggestions for the next stage. Based on the above methods, this application not only enables functional assessment of the spine-pelvis-lower limb kinetic chain in low-cost deployment scenarios such as outpatient clinics and home settings, but also ensures the accuracy and effectiveness of user assessment based on the calculated functional parameters of the spine-pelvis-lower limb kinetic chain and the generated pain expression scores of the facial region, meeting the clinical needs for collaborative assessment of the spine-pelvis-lower limb; it also automatically outputs training suggestions for the next stage, facilitating user training or doctor assessment. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a functional assessment method for the spine-pelvis-lower limb kinetic chain provided in an embodiment of this application is shown. Figure 2 A schematic diagram of the process for obtaining three-dimensional reconstruction results provided in an embodiment of this application is shown; Figure 3 A schematic diagram illustrating the process for obtaining next-stage training suggestions provided in an embodiment of this application is shown; Figure 4 This illustration shows a structural block diagram of a functional assessment device for the spine-pelvis-lower limb kinetic chain provided in an embodiment of this application; Figure 5 A structural block diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0020] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0022] There is an urgent need for a complete technical solution that relies solely on monocular video from a mobile phone, is suitable for outpatient and home settings, and can achieve 3D reconstruction, movement stage understanding, pain expression recognition, spinal-pelvic-lower limb functional assessment, and rehabilitation training recommendations.

[0023] Based on this, the present application provides a method and apparatus for functional assessment of the spine-pelvis-lower limb kinetic chain, which will be described below through embodiments.

[0024] Example 1 To facilitate understanding of this embodiment, a functional assessment method for the spine-pelvis-lower limb kinetic chain disclosed in this application will first be described in detail. For example... Figure 1 The diagram shows a flowchart of a functional assessment method for the spine-pelvis-lower limb kinetic chain. This application provides a functional assessment method for the spine-pelvis-lower limb kinetic chain, the method comprising: S101. Acquire a video sequence of a user performing a preset functional action, which is captured and uploaded by the monocular camera of the mobile terminal, and perform quality screening on the video sequence to obtain a target video sequence; the preset functional action is an action that characterizes the coordinated function of the spine-pelvis-lower limb. S102. Perform multi-dimensional processing on the target video sequence to reconstruct the user's three-dimensional reconstruction result, and establish dynamic multi-view attention features based on the target video sequence and the three-dimensional reconstruction result; S103. The motion analysis features are obtained by fusing the dynamic multi-viewpoint attention features with the position features in the three-dimensional reconstruction results, and the motion analysis results are obtained by performing multi-step motion analysis on the preset functional motion based on the motion analysis features. S104. Based on the three-dimensional reconstruction results, calculate the functional parameters of the spine-pelvis-lower limb kinetic chain and generate the pain expression score of the facial region. Analyze the functional parameters, pain expression score, motion analysis results, and preset information through a functional analysis model to obtain the functional analysis results and generate training suggestions for the next stage.

[0025] The method provided in this application is based on an evaluation system, which includes a video acquisition module, an image quality control module, a monocular depth and pose estimation module, a three-dimensional human body reconstruction module, a dynamic multi-viewpoint attention module, a motion routine analysis module, a pain expression analysis module, a functional scoring module, a rehabilitation recommendation module, and a report generation module.

[0026] In step S101, this application is implemented based on a video acquisition module and an image quality control module. In scenarios where a user is undergoing home assessment or a doctor is in an outpatient setting, the user's or doctor's mobile terminal is used to capture or collect videos of the user performing preset functional actions targeting the spine-pelvis-lower limb kinetic chain. This results in a video sequence of the user performing the preset functional actions, captured and uploaded by the monocular camera of the mobile terminal. The capture or collection must meet the following requirements: resolution not less than 720p; frame rate between 25fps and 60fps; recording duration between 5s and 20s; distance between the camera and the user between 1m and 3m; shooting angle including at least one of lateral, oblique, and posterior oblique views; and the video must simultaneously include the head, face, trunk, pelvis, and target lower limb. The preset functional actions are actions characterizing the coordinated function of the spine-pelvis-lower limb, including but not limited to putting on socks and wiping the buttocks backward. The video sequence is then quality-screened to obtain a target video sequence, specifically by extracting frames from the video sequence to obtain an image sequence. ,in I t ∈ R H×W×3t represents the frame number; H represents the image height; W represents the image width; 3 represents the RGB three channels, used to evaluate the image sequence. I t Image quality is assessed by: determining image blur based on sharpness evaluation metrics; determining the coverage and occlusion ratio of key anatomical regions based on human segmentation results; determining lighting conditions based on exposure and brightness distribution; and determining the integrity of action phases based on preset action scripts and video temporal features. Key regions refer to the head and face, torso, pelvis, target lower limbs, or target contact area. If any of the following—blur, occlusion ratio, lighting conditions, or key region integrity—is substandard, the image sequence... I t If the corresponding video sequence is of substandard quality, a reshoot is prompted; if the quality is satisfactory, the image sequence... I t Entering the 3D modeling stage.

[0027] The sock-wearing action is used to expose and assess: lumbar flexion ability; pelvic tilt and rotation control; coordination of hip flexion, external rotation, and knee flexion; single-leg support and center of gravity stability; hand-foot end-point accessibility; protective movements and facial expressions of pain during pain induction; its routine can be defined as: ready position, target foot location, trunk flexion, hand-foot approach, end-point contact / simulated slip-in return. The hand-wiping-buttocks action is mainly used to expose and assess: lumbar rotation, lateral flexion, and flexion-extension combination ability; pelvic rotation and trunk coupling; shoulder extension, internal rotation, and scapular girdle compensation; hand-buttock end-point accessibility; pain avoidance, shortened movements, and facial expressions of pain; its routine can be defined as: ready position, posterior target area location, trunk rotation / lateral flexion, shoulder extension and internal rotation, end-point contact / reciprocating simulation, return. The method of defining the motion mileage for both actions is to transfer the idea of ​​"comprising complex tasks with sequential sub-operations and requiring the result of the previous step to the next step" to the spinal function assessment scenario.

[0028] In step S102, this application is implemented based on a monocular depth and pose estimation module, a 3D human body reconstruction module, and a dynamic multi-view attention module. After obtaining the target video sequence, multi-dimensional processing is performed, specifically including human body detection, human body segmentation, face detection, and initial localization of key points throughout the body. Depending on the type of mobile terminal, different methods are used for 3D reconstruction to reconstruct the user's 3D reconstruction result. The 3D reconstruction result includes 3D key points. A 3D skeleton and / or 3D human body mesh were constructed and 3D scale restoration was performed. Since the truly diagnostically valuable regions in the target video sequence typically occupy only a small proportion of the entire frame, such as the face, lumbosacral region, pelvis, hip, knee, ankle, hand-foot proximity area, and posterior buttock target area, this invention preferably does not perform uniform processing on the entire frame, but instead employs a dynamic multi-viewpoint attention feature mechanism. This mechanism, consisting of GlimpseNetwork, Recurrent Network, Emission Network, and Q-Network working collaboratively, selects the most valuable local regions through multi-viewpoint, recurrent memory, and gated fusion. Specifically: Let the... t Frame prediction M One focus area: L t = { l t,m}, m = 1, 2, …, M The m-th center of interest is given by the launch network: l t,m = tanh( W m h t-1 + b m ), m = 1, 2, …, M ; Normalized coordinates l t,m = ( l t,m x , l t,m y Mapped to image pixel coordinates: u t,m = ( W - 1)( l t,m x + 1) / 2; v t,m = ( H - 1)( l t,m y + 1) / 2; exist( u t,m , v t,m Local blocks are extracted from the vicinity using a spatial transformer: Gt,m = ST( I t , u t,m , v t,m , s m ); Local features are extracted using a convolutional network: f t,m =CNN( G t,m ) ; in, W m , b m These are the weights and biases of the m-th branch of the transmission network, respectively. h t 1 represents the temporal hidden state of the previous time step; W and H are the image width and height, respectively; ST(·) is the spatial transformer operator; s m Let be the scale parameter of the m-th glimpse; I t Let t be the image of frame t; CNN(·) is a convolutional network for local feature extraction. Based on the above formula, dynamic multi-viewpoint attention features are established using the target video sequence and the 3D reconstruction results, including local spatiotemporal features corresponding to multiple viewpoints such as face, lumbosacral region, pelvis, hip, knee, ankle, hand, foot, and the task's final reach area.

[0029] In the specific implementation of step S102, one embodiment is as follows: Figure 2 As shown, the target video sequence undergoes multi-dimensional processing to reconstruct the user's 3D reconstruction result, including: S1021. Perform human body detection, human body segmentation, face detection, target region detection and initial localization of two-dimensional human key points on the target video sequence to obtain positional features including head and face, trunk, pelvis, upper limbs, lower limbs and action target region. S1022. Based on the location features and the depth map and camera relative pose obtained from the target video sequence, perform three-dimensional reconstruction to obtain the user's three-dimensional reconstruction result.

[0030] In step S1021, for the target frame in the target video sequence, this application first extracts the multi-scale image features of the target frame through a pre-trained convolutional neural network; then, based on the multi-scale image features, it performs human body detection, face detection, and action target region detection in parallel.

[0031] The process involves three main branches: a human detection branch performs human detection, outputting the user's bounding box and detection confidence score; a face detection branch performs face detection, regressing the face bounding box, facial key points, and detection confidence score within the area defined by the human bounding box; and a motion target region detection branch detects target regions related to the current preset function action, outputting the bounding box, category label, and detection confidence score of these target regions. These target regions include, but are not limited to, the target foot region, the target buttocks region, the chair surface region, the buttocks contact region, or other regions related to the final touch action. This allows for the preliminary determination of the head and face region, torso region, and motion target region within a two-dimensional image plane.

[0032] Furthermore, the target frame is cropped, scaled, and normalized based on the human body bounding box, and the processed image is input into a human image segmentation network to obtain a pixel-level human foreground mask. This human foreground mask is used to distinguish the human foreground and background regions and provides constraints for subsequent human body part segmentation and key point localization. Based on the human foreground mask, combined with prior knowledge of human anatomy and the geometric proportions of the human body, the human foreground region is divided into vertical and horizontal partitions to obtain candidate regions for the head and face, torso, pelvis, upper limbs, and lower limbs.

[0033] Furthermore, the candidate regions obtained from the human foreground region segmentation are input into a two-dimensional pose estimation network to obtain the two-dimensional coordinates and corresponding confidence scores of no less than 17 skeletal keypoints, including the eyes, nose, ears, shoulders, elbows, wrists, hips, knees, and ankles. Based on the two-dimensional coordinates, confidence scores, and anatomical connections between the skeletal keypoints, the pelvic projection position, spinal tilt angle, and related joint angles are calculated. The candidate regions obtained based on human geometric proportions are then corrected using the geometric connections between the keypoints, thereby obtaining the corrected head and face, trunk, pelvis, upper limbs, and lower limb regions, as well as the corresponding skeletal keypoint sequences.

[0034] For preset functional actions such as "wiping the buttocks backward" and "putting on socks," the proximity distance between the hand and the target area is calculated by combining the detection results of the target area and the temporal trajectories of key points on the hand and foot. Specifically, when the preset functional action is wiping the buttocks backward, the proximity distance between the key points on the hand and the target area behind the buttocks or the contact area of ​​the buttocks is calculated; when the preset functional action is putting on socks, the proximity distance between the key points on the hand and the target foot area is calculated. To improve the positioning stability of the target area in consecutive frames, the key point trajectory and the target area position can be compensated based on the optical flow information between adjacent frames. When the target area is occluded by the hand, torso, or other body parts, the position of the occluded target area in the current frame can be continuously estimated by combining the target area position in historical frames, the key point motion trajectory, and the prior knowledge of the left-right symmetry of the human body.

[0035] Furthermore, the human body bounding box, face bounding box, facial key points, human foreground mask, skeletal key points, action target region bounding box, and corresponding confidence scores are uniformly mapped back to the original frame coordinate system according to the inverse transformation relationship of image cropping, scaling, and normalization. Then, the above information is encapsulated according to the timestamp to form the positional features. The positional features at least include the pixel coordinates, region mask, key point coordinates, and corresponding confidence scores of the head and face, torso, pelvis, upper limbs, lower limbs, and action target region in the original frame coordinate system. Thus, the positional features can provide a unified two-dimensional spatial constraint for subsequent monocular depth estimation, camera relative pose estimation, 3D reconstruction, and functional evaluation.

[0036] In step S1022, after obtaining the positional features, this application performs monocular depth estimation and camera relative pose estimation based on the target video sequence to obtain the corresponding depth map and camera relative pose. Combining the positional features, the depth map, and the camera relative pose, the two-dimensional human keypoints, human body parts, and action target areas are back-projected into three-dimensional space to reconstruct the user's three-dimensional keypoints, three-dimensional skeleton, and / or three-dimensional human body mesh, serving as the user's three-dimensional reconstruction result. This three-dimensional reconstruction result is further used for subsequent dynamic multi-view attention feature establishment, action phase analysis, and calculation of functional parameters of the spine-pelvis-lower limb kinetic chain.

[0037] In a specific implementation of step S1022, one embodiment involves performing three-dimensional reconstruction based on the location features, the depth map obtained from the target video sequence, and the camera's relative pose, including: S10221. Perform monocular depth estimation and camera relative pose estimation based on the target frame and adjacent source frames to obtain a depth map and camera relative pose. S10222, The adjacent source frames are reprojected onto the target frame according to the camera model to optimize the depth map and the relative pose of the camera.

[0038] In steps S10221-S10222, this application employs self-supervised monocular depth estimation based on the principle that if the depth map of the target frame and the relative camera pose estimation between the target frame and adjacent frames are correct, then the target frame should be reconstructed from adjacent frames after projection, and the reconstruction error should be as small as possible. Specifically, this employs a differentiable DIBR approach, photometric reconstruction error, smoothing constraints, and camera model-driven training. The camera model described in this application is either a pinhole camera model or a unified omnidirectional camera model. The monocular depth estimation and relative camera pose estimation are implemented through trained depth and pose models, with the trained models using photometric reconstruction consistency and edge-aware depth smoothing constraints between the target frame and adjacent source frames as optimization criteria.

[0039] This application uses differentiable reprojection to reproject the depth map and camera relative pose to the target frame according to the camera model, thereby optimizing the depth map and camera relative pose. Let the target frame be... I t Adjacent source frames are I t' ,in t '∈{ t -1, t +1}, the depth map of the target frame output by the deep network is D t The pose network outputs a relative pose estimate as follows: T t→t' The target frame obtained by reprojecting the source frame is: I t'→t = I t '[proj( D t , T t→t' , i intr )]; In the formula: proj(·) is the projection function based on the selected camera model; i intr This is the camera intrinsic parameter vector; I t'→t This represents the target frame obtained by reprojecting the source frame.

[0040] If the target video sequence is obtained from a regular mobile phone's main camera, a pinhole camera model can be used, let the first... t The coordinates of a 3D point in the frame in the camera coordinate system are: xt = ( x t , y t , z t ) T The homogeneous coordinates of the corresponding pixel are represented as: p t = ( u t , v t , 1) T = (1 / z t ) Kx t ; In the formula: p t These are the homogeneous coordinates of pixels; u t , v t ) represents the image plane coordinates; K This is the camera intrinsic parameter matrix; z t This represents the depth value.

[0041] To ensure compatibility with wide-angle lenses, fisheye lenses, and wide field-of-view shooting on mobile phones, this application supports a unified omnidirectional camera model, with an auxiliary translation vector: t ξ = (0, 0, x ) T Then, the representation of a three-dimensional point in the auxiliary coordinate system is: x t c = x t + t ξ · ‖ x t ‖2; By normalizing the 3D points and projecting them onto the image plane, we can obtain: p t = 1 / ( z t + x || x t ||2) · K ( x t + tξ || x t ||2); in, x t This indicates the current position of the target's 3D point in the camera coordinate system. z t This indicates the depth of the three-dimensional point. K Represents the camera intrinsic parameter matrix. x To unify the parameters of the omnidirectional camera model, t ξ = (0, 0, x ) T ;|| x t ||2 represents the L2 norm of a 3D point; the meanings of the other symbols are the same as before. The above describes the unified omnidirectional camera modeling and the projection / re-projection logic extended therefrom.

[0042] The total reprojection loss caused by differentiable reprojection is defined as: L reproj = S t'∈N(t) pe( I t , I t'→t ); In the formula, the photometric error function pe(·) is defined as: pe( I a , I b ) = α / 2·(1-SSIM( I a , I b ))+(1- α )|| I a - I b ||1; Where N(t) is the set of neighboring source frames of target frame t, SSIM(·) is the structural similarity index; ||·||1 is L 1-norm; α ∈ [0, 1] are weight parameters.

[0043] To avoid drastic oscillations in the depth map in flat areas, this application introduces an edge-aware smoothing term: L smooth = | x dt * | e -| x It| + | y d t * | e -| y It| ; In the formula, d t * Normalized inverse depth; x , y These are the difference operators in the x and y directions, respectively. Therefore, the total depth loss can be written as: L depth = l r L reproj + l s L smooth ; In the formula, l r Weights for reprojection loss; l s Weights are used to smooth out the loss.

[0044] In the 3D keypoint recovery process, this application sets the first... t The first frame j The two-dimensional key points are p j,t Its three-dimensional point representation in the camera coordinate system is as follows: X j,t = unproj( p j,t , D t ( p j,t ), i intr ) ; In the formula: unproj(·) is a function that obtains a 3D point by backprojection from pixel coordinates and depth values; D t ( p j,t ) represents the depth value at the key point. X j,t for X j,t homogeneous coordinates; j,t The world coordinates are not restored to their absolute scale. During world coordinate transformation, let the cumulative pose be... T t Then the location of the key point in the indeterminate scale world coordinate system is: W j,t = T t -1 X j,t ; In the formula: X j,t for X j,t homogeneous coordinates; W j,t This refers to world coordinates where the absolute scale has not been recovered. Due to scale ambiguity in monocular reconstruction, this application recovers the scale using any of the following methods: using a reference object of known size; using user-inputted height; using anthropometric priors, such as shoulder width, foot length, and pelvic width; or using the ground plane and the gravity direction of the mobile phone IMU. For example, when the reference length is known... L ref When the scaling factor is used, it can be defined as: s scale = L ref / || W a,t0 - W b,t0 ||2; The final scaled and aligned 3D points are: W j,t = s scale R g W j,t ; In the formula: R g A rotation matrix aligned with the direction of gravity; a , b The two endpoints of the reference line segment; t0 is the reference time. To mitigate the accumulation of pose errors in long videos, this application divides the complete target video sequence into multiple short segments, estimates the depth and pose of each segment separately, performs cross-frame matching and bundle adjustment to uniformly optimize camera pose and spatial point positions, and finally merges the reconstruction results of each segment to obtain the final 3D reconstruction result. Let the optimized i-th segment be... j The world points are w j , No. t The extrinsic parameters of the frame camera are ( R t , t t If the bundle adjustment is then written as: argmin {wj},{Rt},{tt} S t,j r ( || h ( x t,j ) - h ( R t w j + t t ) ||2) ; In the formula: w j Indicates the first j A spatial point, R t and t t They represent the first t Rotation and translation in the extrinsic parameters of the frame camera r t,j Indicates by the first t The line-of-sight direction after backprojection and normalization of the frame observation points. h ( x )= x / ‖ x ‖2 indicates that the predicted 3D vector is normalized to the unit sphere. r (·) denotes the robust loss function. To mitigate scale instability, camera pose and spatial points can be optimized alternately, rather than being optimized completely synchronously. These are the key technical aspects of this application that avoid the vulnerability that "monocular video can only obtain relative depth and cannot be used for functional quantization."

[0045] In step S103, this application, based on a dynamic multi-view attention module and an action routine parsing module, implements action analysis features by fusing the dynamic multi-view attention features with the positional features in the 3D reconstruction results. Specifically, it fuses the local spatiotemporal features of multiple viewpoints, such as the face, lumbosacral region, pelvis, hip, knee, ankle, hand, foot, and task endpoint with their positional features in the 3D reconstruction results to obtain the action analysis features. The local spatiotemporal features include local content features extracted from corresponding local image patches and the temporal variation features of these local content features. Specifically, this includes the previously hidden state... h t-1 As the query vector, calculate the weights of each local block: q t = W q h t-1 ; k t,m = W k f t,m ; v t,m f = W v f t,m ; Attention weights are: α t,m = exp( q t T k t,m / sqrt( d k )) / S r=1 M exp( q t T k t,r / sqrt( d k )); The context vector is: c t = S m=1 M α t,m vt,m f ; To simultaneously utilize both the content representation and the corresponding location features in "local spatiotemporal features," the content gating variable and the location gating variable are defined as follows: r t c = s ( W c c t + b c ); r t l = s ( W l vec( L t ) + b l ); By fusing the content-gated and position-gated variables, the resulting gating fusion is as follows: z t = [ r t c ⊙ c t ; r t l ⊙vec( L t ) ]; The final timing state is updated as follows: h t = LSTM( z t , h t-1 ); in, W q , W k , W v , W c , W l The weight matrix is ​​a learnable weight matrix; b c , b l For learnable bias; ST(·) is the space transformer; s(·) represents the Sigmoid activation function; ⊙ represents element-wise multiplication; vec( L t This indicates that multiple viewpoint positions are flattened into vectors; d k为注意力维度 LSTM(·) is a Long Short-Term Memory network unit; M is the number of attention centers, which can be 4 to 24. Larger glimpses usually carry more local context, while too many or too few glimpses are not necessarily better. Therefore, this application seeks a balance between "coverage" and "local context capacity". This application also performs multi-step action parsing on the preset functional actions based on the action analysis features to obtain action analysis results.

[0046] In a specific implementation of step S103, one embodiment is as follows: Based on the action analysis features, a multi-step action parsing is performed on the preset functional action to obtain the action analysis result, including: S1031. Update the stage memory state based on the intermediate results of the current action stage; different preset function actions correspond to different action stages; S1032. Output the stage probability based on the temporal state and the stage memory state, and determine the optimal stage path that satisfies the stage order constraint with the preset stage transition constraint. S1033. Based on the optimal stage path, analyze the preset functional actions to obtain action analysis results including the action stage sequence and the start and end times, duration, stage completion degree and stage limitation information of each stage.

[0047] In the steps S 1031- S In 1033, this application understands a preset functional action as consisting of several sequential sub-stages, and passes the result of the previous stage to the next stage through intermediate memory. Let the first... t The enhanced activity graph or intermediate result generated in the current stage of the frame is A t Then, the stage memory state is defined as: B t = or · B t 1+ ( 1 or ) · A t ; In the formula: B t For intermediate stage memory, A t This represents the enhanced activity graph or intermediate result generated in the current stage. orThe forgetting coefficient is ∈ [0,1]. The stage probability is determined by both the temporal state and the stage memory state: π t = softmax( W s h t + U s B t + b s ) ; The current stage is tagged as: t = argmax k π t,k ; To prevent confusion in the stage sequence, a stage transition constraint matrix is ​​introduced. M stage Solve for the optimal stage path: 1:T = argmax S1:T [ S t=1 T log π t,St + S t=2 T log M stage ( S t-1 , S t ) ] ; In the formula: π t,St Indicates the first t The probability that a moment belongs to a certain stage. M stage (·, ·) denotes the stage transition constraint matrix. 1:T This represents the optimal sequence of stages for the entire action routine. W s , U s The weight matrix is ​​a learnable matrix. b s This is for bias. For example, for the action of putting on socks, the set of stages can be defined as: S sock= {Preparation, Target Foot Positioning, Trunk Flexion, Hand-Foot Approach, Terminal Contact / Loop Simulation, Return}; For backward wiping motions, the phase set can be defined as: S wipe = {Preparation, Rear target area localization, Trunk rotation and lateral flexion, Shoulder extension and internal rotation, Terminal contact / wiping simulation, Return}. The motion analysis results include the sequence of motion phases, as well as the start and end times, duration, completion rate, and limitations of each phase. Based on the above, this application not only determines "what action the user performed," but also "which phase was reached," "which phase experienced limitations," "which phase involved compensation," and "which phase was accompanied by pain."

[0048] In step S104, this application is implemented based on a pain expression analysis module, a functional scoring module, a rehabilitation recommendation module, and a report generation module. Based on the three-dimensional reconstruction results, functional parameters of the spine-pelvis-lower limb kinetic chain and a pain expression score for the facial region are calculated. The pain expression score is generated when the user completes a preset functional movement. The functional parameters, pain expression score, movement analysis results, and preset information are analyzed through a functional analysis model to obtain a functional analysis result for the user's spine-pelvis-lower limb kinetic chain. Specifically, this application integrates task completion, range of motion, coordination, symmetry, stability, and pain manifestation to form a comprehensive spinal function score, where the range of motion score is defined as: S ROM =(1 / | Q primary |) S q∈Qprimary clip(ROM q / ROM q ref , 0, 1); The asymmetric penalty term is defined as: S asym = (1 / | Q pairs |) S q∈Qpairs A q ; Coordination score is defined as: S coord = exp(-| C comp - m comp | / ( s comp + e )); The stability score is defined as: Sstab = exp( - s COM / s ref ); in, Q pairs ROM is a set of segments that have left-right pairing relationships. q ref For reference only; m comp , s comp This is a normal co-reference statistic; s COM The dispersion of the center of gravity trajectory within the action window; s ref As a reference stability threshold; clip( x , 0, 1) indicates that x The interval is truncated to [0, 1]. Therefore, the comprehensive spinal function score is defined as: ; in, It's the task completion rate. It is the activity range score. It's a score for coordination. It is an asymmetrical punishment. It is a stability score, and Based on this, the functional state classification network outputs: = softmax( W cls [ z motion ; z face ; z hist ] + b cls ); in: z motion These are the kinematic eigenvectors; z face This represents the facial pain feature vector. z hist For historical follow-up feature vectors; W cls This is the classification layer weight matrix; b clsThis is a classification layer bias. The preset information refers to the user's historical follow-up characteristics, i.e., the characteristics generated by the user during multiple historical follow-ups. Examples include functional parameters and pain expression scores. The functional analysis results include lumbar flexion limitation type, pelvic compensation-dominant type, hip dominance deficiency type, lower limb asymmetry type, pain-protective type, comprehensive limitation type, and essentially normal type, generating next-stage training suggestions. Based on these next-stage training suggestions, the user can proceed with the next stage of training. An interpretable report is generated based on the report generation module for explanation by doctors or users.

[0049] In a specific implementation of step S104, one embodiment involves calculating the functional parameters of the spine-pelvis-lower limb kinetic chain and generating pain expression scores for the facial region based on the three-dimensional reconstruction results, including: S10411. Calculate the lumbar flexion / extension surrogate angle, lumbar lateral flexion surrogate angle, and lumbar rotation surrogate angle based on the relative rotation of the thoracic or trunk coordinate system with respect to the pelvic coordinate system. S10412. Calculate the pelvic anteroposterior tilt angle, pelvic elevation tilt angle, and pelvic rotation angle based on the orientation of the pelvic coordinate system relative to the world coordinate system. S10413. Calculate the activity range based on the difference between the maximum and minimum values ​​of the corresponding angle variables in the action window; S10414. Calculate left-right symmetry based on the difference in the range of motion on the left and right sides; S10415. Calculate terminal reachability based on the distance between the three-dimensional points of the hand and the three-dimensional points of the task target area of ​​the preset functional action. S10416, and calculate the degree of compensation based on the ratio of the weighted range of motion of the compensating segment to the weighted range of motion of the main exerting segment, and calculate the stability based on the dispersion of the whole body center of mass trajectory, the pelvic center trajectory or the projection trajectory of the supporting base.

[0050] In steps S10411-S10416, this application sets R pelvis ( t ) is the rotation matrix of the pelvic coordinate system relative to the world coordinate system. R thorax ( t If is the rotation matrix of the thorax / trunk coordinate system, then the rotation matrix of the trunk relative to the pelvis is: R rel ( t ) = R pelvis ( t ) T R thorax ( t ) ; By performing Euler angle decomposition, the lumbar spine functional surrogate angle can be obtained: ( i lumbar flex ( t ), i lumbar lat ( t ), i lumbar rot ( t ))= EulerXYZ( R rel ( t )); in, i lumbar flex ( t () indicates the lumbar flexion / extension synergistic angle; i lumbar lat ( t () indicates the lateral flexion angle of the lumbar spine; i lumbar rot ( t The ) represents the lumbar spine rotational proxy angle, which is calculated based on the above formula to obtain the lumbar spine flexion / extension proxy angle, lumbar spine lateral flexion proxy angle, and lumbar spine rotational proxy angle.

[0051] This application sets the attitude angle of the pelvis relative to the world coordinate system as: ( β pelvis tilt ( t ), β pelvis obl ( t ), β pelvis rot ( t )) = EulerXYZ( R world T R pelvis ( t )) ; in, β pelvis tilt ( t () represents the anteroposterior tilt angle of the pelvis; β pelvis obl ( t ( ) represents the pelvic tilt angle; β pelvisrot ( t ( ) represents the pelvic rotation angle. R world The rotation matrix is ​​the world coordinate system. Based on the above formula, the pelvic anteroposterior tilt angle, pelvic vertical tilt angle, and pelvic rotation angle are calculated. This application defines arbitrary joint angle or segment angle variables. q ( t In the Actions window T task The scope of activities within is: ROM q = max_( t ∈ T task ) q ( t ) - min_( t ∈ T task ) q ( t ) ; This application defines the left-right symmetry index as: A q = |ROM q,L - ROM q,R | / (ROM q,L + ROM q,R + e ) ; Among them, ROM q,L The left-side activity range; ROM q,R The right-side activity range; e To prevent extremely small constants with a denominator of zero, the left-right symmetry is calculated based on the above formula.

[0052] This application calculates final reachability based on the following formula: For the action of putting on socks, the target point is defined. W target sock ( t For a backward wiping motion, define the target area as the ankle or forefoot. W target wipe ( t The target area is the rear hip region, and the corresponding approach distance is defined as follows: d sock ( t ) = || W hand ( t ) - W target sock( t )||2; d wipe ( t ) = || W hand ( t ) - W target wipe ( t )||2; Task completion rate is defined as: C task = 1{ min_( t ∈ T task ) d task ( t ) ≤ d task}; in, d task ( t () indicates the proximity of the current task; d task is the final reach threshold; 1{·} is the indicator function; W hand ( t ) is the first t The world coordinates of the three-dimensional points of the hand in the frame. To distinguish between "normal coordination" and "completing actions with the help of other segments," this application defines the compensation ratio as follows: C comp = ( S _( q ∈ Q comp ) r q ROM q ) / ( S _( r ∈ Q primary ) r r ROM r + e ); in, Q primary This refers to the set of segments that the action should primarily accomplish, such as lumbar flexion, hip flexion, and knee flexion that may be involved in putting on socks. Q comp Compensatory segmental aggregates; such as pelvic rotation, excessive lateral flexion of the trunk, abnormal support of the contralateral lower limb, shoulder girdle compensation, etc. r q , r r As weight, e To prevent extremely small constants with a denominator of zero, this index enables the system to distinguish between two situations: "the action is completed but the compensation is large" and "the action has truly returned to normal."

[0053] In a specific implementation of step S10416, one embodiment includes: calculating the functional parameters of the spine-pelvis-lower limb kinetic chain and generating pain expression scores for the facial region based on the three-dimensional reconstruction results, and further including: A1. Pain recognition is performed on the facial area of ​​the 3D reconstruction results to obtain the initial pain probability of the user when performing preset function actions; A2. By integrating the proximity between the preset functional actions and the target area of ​​the task, the current stage of the action, the protective behavior of the action, and the initial pain probability, a pain expression score is obtained.

[0054] In steps A1-A2, this application analyzes the movement process of performing a preset functional action, and also analyzes the user's pain expressions during the key stages of performing the preset functional action. Because many users can "barely complete" the action, but exhibit obvious eyebrow drooping, eyelid closure, mouth twitching, head retraction, or sudden interruption of the action in the final approach stage, pain recognition is performed on the facial area of ​​the 3D reconstruction result to obtain the initial pain probability of the user when performing the preset functional action. Let the first... t Frame facial region image I t face The initial probability of pain output by the facial network is: P t pain = Softmax( f face ( I t face )) pain ; To highlight pain information at the end of a pre-defined functional action, the weights of proximity between the hand and the target area, the current stage of the action, and protective behaviors are calculated based on the following formula: oh t = exp( - d task ( t ) 2 / s d 2 ) ; The overall pain score is defined as follows: S pain = ( S t oh t P t pain ) / ( S t oh t ); in, d task ( t The distance between the current action and the target area is the approximation distance, where the target area is the standard action range of the current preset function. P t pain This represents the initial probability of pain output by the facial network. s d This is the distance attenuation parameter; S pain The higher the score, the more pronounced the facial pain experienced by the user during the critical phase of the movement. Its significance is clear: users often exhibit brow drooping, eye closure, mouth twitching, sudden withdrawal, or protective deceleration at the moment closest to the end of the movement, when pain is most likely to be induced. Information from this moment is more valuable than that from the preparation or return phases. This application can further integrate protective indicators of the movement, such as sudden deceleration in the terminal phase, movement reversal, abnormally shortened reach distance, and abnormal support on the non-task side, to further ensure the accuracy of the obtained pain expression score.

[0055] In the specific implementation of step S104, another embodiment exists as follows: Figure 3 As shown, the functional analysis results are obtained, and suggestions for the next stage of training are generated, including: S10421. Construct user status based on user function analysis results and historical training records, and match various training schemes including multiple training actions based on the user status. S10422. Evaluate the various training movements in each training program to obtain the final recommended program, which will serve as a training suggestion for the next stage.

[0056] In steps S10421-S10422, this application assumes that after the nth evaluation, the user's status is: ; in, S func , n This is the total score for this function; It is the functional category obtained from the nth evaluation. It is a detailed feature vector. If it is a historical evaluation and training record, then it is based on the user state. s n Match each candidate training program with a variety of training movements; This application defines the set of safety actions as: A safe ( s n ) = { a ∈A|P risk ( s n , a )≤ρand P _(pain↑)( s n , a ) ≤ π max}; In the formula: A is the set of all candidate training actions; P risk ( s n , a ) for in state s n Take action below a Predicted safety risks; P _( pain↑ )( s n , a () represents the corresponding predicted risk value for increased pain. r max , π max These are the safety risk threshold and the pain aggravation threshold, respectively. This application only allows selection from training programs where both the safety risk of the safe action and the pain aggravation risk do not exceed the thresholds. This application also defines the single-step reward as: r n = l 1( S func , n +1 - S func , n ) - l 2max(0, S pain , n +1 - S pain , n ) - l 3Risk n - l 4NonAdherence n ; Among them, Risk n Indicates training risk; Nonadherence n This indicates a punishment for insufficient compliance; l 1 to l If 4 is the weight, then the optimal action value satisfies the Bellman equation: Q * ( s n , a n ) = E [ r n + c max a' Q * ( s n+1 , a ') ]; Preferably, a Dueling structure is used to estimate the value of actions: Q ( s , a ) = V ( s ) + ( A ( s , a ) - (1 / | A |) S a'∈A A ( s , a ') ); The final recommended solution is: ; in, c ∈ [0, 1] is the discount factor; V(s) is the state value function; A(s, a) is the action advantage function; is the set of safe actions allowed within the risk and pain aggravation thresholds, and the training actions. a n *The following parameter combinations can be used: movement type, training difficulty, number of repetitions, number of sets, hold duration, rhythm, left / right side, whether auxiliary support is needed, whether offline follow-up is recommended, and whether further imaging examinations are recommended. When generating training recommendations for the next stage, this application is not entirely black-box; it employs a two-layer architecture of "rule constraints and value assessment network": the first layer excludes high-risk movements based on medical safety rules; the second layer selects the training program with the highest expected benefit from the safety set. This approach reflects individualization while improving clinical acceptability.

[0057] This application transforms ordinary outpatient / home videos into 3D motion data through mobile phone monocular video capture, self-supervised depth estimation, camera pose estimation, and 3D reconstruction. This addresses the problem of existing technologies being "only capable of qualitative observation, difficult to quantify in 3D," eliminating the need for multi-camera motion capture systems, infrared markers, and dedicated depth cameras. This significantly reduces outpatient deployment costs and is suitable for users to independently capture data at home using standard motion scripts. Furthermore, it recovers the 3D relationships of user movements from ordinary RGB video, quantifying the spatial relationships between target areas such as the spine, pelvis, hip, knee, ankle, hands and feet, and hands and buttocks, overcoming the limitation of 2D video in directly describing depth and compensation chains. By employing dynamic multi-view attention, instead of uniformly processing the entire frame, it adaptively focuses on a small number of high-value regions, thus exhibiting stronger robustness to background clutter, occlusion, and lighting changes in home environments. Multiple glimpses select key regions instead of processing the entire frame, improving efficiency and stability in complex visual environments.

[0058] This application, through multi-step action routine analysis, upgrades actions such as putting on socks and wiping backwards from single classifications to process modeling based on "stage sequence, intermediate memory, and information transmission," solving the problem that existing technologies cannot identify "which step is limited, which step is compensated for, and which step is painful." It identifies specific limiting points "from preparation to final contact," thus better aligning with the logic of clinical functional assessment. The mechanism of sequential execution of subtasks and transmission of intermediate results demonstrates the rationality of this staged modeling; it not only considers whether the task is completed but also identifies situations of "superficial completion but functional recovery" through compensation ratio, pelvic posture changes, asymmetry, and pain scores, thus being closer to real clinical needs than a simple completion / incompleteness judgment.

[0059] This application also utilizes dynamic multi-viewpoint attention, temporal cyclic memory, gating fusion, and value function recommendations based on safety constraints to focus on the face, lower back, pelvis, hip, knee, ankle, and end-contact areas in complex backgrounds, occlusion, and home environments. This further forms a closed-loop training recommendation system, addressing the disconnect between assessment and intervention in existing technologies. Simultaneous analysis of facial pain expressions and kinematic characteristics distinguishes between different functional states such as "motor limitation," "pain protection," and "compensatory," improving the interpretability of results. It automatically recommends the next stage of training movements, intensity, sets, rhythm, and risk warnings, thus forming a closed loop of "assessment-intervention-reassessment," rather than a one-time static report. Because this invention uses a unified action script, unified three-dimensional indicators, and a unified scoring system, it facilitates longitudinal comparisons of results from different dates and training stages, and can be used for outpatient follow-up and home rehabilitation follow-up.

[0060] The following are examples of this application in actual use: (1) Assessment of sock-wearing action in outpatient setting: The user is guided by medical staff in the outpatient clinic and a 10-second video is taken from an oblique position using a mobile phone. The system detects that the user completes the action routine of "preparation - target foot positioning - trunk flexion - hand-foot approach - terminal contact - return". First, it completes monocular depth estimation, scale recovery and 3D key point reconstruction; then it extracts attention features of the face, back, pelvis, hip, knee and ankle, and hand-foot approach areas; then it completes action stage recognition; it also calculates the lumbar flexion proxy angle, pelvic rotation angle, hip flexion range, knee flexion range, terminal approach distance and compensation ratio; it also calculates the pain expression score; and it integrates the functional score and functional state category. If the result shows: S func Low, S pain medium to high C comp The results were elevated and classified as "pelvic compensation-dominant type"; further training suggestions for the next stage were generated, such as: pelvic neutral position control training; hip-dominant hinge training; lumbar low-load flexion tolerance training; single-leg support stability training; and reducing end-position intensity to avoid a rapid increase in pain. (2) Evaluation of backward wiping motion in a home setting: Users record and upload videos of backward wiping motions at home using the app as prompted. After automatic quality inspection, the same 3D reconstruction, attention feature extraction, and motion phase analysis are performed. If the results show: insufficient trunk rotation angle; excessive pelvic rotation; insufficient shoulder extension and internal rotation; terminal contact distance... d wipe ( tIf the threshold is not reached; the facial pain score increases in the terminal stage; it can be judged as "restricted posterior access with pain protection type", and the following are recommended: thoracolumbar rotation control training; shoulder extension / internal rotation range of motion training; scapular girdle stabilization training; posterior access graded exposure training; when the pain continues to increase, offline follow-up visit is recommended.

[0061] Example 2 This application also provides a functional assessment device for the spine-pelvis-lower limb kinetic chain, such as Figure 4 The diagram shows a block diagram of a functional assessment device for the spine-pelvis-lower limb kinetic chain. This device performs functions corresponding to the steps of the aforementioned method for performing a functional assessment of the spine-pelvis-lower limb kinetic chain on a terminal device. The device can be understood as a server component including a processor. The functional assessment device for the spine-pelvis-lower limb kinetic chain described in this application includes: The acquisition module is used to acquire video sequences of users performing preset functional actions captured and uploaded by the monocular camera of the mobile terminal, and to perform quality screening on the video sequences to obtain target video sequences; the preset functional actions are actions that characterize the coordinated function of the spine-pelvis-lower limbs. The processing module is used to perform multi-dimensional processing on the target video sequence to reconstruct the user's three-dimensional reconstruction result, and to establish dynamic multi-view attention features based on the target video sequence and the three-dimensional reconstruction result; The fusion module is used to fuse the dynamic multi-viewpoint attention features with the position features in the 3D reconstruction results to obtain motion analysis features, and to perform multi-step motion parsing on the preset functional motion based on the motion analysis features to obtain motion analysis results; The analysis module is used to calculate the functional parameters of the spine-pelvis-lower limb kinetic chain and generate the pain expression score of the facial region based on the three-dimensional reconstruction results. It then analyzes the functional parameters, pain expression score, motion analysis results, and preset information through a functional analysis model to obtain the functional analysis results and generate training suggestions for the next stage.

[0062] In one feasible implementation, the processing module includes: The target video sequence is subjected to human detection, human segmentation, face detection, target region detection and initial localization of two-dimensional human key points to obtain positional features including head and face, trunk, pelvis, upper limbs, lower limbs and action target region; Based on the location features and the depth map and camera relative pose obtained from the target video sequence, a 3D reconstruction is performed to obtain the user's 3D reconstruction result.

[0063] In one feasible implementation, the processing module further includes: Based on the target frame and adjacent source frames, monocular depth estimation and camera relative pose estimation are performed to obtain a depth map and camera relative pose. The depth map and the relative pose of the camera are reprojected onto the target frame according to the camera model to optimize the depth map and the relative pose of the camera.

[0064] In one feasible implementation, the analysis module includes: Calculate the lumbar flexion / extension surrogate angle, lumbar lateral flexion surrogate angle, and lumbar rotation surrogate angle based on the relative rotation of the thoracic or trunk coordinate system with respect to the pelvic coordinate system. The pelvic anteroposterior tilt angle, pelvic elevation tilt angle, and pelvic rotation angle are calculated based on the orientation of the pelvic coordinate system relative to the world coordinate system. The activity range is calculated based on the difference between the maximum and minimum values ​​of the corresponding angle variable within the action window; Calculate left-right symmetry based on the difference in the range of motion on the left and right sides; Terminal reachability is calculated based on the distance between the three-dimensional points of the hand and the three-dimensional points of the target area of ​​the preset functional action. The degree of compensation is calculated based on the ratio of the weighted range of motion of the compensating segment to the weighted range of motion of the main exerting segment, and the stability is calculated based on the dispersion of the whole body center of mass trajectory, the pelvic center trajectory, or the projection trajectory of the supporting base.

[0065] In one feasible implementation, the analysis module further includes: Pain recognition is performed on the facial region of the 3D reconstruction results to obtain the initial pain probability of the user when performing preset function actions; A pain expression score is obtained by integrating the proximity between the preset functional action and the target area of ​​the task, the current stage of the action, the protective behavior of the action, and the initial pain probability.

[0066] In one feasible implementation, the analysis module also includes: The user status is constructed based on the user's functional analysis results and historical training records, and various training schemes including multiple training actions are matched based on the user status. The various training movements in each training program are evaluated to obtain the final recommended program, which will serve as a training suggestion for the next stage.

[0067] In one feasible implementation, the fusion module includes: The stage memory state is updated based on the intermediate results of the current action stage; different preset function actions correspond to different action stages; The stage probability is output based on the temporal state and the stage memory state, and the optimal stage path that satisfies the stage sequence constraint is determined by the preset stage transition constraint. Based on the optimal stage path, the preset functional actions are analyzed to obtain action analysis results including the action stage sequence and the start and end times, duration, stage completion degree, and stage constraint information of each stage.

[0068] Example 3 This application also provides an electronic device, such as Figure 5 As shown, it includes: a processor 501, a memory 502, and a bus 503. The memory 502 stores machine-readable instructions that can be executed by the processor 501. When the electronic device is running, the processor 501 and the memory 502 communicate through the bus 503. When the machine-readable instructions are executed by the processor 501, the steps of any of the functional assessment methods of the spine-pelvis-lower limb kinetic chain described above are executed.

[0069] Example 4 This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of any of the methods for functional assessment of the spine-pelvis-lower limb kinetic chain described in this application.

[0070] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0071] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0072] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0073] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a platform server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0074] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for functional assessment of the spine-pelvis-lower limb kinetic chain, characterized in that, The method includes: The system acquires video sequences of users performing preset functional actions, captured and uploaded by a monocular camera on a mobile terminal, and performs quality screening on the video sequences to obtain target video sequences; the preset functional actions are actions that characterize the coordinated function of the spine-pelvis-lower limbs. The target video sequence is processed in multiple dimensions to reconstruct the user's three-dimensional reconstruction result, and dynamic multi-view attention features are established based on the target video sequence and the three-dimensional reconstruction result; The motion analysis features are obtained by fusing the dynamic multi-viewpoint attention features with the positional features in the 3D reconstruction results, and the motion analysis results are obtained by performing multi-step motion analysis on the preset functional motion based on the motion analysis features. Based on the three-dimensional reconstruction results, the functional parameters of the spine-pelvis-lower limb kinetic chain and the pain expression score of the facial region are calculated. The functional parameters, pain expression scores, motion analysis results and preset information are analyzed by the functional analysis model to obtain the functional analysis results and generate training suggestions for the next stage.

2. The method according to claim 1, characterized in that, The target video sequence is processed in multiple dimensions to reconstruct the user's three-dimensional reconstruction result, including: The target video sequence is subjected to human detection, human segmentation, face detection, target region detection and initial localization of two-dimensional human key points to obtain positional features including head and face, trunk, pelvis, upper limbs, lower limbs and action target region; Based on the location features and the depth map and camera relative pose obtained from the target video sequence, a 3D reconstruction is performed to obtain the user's 3D reconstruction result.

3. The method according to claim 2, characterized in that, Three-dimensional reconstruction is performed based on the location features, the depth map obtained from the target video sequence, and the camera relative pose, including: Based on the target frame and adjacent source frames, monocular depth estimation and camera relative pose estimation are performed to obtain a depth map and camera relative pose. The depth map and the relative pose of the camera are reprojected onto the target frame according to the camera model to optimize the depth map and the relative pose of the camera.

4. The method according to claim 1, characterized in that, Based on the three-dimensional reconstruction results, the functional parameters of the spine-pelvis-lower limb kinetic chain and the pain expression scores of the facial region were calculated, including: Calculate the lumbar flexion / extension surrogate angle, lumbar lateral flexion surrogate angle, and lumbar rotation surrogate angle based on the relative rotation of the thoracic or trunk coordinate system with respect to the pelvic coordinate system. The pelvic anteroposterior tilt angle, pelvic elevation tilt angle, and pelvic rotation angle are calculated based on the orientation of the pelvic coordinate system relative to the world coordinate system. The activity range is calculated based on the difference between the maximum and minimum values ​​of the corresponding angle variable within the action window; Calculate left-right symmetry based on the difference in the range of motion on the left and right sides; Terminal reachability is calculated based on the distance between the three-dimensional points of the hand and the three-dimensional points of the target area of ​​the preset functional action. The degree of compensation is calculated based on the ratio of the weighted range of motion of the compensating segment to the weighted range of motion of the main exerting segment, and the stability is calculated based on the dispersion of the whole body center of mass trajectory, the pelvic center trajectory, or the projection trajectory of the supporting base.

5. The method according to claim 4, characterized in that, Based on the three-dimensional reconstruction results, the functional parameters of the spine-pelvis-lower limb kinetic chain and the pain expression scores of the facial region are calculated, and the calculation also includes: Pain recognition is performed on the facial region of the 3D reconstruction results to obtain the initial pain probability of the user when performing preset function actions; A pain expression score is obtained by integrating the proximity between the preset functional action and the target area of ​​the task, the current stage of the action, the protective behavior of the action, and the initial pain probability.

6. The method according to claim 1, characterized in that, The functional analysis results are obtained, and suggestions for the next stage of training are generated, including: The user status is constructed based on the user's functional analysis results and historical training records, and various training schemes including multiple training actions are matched based on the user status. The various training movements in each training program are evaluated to obtain the final recommended program, which will serve as a training suggestion for the next stage.

7. The method according to claim 1, characterized in that, Based on the motion analysis features, the preset functional motion is subjected to multi-step motion parsing to obtain motion analysis results, including: The stage memory state is updated based on the intermediate results of the current action stage; different preset function actions correspond to different action stages; The stage probability is output based on the temporal state and the stage memory state, and the optimal stage path that satisfies the stage sequence constraint is determined by the preset stage transition constraint. Based on the optimal stage path, the preset functional actions are analyzed to obtain action analysis results including the action stage sequence and the start and end times, duration, stage completion degree, and stage constraint information of each stage.

8. A functional assessment device for the spine-pelvis-lower limb kinetic chain, characterized in that, The device includes: The acquisition module is used to acquire video sequences of users performing preset functional actions captured and uploaded by the monocular camera of the mobile terminal, and to perform quality screening on the video sequences to obtain target video sequences; the preset functional actions are actions that characterize the coordinated function of the spine-pelvis-lower limbs. The processing module is used to perform multi-dimensional processing on the target video sequence to reconstruct the user's three-dimensional reconstruction result, and to establish dynamic multi-view attention features based on the target video sequence and the three-dimensional reconstruction result; The fusion module is used to fuse the dynamic multi-viewpoint attention features with the position features in the 3D reconstruction results to obtain motion analysis features, and to perform multi-step motion parsing on the preset functional motion based on the motion analysis features to obtain motion analysis results; The analysis module is used to calculate the functional parameters of the spine-pelvis-lower limb kinetic chain and generate the pain expression score of the facial region based on the three-dimensional reconstruction results. It then analyzes the functional parameters, pain expression score, motion analysis results, and preset information through a functional analysis model to obtain the functional analysis results and generate training suggestions for the next stage.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of a functional assessment method for the spine-pelvis-lower limb kinetic chain as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of a method for functional assessment of the spine-pelvis-lower limb kinetic chain as described in any one of claims 1 to 7.

Images