Motion analysis method and system based on image and sole contact modeling

By collecting image data from multiple cameras to reconstruct a personalized bone and plantar contact model, and combining it with a unified pressure model to calculate GRF, the problem of expensive and poor portability of traditional equipment is solved, and high-precision GRF estimation and COP trajectory calculation are achieved, which is suitable for biomechanical and clinical applications.

CN120766340AActive Publication Date: 2025-10-10FIRST HOSPITAL AFFILIATED TO GENERAL HOSPITAL OF PLA
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202510659283.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-21
Publication Date
2025-10-10
Estimated Expiration
2045-05-21

AI Technical Summary

Technical Problem

Existing technologies have difficulty estimating ground reaction forces (GRFs) with high precision in open fields and uncalibrated environments. Traditional equipment is expensive and has poor portability, and image-driven methods lack the ability to model individual contact.

Method used

Multi-view image data is collected by multiple cameras to reconstruct a personalized bone model and plantar contact model. The GRF is calculated by combining the unified pressure model. The k-means++ clustering algorithm and deep learning algorithm are used to divide the plantar contact sphere and calculate the COP trajectory.

Benefits of technology

High-precision individualized modeling is achieved, the accuracy and robustness of GRF solutions are improved, and COP trajectories and torques can be accurately calculated under different motion types and individual conditions, making it suitable for biomechanical and clinical applications.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120766340A_ABST
    Figure CN120766340A_ABST
Patent Text Reader

Abstract

The invention provides a motion analysis method and system based on image and sole contact modeling, and the method comprises the steps: reconstructing a three-dimensional posture of the motion of a subject according to recognized 2D key points, and generating a personalized skeleton model matched with the subject according to the height data of the subject; the method comprises the following steps: acquiring a pelma image of a subject, automatically generating a plurality of contact balls in a pelma region according to an extracted pelma contour shape to approximately represent each region where the pelma is in contact with the ground, and optimally setting the positions and radiuses of the contact balls according to the foot type of the subject to form a personalized pelma contact model; according to the personalized skeleton model, the joint movement data and the personalized plantar contact model, solving a ground reaction force GRF which enables dynamics to be consistent under a given kinematics condition; determining a moving path of a resultant force acting point of stress along with time according to the stress condition of each contact ball of the two feet, and calculating a track of a plantar pressure center COP. Motion analysis can be carried out based on image and plantar contact modeling.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of motion analysis, and in particular to a motion analysis method and system based on image and sole contact modeling. Background Art

[0002] Ground reaction force (GRF) is one of the core dynamic parameters that describes the interaction between the human body and the ground during movement. It is widely used in a variety of fields, including sports biomechanics analysis, rehabilitation assessment, gait recognition, and smart wearable device control. Traditional methods for acquiring GRF rely on devices such as force plates or pressure sensing mats. Although these devices provide high data accuracy, they are expensive, require stringent deployment conditions, lack portability, and can only be used in laboratory environments. They are unable to meet the long-term, low-cost, and open-space requirements of real-world scenarios.

[0003] With the development of computer vision and human posture recognition technologies, researchers have begun to use cameras to capture images of people in motion. Through posture estimation and model solving, they can indirectly estimate the GRF during motion, achieving "sensorless" and "forceless platform" ground reaction force acquisition. This method combines the advantages of visual perception and physical modeling, has good generalization and scalability, and has become a hot topic in recent years at the intersection of biomechanics and artificial intelligence research.

[0004] However, current image-driven GRF estimation methods either rely on black-box models or lack the ability to model individual contact, making it difficult to meet the demand for high-precision GRF estimation in open fields and uncalibrated environments. Summary of the Invention

[0005] In order to solve the technical problems existing in the above-mentioned prior art, the present invention provides a motion analysis method and system based on image and sole contact modeling. The technical solution is as follows:

[0006] In one aspect, a motion analysis method based on image and plantar contact modeling is provided, the method comprising:

[0007] S1, using multiple cameras to collect multi-view image data of the subject's movement;

[0008] S2. Perform human posture recognition using the collected multi-view image data. Reconstruct the subject's 3D posture based on the identified 2D key points. Based on the subject's height data, scale the standard skeleton model to generate a personalized skeleton model that matches the subject. Output the subject's joint positions and posture sequence throughout the entire movement process, as well as the scaled personalized skeleton model.

[0009] S3. Obtain an image of the subject's sole, extract the sole contour through image recognition, and automatically generate multiple contact spheres in the sole area based on the extracted sole contour shape to approximate the various areas where the sole contacts the ground. The positions and radii of these contact spheres are optimized based on the subject's foot shape to form a personalized sole contact model, and the center position C of the contact sphere is output. i and radius R i ;

[0010] S4. Solving the ground reaction force GRF that makes the dynamics consistent under given kinematic conditions using a unified pressure model based on the personalized skeletal model, joint motion data, and personalized plantar contact model;

[0011] S5. Conduct in-depth analysis of the ground reaction force (GRF) data, determine the temporal movement path of the resultant force point based on the force conditions of each contact ball of the two feet, and calculate the trajectory of the center of pressure (COP) of the plantar surface.

[0012] Optionally, the S3 specifically includes:

[0013] S31, plantar contour extraction;

[0014] Obtain the subject's sole image, extract the sole contour using the OpenCV Canny algorithm, and output the sequence of upper edge points of the sole contour. And a binary mask representing the contact and non-contact areas of the sole of the foot with the ground

[0015] S32, gridding;

[0016] Mask on soles Perform grid processing to generate a two-dimensional grid (x i , z j );

[0017] S33, use k-means++ clustering algorithm to select N balls The center position C of the initial contact ball i ;

[0018] S34. Calculate the radius of the contact ball;

[0019] For each cluster center C i , calculate its maximum circumscribed circle, the contact sphere radius R i Calculation formula:

[0020]

[0021] S35, adaptively adjusting the size and position of the contact ball;

[0022] Calculate the total area of ​​all contacting balls:

[0023] Make sure the total area is close to the actual area of ​​the sole contact area A contact If it does not match, adjust the radius R i :

[0024]

[0025] If the radius of some balls exceeds R max , then split it into two smaller balls and update the number of balls;

[0026] S36, output contact ball distribution;

[0027] The final contact ball center position C i and radius R i Output as the contact ball parameters in subsequent simulation calculations, and for each contact ball C i , assigning a stiffness k to each contact ball based on the material properties and physical model of the area i and damping ε i .

[0028] Optionally, the S33 specifically includes:

[0029] S331, sampling effective pixels;

[0030] Collect all pixel coordinates in the white mask representing the contact area of ​​the sole into a point set: P = {(x, z)};

[0031] S332, determining the first sphere center;

[0032] Randomly select a point from P as the first sphere center C1;

[0033] S333, select new ball centers in sequence;

[0034] For each point p in P, calculate its distance D(p) from the nearest selected center, and press D(p) 2 Do probability sampling, the point drawn is the new center, until N is selected balls indivual;

[0035] S334, fine-tune the sphere centers through classic k-means so that the sphere centers are located at the geometric center of gravity of each cluster;

[0036] Steps: clustering → mean → update center, cycle until convergence.

[0037] Optionally, the S4 specifically includes:

[0038] S41, kinematic analysis;

[0039] Input: frame number N;

[0040] calculate:

[0041] Initial inclination angle: The initial inclination angle θ0 is obtained by preset or initial inclination angle acquisition algorithm, which represents the angle at which the sole tilts forward and upward when the heel just touches the ground;

[0042] Maximum indentation: d max ≈5~mm, which indicates the maximum deformation or compression depth that may occur in the plantar soft tissue during contact with the ground, and is preset to 5mm;

[0043] Output: θ k =θ0(1-t k )(k=1...N),d k =d max t k ;

[0044] S42, calculation of indentation displacement;

[0045] Input: Sole mask θ k ;d k ;

[0046] calculate:

[0047] Depression field:

[0048] Output: Depression field δ k ;

[0049] S43, calculation of indentation velocity;

[0050] Input: δ k , δ k-1 ;

[0051] calculate:

[0052] Indentation speed:

[0053] Output: Indentation velocity

[0054] S44, pressure calculation;

[0055] Input: Subject mass m, subject's vertical acceleration a y , gravitational acceleration g, equivalent stiffness coefficient k, equivalent damping coefficient c, exponents p, q, indentation field δ k , indentation speed Sole mask

[0056] calculate:

[0057] Pressure distribution:

[0058] Output: pressure field p k (x, z), the pressure field p k The integral of (x, z) on the contact surface ∫∫ A p k (x, z) dxdz represents the ground reaction force GRF;

[0059] Among them, p k (x, z): instantaneous pressure at the grid (x, z) / Pa; m: subject mass / kg; a y :Vertical acceleration of the subject's center of gravity / m·s -2 ; g: gravitational acceleration; k: equivalent stiffness coefficient / N·m -p ; c: equivalent damping coefficient / N·s·m -(q+1) ; p, q: index; δ k (x, z): indentation displacement / m; Indentation speed / m·s -1 ; Sole mask, contact = 1, non-contact = 0; A c : current frame contact area; t k : Current frame time ratio; θ k : current frame inclination; d k : Current frame depression depth; A: grid area / m 2 .

[0060] Optionally, the S5 specifically includes:

[0061] Input: pressure field p k ;

[0062] calculate:

[0063] COP coordinates:

[0064] Output: COP trajectory (x cop,k , z cop,k );

[0065] Among them, x cop,k :x coordinate of COP in the current frame / m;z cop,k : z coordinate of COP in the current frame / m.

[0066] Optionally, the obtaining of the initial tilt angle θ0 by using an initial tilt angle obtaining algorithm specifically includes:

[0067] Step 1: Extract 2D foot features using key point method or mask method;

[0068] Among them, the key point method: detects multiple 2D foot points; the mask method: obtains the binary mask of the sole through semantic segmentation, makes a minimum bounding rectangle for the mask, and extracts the four corner points of the minimum bounding rectangle as feature points;

[0069] Step 2: Multi-view geometry 3D reconstruction

[0070] Multiple views of the same physical point 2D coordinates (u, v) c , use the known camera transformation matrix to do DLT / triangulation to obtain the 3D point P i =(x i ,y i , z i ); If mask corners are used, use the known camera transformation matrix to perform DLT / triangulation to triangulate the four corner points of the mask circumscribed rectangle in each frame;

[0071] Step 3: sole plane fitting;

[0072] Input: ≥3 3D points;

[0073] RANSAC Iterations:

[0074] Randomly select 3 points to fit a plane → find the plane equation n·(Xd)=0, where n=(n x , n y , n z ) is the normal vector of the plane, X = (x, y, z) represents any point in space, d is a known point on the plane, and "·" represents the dot product operation;

[0075] Count all the 3D points in the sole mask area that fall near the current plane. These points are called inliers. If the distance between these inliers and the fitting plane is within the set error threshold, they are considered to conform to the model.

[0076] Repeat the iteration multiple times, fitting a candidate plane each time, recording the number of its corresponding inliers, and finally selecting the set of plane parameters with the largest number of inliers as the best fitting result;

[0077] Output the unit normal vector n of the optimal plane;

[0078] Step 4: Calculate the initial inclination angle θ0

[0079] Set ground normal Indicates that the Y axis is vertically upward, and the unit normal vector n of the sole should be close to it;

[0080] Initial inclination angle = the supplementary angle of the two normal angles:

[0081] The smaller the initial inclination angle, the closer the sole is to standing on flat ground.

[0082] In another aspect, a motion analysis system based on image and sole contact modeling is provided, the system comprising:

[0083] An acquisition module, configured to acquire multi-view image data of a subject's motion using multiple cameras;

[0084] The first generation module is used to perform human posture recognition using the collected multi-view image data, reconstruct the subject's 3D posture based on the identified 2D key points, and scale the standard skeleton model based on the subject's height data to generate a personalized skeleton model that matches the subject. The module then outputs the subject's joint positions and posture sequence throughout the entire movement process, as well as the scaled personalized skeleton model.

[0085] The second generation module is used to obtain the subject's sole image, extract the sole contour through image recognition, and automatically generate multiple contact spheres in the sole area according to the extracted sole contour shape to approximate the various areas where the sole contacts the ground. The positions and radii of these contact spheres are optimized according to the subject's foot shape to form a personalized sole contact model and output the center position C of the contact sphere. i and radius R i ;

[0086] A solving module, configured to solve the ground reaction force GRF that makes the dynamics consistent under given kinematic conditions using a unified pressure model based on the personalized skeletal model, the joint motion data and the personalized plantar contact model;

[0087] The calculation module is used to conduct in-depth analysis of the ground reaction force GRF data, determine the movement path of the resultant force point over time based on the force conditions of each contact ball of the two feet, and calculate the trajectory of the center of pressure COP of the plantar.

[0088] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the above-mentioned motion analysis method based on image and sole contact modeling.

[0089] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the above-mentioned motion analysis method based on image and sole contact modeling.

[0090] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0091] The present invention has achieved significant improvements in individual modeling accuracy, GRF solution, and COP calculation accuracy, as shown in the following aspects:

[0092] 1) Higher individualized modeling capability: The present application automatically generates a plantar contact sphere model driven by images, enabling personalized modeling for different subjects' plantar shape and anatomical features. The plantar model of the present application can be adjusted according to individual foot type differences, greatly improving the model's adaptability to individual differences and ensuring that the contact model fits each subject more accurately.

[0093] 2) Improved GRF solving accuracy and robustness: The method of the present application does not rely on empirical fitting of specific data sets, but calculates GRF through a unified pressure model, thus maintaining high prediction accuracy in different motion types and individual cases. This means that regardless of the subject's body type, gait, or the action performed, the present application can robustly provide accurate GRF solving results, with better universality.

[0094] 3) More accurate contact point representation and COP calculation: Due to the use of a refined, multi-contact point plantar sphere division model, the ground reaction force acting area is more accurately represented, with each contact sphere only being subjected to force within its plantar region. This makes the GRF action point calculation (COP) more accurate, and the present application can more realistically reflect the position changes of the plantar pressure center, thus improving the inference accuracy of key indicators such as COP.

[0095] 4) High-precision dynamic indicator inference capability: With accurate individualized models and unified pressure model calculations, the present application can not only obtain three-dimensional GRF time curves, but also infer high-precision dynamic indicators such as torque (M y ) around the vertical axis. The results provided by the present application are closer to the true values in terms of force size, direction, and action point, making the calculated COP trajectory, torque M y , and other indicators more reliable. This high-precision dynamic analysis is of great significance for clinical gait analysis, motion injury assessment, and other applications, demonstrating the significant superiority of the present application in practical applications.

[0096] In summary, by integrating multi-camera pose recognition, individualized plantar contact geometry modeling, and solving based on a unified pressure model, the present application realizes high-precision estimation of ground reaction force, overcoming the shortcomings of simple plantar contact models and poor individual adaptability in existing technologies, and can provide more accurate and reliable analysis tools for biomechanics research and clinical applications, with significant benefits. BRIEF DESCRIPTION OF DRAWINGS

[0097] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0098] Figure 1 This is a flow chart of a motion analysis method based on image and sole contact modeling provided by an embodiment of the present invention;

[0099] Figure 2 This is an overall block diagram of a motion analysis method based on image and sole contact modeling provided by an embodiment of the present invention;

[0100] Figure 3 This is a diagram illustrating the arrangement of acquisition equipment for a motion analysis method based on image and sole contact modeling provided by an embodiment of the present invention;

[0101] Figure 4 This is a schematic diagram of the division of the sole area contacting the ball provided by an embodiment of the present invention;

[0102] Figure 5 This is a block diagram of a motion analysis system based on image and sole contact modeling provided by an embodiment of the present invention;

[0103] Figure 6 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0104] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0105] An embodiment of the present invention provides a motion analysis method based on image and sole contact modeling. The method can be implemented by an electronic device, which can be a terminal or a server. Figure 1 The flow chart of this method is shown below. Figure 2 The overall block diagram of the method is as follows:

[0106] S1, using multiple cameras to collect multi-view image data of the subject's movement;

[0107] The embodiment of the present invention collects image data of the subject's movement process from multiple perspectives, using three cameras (such as the attached Figure 3 (as shown), synchronously capturing multi-view videos or image sequences of the subject's movements. The coordinated acquisition of three cameras provides complete information about the subject's motion and appearance, providing data support for subsequent posture recognition and modeling.

[0108] S2. Perform human posture recognition using the collected multi-view image data. Reconstruct the subject's 3D posture based on the identified 2D key points. Based on the subject's height data, scale the standard skeleton model to generate a personalized skeleton model that matches the subject. Output the subject's joint positions and posture sequence throughout the entire movement process, as well as the scaled personalized skeleton model.

[0109] The embodiment of the present invention can use the HRNet deep learning algorithm to recognize human posture and identify multiple 2D key points (including shoulders, knees, hip joints, elbow joints, etc.). The specific recognition process is existing technology and will not be repeated here.

[0110] The embodiment of the present invention can combine the 2D key points of multiple cameras to reconstruct 3D key points using triangulation, camera calibration, and weighted least squares method. The specific reconstruction process is existing technology and will not be repeated here.

[0111] S3. Obtain an image of the subject's sole, extract the sole contour through image recognition, and automatically generate multiple contact spheres in the sole area based on the extracted sole contour shape to approximate the various areas where the sole contacts the ground (such as the heel, inner forefoot, outer forefoot, toes, etc., see Appendix Figure 4 ), the position and radius of these contact balls are optimized according to the subject's foot shape to form a personalized plantar contact model, and the center position C of the contact ball is output. i and radius R i ;

[0112] The embodiment of the present invention realizes individualized reconstruction of plantar contact geometry through image-driven approach, so that each subject has a contact model that matches the shape of his or her foot. The generated personalized plantar contact model will serve as the basis for the subsequent calculation of foot-ground contact force.

[0113] Optionally, the S3 specifically includes:

[0114] S31, plantar contour extraction;

[0115] Obtain the subject's sole image, extract the sole contour using the OpenCV Canny algorithm, and output the sequence of upper edge points of the sole contour. And a binary mask representing the contact and non-contact areas of the sole of the foot with the ground

[0116] S32, gridding;

[0117] Mask on soles Perform grid processing to generate a two-dimensional grid (x i , z j );

[0118] S33, selecting N initial contact ball centers C using k-means++ clustering algorithm balls i ;

[0119] S34, calculating the radius R of each contact ball

[0120] For each cluster center C i , calculate its maximum circumscribed circle, the contact ball radius R i The calculation formula is:

[0121]

[0122] S35, adaptively adjusting the size and position of the contact ball

[0123] Calculate the total area of all contact balls:

[0124] Ensure that the total area is close to the actual area A of the shoe sole contact area contact If it does not match, adjust the radius R i :

[0125]

[0126] If the radius of some balls exceeds R max , split it into two smaller balls, and update the number of balls;

[0127] S36, output the contact ball distribution

[0128] Output the final contact ball center position C i and radius R i as the contact ball parameters in subsequent simulation calculations, and for each contact ball C i , based on the material properties and physical model of the area where it is located, assign each contact ball a stiffness k i and damping ε i .

[0129] Optionally, the S33 specifically includes:

[0130] S331, sampling valid pixels

[0131] Collect all pixel coordinates in the white mask representing the contact area of the shoe sole into a point set: P = {(x, z)};

[0132] S332, determine the first ball center

[0133] Randomly draw 1 point from P as the first ball center C1;

[0134] S333, select new ball centers in turn​

[0135] For each point p in P, calculate its distance D(p) from the nearest selected center, and press D(p) 2 Do probability sampling, the point drawn is the new center, until N is selected balls indivual;

[0136] S334, fine-tune the sphere centers through classic k-means so that the sphere centers are located at the geometric center of gravity of each cluster;

[0137] Steps: clustering → mean → update center, cycle until convergence.

[0138] S4. Solving the ground reaction force GRF that makes the dynamics consistent under given kinematic conditions using a unified pressure model based on the personalized skeletal model, joint motion data, and personalized plantar contact model;

[0139] Optionally, the S4 specifically includes:

[0140] S41, kinematic analysis;

[0141] Input: frame number N;

[0142] calculate:

[0143] Initial inclination angle: The initial inclination angle θ0 is obtained by preset or initial inclination angle acquisition algorithm, which represents the angle at which the sole tilts forward and upward when the heel just touches the ground;

[0144] Maximum indentation: d max ≈5~mm, which indicates the maximum deformation or compression depth that may occur in the plantar soft tissue during contact with the ground, and is preset to 5mm;

[0145] Output: θ k =θ0(1-t k )(k=1...N),d k =d max t k ;

[0146] S42, calculation of indentation displacement;

[0147] Input: Sole mask θ k ;d k ;

[0148] calculate:

[0149] Depression field:

[0150] Output: Depression field δ k ;

[0151] S43, calculation of indentation velocity;

[0152] Input: δ k , δ k-1 ;

[0153] calculate:

[0154] Indentation speed:

[0155] Output: Indentation velocity

[0156] S44, pressure calculation;

[0157] Input: Subject mass m, subject's vertical acceleration a y , gravitational acceleration g, equivalent stiffness coefficient k, equivalent damping coefficient c, exponents p, q, indentation field δ k , indentation speed Sole mask

[0158] calculate:

[0159] Pressure distribution:

[0160] Output: pressure field p k (x, z), the pressure field p k The integral of (x, z) on the contact surface ∫∫ A p k (x, z) dxdz represents the ground reaction force GRF;

[0161] Among them, p k (x, z): instantaneous pressure at the grid (x, z) / Pa; m: subject mass / kg; a y :Vertical acceleration of the subject's center of gravity / m·s -2 ; g: gravitational acceleration; k: equivalent stiffness coefficient / N·m -p ; c: equivalent damping coefficient / N·s·m -(q+1) ; p,q: index; δ k (x,z): indentation displacement / m; Indentation speed / m·s -1 ; Sole mask, contact = 1, non-contact = 0; A c : current frame contact area; t k : Current frame time ratio; θ k : current frame inclination; d k : Current frame depression depth; A: grid area / m 2 .

[0162] Optionally, the obtaining of the initial tilt angle θ0 by using an initial tilt angle obtaining algorithm specifically includes:

[0163] Step 1: Extract 2D foot features using key point method or mask method;

[0164] Among them, the key point method: detects multiple 2D foot points; the mask method: obtains the binary mask of the sole through semantic segmentation, makes a minimum bounding rectangle for the mask, and extracts the four corner points of the minimum bounding rectangle as feature points;

[0165] Step 2: Multi-view geometry 3D reconstruction

[0166] Multiple views of the same physical point 2D coordinates (u, v) c , use the known camera transformation matrix to do DLT / triangulation to obtain the 3D point P i =(x i ,y i , z i ); If mask corners are used, use the known camera transformation matrix to perform DLT / triangulation to triangulate the four corner points of the mask circumscribed rectangle in each frame;

[0167] Step 3: sole plane fitting;

[0168] Input: ≥3 3D points;

[0169] RANSAC Iterations:

[0170] Randomly select 3 points to fit a plane → find the plane equation n·(Xd)=0, where n=(n x , n y , n z ) is the normal vector of the plane, X = (x, y, z) represents any point in space, d is a known point on the plane, and "·" represents the dot product operation;

[0171] Count all the 3D points in the sole mask area that fall near the current plane. These points are called inliers. If the distance between these inliers and the fitting plane is within the set error threshold, they are considered to conform to the model.

[0172] Repeat the iteration multiple times, fitting a candidate plane each time, recording the number of its corresponding inliers, and finally selecting the set of plane parameters with the largest number of inliers as the best fitting result;

[0173] Output the unit normal vector n of the optimal plane;

[0174] Step 4: Calculate the initial inclination angle θ0

[0175] Set ground normal Indicates that the Y axis is vertically upward, and the unit normal vector n of the sole should be close to it;

[0176] Initial inclination angle = the supplementary angle of the two normal angles:

[0177] The smaller the initial inclination angle, the closer the sole is to standing on flat ground.

[0178] S5. Conduct in-depth analysis of the ground reaction force (GRF) data, determine the temporal movement path of the resultant force point based on the force conditions of each contact ball of the two feet, and calculate the trajectory of the center of pressure (COP) of the plantar surface.

[0179] Optionally, the S5 specifically includes:

[0180] Input: pressure field p k ;

[0181] calculate:

[0182] COP coordinates:

[0183] Output: COP trajectory (x cop,k , z cop,k );

[0184] Among them, x cop,k :x coordinate of COP in the current frame / m;z cop,k : z coordinate of COP in the current frame / m.

[0185] The embodiment of the present invention can also calculate the torsional moment around the vertical axis (such as M y ) and other high-level indicators are used to evaluate the torsional stability of the subjects' gait, and the above mechanical curves and indicators are output in the form of visual charts or data reports for users to view and further apply. They can be used in scenarios such as motor function evaluation and rehabilitation training feedback.

[0186] like Figure 5 As shown, an embodiment of the present invention further provides a motion analysis system based on image and sole contact modeling, the system comprising:

[0187] An acquisition module 510 is configured to acquire multi-view image data of a subject's motion using multiple cameras;

[0188] The first generation module 520 is configured to perform human posture recognition using the collected multi-view image data, reconstruct the subject's three-dimensional posture based on the identified 2D key points, and scale the standard skeleton model based on the subject's height data to generate a personalized skeleton model that matches the subject. The module then outputs the subject's joint positions and posture sequence throughout the entire motion process, as well as the scaled personalized skeleton model.

[0189] The second generation module 530 is used to obtain an image of the subject's sole, extract the sole contour through image recognition, and automatically generate multiple contact spheres in the sole area based on the extracted sole contour shape to approximate the various areas where the sole contacts the ground. The positions and radii of these contact spheres are optimized according to the subject's foot shape to form a personalized sole contact model, and output the center position C of the contact spheres. i and radius R i ;

[0190] A solving module 540 is configured to solve the ground reaction force GRF that makes the dynamics consistent under given kinematic conditions using a unified pressure model based on the personalized skeletal model, the joint motion data, and the personalized plantar contact model;

[0191] The calculation module 550 is used to conduct in-depth analysis of the ground reaction force GRF data, determine the movement path of the resultant force point over time based on the force conditions of each contact ball of the two feet, and calculate the trajectory of the center of pressure COP of the plantar.

[0192] An embodiment of the present invention provides a motion analysis system based on image and sole contact modeling, and its functional structure corresponds to an embodiment of the present invention provides a motion analysis method based on image and sole contact modeling, which will not be repeated here.

[0193] Figure 6 It is a structural diagram of an electronic device 600 provided in an embodiment of the present invention. The electronic device 600 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 601 and one or more memories 602, wherein the memory 602 stores at least one instruction, and the at least one instruction is loaded and executed by the processor 601 to implement the steps of the above-mentioned motion analysis method based on image and sole contact modeling.

[0194] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions. The instructions can be executed by a processor in a terminal to implement the above-described motion analysis method based on image and plantar contact modeling. For example, the computer-readable storage medium can be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0195] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0196] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A motion analysis method based on image and sole contact modeling, characterized in that: The method comprises: S1, using multiple cameras to collect multi-view image data of the subject's movement; S2. Perform human posture recognition using the collected multi-view image data. Reconstruct the subject's 3D posture based on the identified 2D key points. Based on the subject's height data, scale the standard skeleton model to generate a personalized skeleton model that matches the subject. Output the subject's joint positions and posture sequence throughout the entire movement process, as well as the scaled personalized skeleton model. S3. Obtain an image of the subject's sole, extract the sole contour through image recognition, and automatically generate multiple contact spheres in the sole area based on the extracted sole contour shape to approximate the various areas where the sole contacts the ground. The positions and radii of these contact spheres are optimized based on the subject's foot shape to form a personalized sole contact model, and the center position C of the contact sphere is output. i and radius R i ; S4. Solving the ground reaction force GRF that makes the dynamics consistent under given kinematic conditions using a unified pressure model based on the personalized skeletal model, joint motion data, and personalized plantar contact model; S5. Conduct in-depth analysis of the ground reaction force (GRF) data, determine the temporal movement path of the resultant force point based on the force conditions of each contact ball of the two feet, and calculate the trajectory of the center of pressure (COP) of the plantar surface.

2. The method according to claim 1, characterized in that Said S3 specifically includes: S31, plantar contour extraction; Obtain the subject's sole image, extract the sole contour using the OpenCV Canny algorithm, and output the sequence of upper edge points of the sole contour. And a binary mask representing the contact and non-contact areas of the sole of the foot with the ground S32, gridding; Mask on soles Perform grid processing to generate a two-dimensional grid (x i , z j ); S33, use k-means++ clustering algorithm to select N balls The center position C of the initial contact ball i ; S34. Calculate the radius of the contact sphere; For each cluster center C i , calculate its maximum circumscribed circle, the contact sphere radius R i Calculation formula: S35, adaptively adjusting the size and position of the contact ball; Calculate the total area of ​​all contacting balls: Make sure the total area is close to the actual area of ​​the sole contact area A contact If it does not match, adjust the radius R i : If the radius of some balls exceeds R max , then split it into two smaller balls and update the number of balls; S36, output contact ball distribution; The final contact ball center position C i and radius R i Output as the contact ball parameters in subsequent simulation calculations, and for each contact ball C i , assigning a stiffness k to each contact ball based on the material properties and physical model of the area i and damping ε i .

3. The method according to claim 2, characterized in that The S33 specifically includes: S331, sampling effective pixels; Collect all pixel coordinates in the white mask representing the contact area of ​​the sole into a point set: P = {(x, z)}; S332, determining the first sphere center; Randomly select a point from P as the first sphere center C1; S333, select new ball centers in sequence; For each point p in P, calculate its distance D(p) from the nearest selected center, and press D(p) 2 Do probability sampling, the point drawn is the new center, until N is selected balls indivual; S334, fine-tune the sphere centers through classic k-means so that the sphere centers are located at the geometric center of gravity of each cluster; Steps: clustering → mean → update center, cycle until convergence.

4. The method according to claim 2, characterized in that Said S4 specifically includes: S41, kinematic analysis; Input: frame number N; calculate: Initial inclination angle: The initial inclination angle θ0 is obtained by preset or initial inclination angle acquisition algorithm, which represents the angle at which the sole tilts forward and upward when the heel just touches the ground; Maximum indentation: d max ≈5~mm, which indicates the maximum deformation or compression depth that may occur in the plantar soft tissue during contact with the ground, and is preset to 5mm; output: i k =θ0(1-t k (k=1...N), d k =d max t k ; S42, calculation of indentation displacement; Input: Sole mask θ k ;d k ; calculate: Depression field: Output: Depression field δ k ; S43, calculation of indentation velocity; Input: δ k , δ k-1 ; calculate: Indentation speed: Output: Indentation velocity S44, pressure calculation; Input: Subject mass m, subject's vertical acceleration a y , gravitational acceleration g, equivalent stiffness coefficient k, equivalent damping coefficient c, exponents p, q, indentation field δ k , indentation speed Sole mask calculate: Pressure distribution: Output: pressure field p k (x, z), the pressure field p k The integral of (x, z) on the contact surface ∫∫ A p k (x, z) dx dz represents the ground reaction force GRF; Among them, p k (x, z): instantaneous pressure at the grid (x, z) / Pa; m: subject mass / kg; a y :Vertical acceleration of the subject's center of gravity / m·s -2 ; g: gravitational acceleration; k: equivalent stiffness coefficient / N·m -p ; c: equivalent damping coefficient / N·s·m -(q+1) ; p, q: index; δ k (x, z): indentation displacement / m; Indentation speed / m·s -1 ; Sole mask, contact = 1, non-contact = 0; A c : current frame contact area; t k : Current frame time ratio; θ k : current frame inclination; d k : Current frame depression depth; A: grid area / m 2 .

5. The method according to claim 4, characterized in that Said S5 specifically includes: Input: pressure field p k ; calculate: COP coordinates: Output: COP trajectory (x cop,k , z cop,k ); Among them, x cop,k :x coordinate of COP in the current frame / m;z cop,k : z coordinate of COP in the current frame / m.

6. The method according to claim 5, characterized in that The initial tilt angle θ0 is obtained by using the initial tilt angle acquisition algorithm, specifically including: Step 1: Extract 2D foot features using key point method or mask method; Among them, the key point method: detects multiple 2D foot points; the mask method: obtains the binary mask of the sole through semantic segmentation, makes a minimum bounding rectangle for the mask, and extracts the four corner points of the minimum bounding rectangle as feature points; Step 2: Multi-view geometry 3D reconstruction Multiple views of the same physical point 2D coordinates (u, v) c , use the known camera transformation matrix to do DLT / triangulation to obtain the 3D point P i =(x i ,y i , z i ); If mask corners are used, use the known camera transformation matrix to perform DLT / triangulation to triangulate the four corner points of the mask circumscribed rectangle in each frame; Step 3: sole plane fitting; Input: ≥3 3D points; RANSAC Iterations: Randomly select 3 points to fit a plane → find the plane equation n·(Xd)=0, where n=(n x , n y , n z ) is the normal vector of the plane, X = (x, y, z) represents any point in space, d is a known point on the plane, and "·" represents the dot product operation; Count all the 3D points in the sole mask area that fall near the current plane. These points are called inliers. If the distance between these inliers and the fitting plane is within the set error threshold, they are considered to conform to the model. Repeat the iteration multiple times, fitting a candidate plane each time, recording the number of its corresponding inliers, and finally selecting the set of plane parameters with the largest number of inliers as the best fitting result; Output the unit normal vector n of the optimal plane; Step 4: Calculate the initial inclination angle θ0 Set ground normal Indicates that the Y axis is vertically upward, and the unit normal vector n of the sole should be close to it; Initial inclination angle = the supplementary angle of the two normal angles: The smaller the initial inclination angle, the closer the sole is to standing on flat ground.

7. A motion analysis system based on image and sole contact modeling, characterized in that: The system comprises: An acquisition module, configured to acquire multi-view image data of a subject's motion using multiple cameras; The first generation module is used to perform human posture recognition using the collected multi-view image data, reconstruct the subject's 3D posture based on the identified 2D key points, and scale the standard skeleton model based on the subject's height data to generate a personalized skeleton model that matches the subject. The module then outputs the subject's joint positions and posture sequence throughout the entire movement process, as well as the scaled personalized skeleton model. The second generation module is used to obtain the subject's sole image, extract the sole contour through image recognition, and automatically generate multiple contact spheres in the sole area according to the extracted sole contour shape to approximate the various areas where the sole contacts the ground. The positions and radii of these contact spheres are optimized according to the subject's foot shape to form a personalized sole contact model and output the center position C of the contact sphere. i and radius R i ; A solving module, configured to solve the ground reaction force GRF that makes the dynamics consistent under given kinematic conditions using a unified pressure model based on the personalized skeletal model, the joint motion data and the personalized plantar contact model; The calculation module is used to conduct in-depth analysis of the ground reaction force GRF data, determine the movement path of the resultant force point over time based on the force conditions of each contact ball of the two feet, and calculate the trajectory of the center of pressure COP of the plantar.

8. An electronic device comprising a processor and a memory, wherein the memory stores at least one instruction, characterized in that: The at least one instruction is loaded and executed by the processor to implement the motion analysis method based on image and sole contact modeling as described in any one of claims 1-6.

9. A computer-readable storage medium, wherein at least one instruction is stored in the storage medium, characterized in that: The at least one instruction is loaded and executed by the processor to implement the motion analysis method based on image and sole contact modeling as described in any one of claims 1-6.

Citation Information

Patent Citations

  • Gait analysis method and system based on intelligent shoes

    CN119632547A

  • Method and system for decomposing superimposed ground reaction forces into individual force profiles

    US20160045139A1

  • Method and apparatus for estimating human motion kinetics information

    US20200243198A1

  • Biometric system and method

    US20210121100A1