A method and system for motion analysis based on image and plantar contact modeling

Abstract

The application provides a motion analysis method and system based on image and foot bottom contact modeling, comprising the following steps: reconstructing the three-dimensional posture of the subject motion according to the identified 2D key points, and generating a personalized skeletal model matched with the subject according to the height data of the subject; obtaining the foot bottom image of the subject, and automatically generating a plurality of contact spheres in the foot bottom area according to the extracted foot bottom contour shape to approximately represent each area of the foot bottom contact with the ground, the positions and radii of the contact spheres are optimized and set according to the foot type of the subject, and a personalized foot bottom contact model is formed; solving the ground reaction force (GRF) consistent with the dynamics under the given kinematic conditions according to the personalized skeletal model, joint motion data and personalized foot bottom contact model; determining the moving path of the force point of action of the resultant force with time according to the force condition of each contact sphere of the double feet, and calculating the trajectory of the foot bottom pressure center (COP). The application can perform motion analysis based on image and foot bottom contact modeling.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of action analysis, in particular to an action analysis method and system based on image and foot contact modeling. BACKGROUND

[0002] Ground Reaction Force (GRF) is one of the core dynamic parameters describing the interaction between the human body and the ground during movement, and is widely used in sports biomechanics analysis, rehabilitation assessment, gait recognition, intelligent wearable device control and other fields. Traditional GRF acquisition methods rely on force platforms or pressure sensing mats and other devices. Although these devices have high data accuracy, they are expensive, have harsh layout conditions, poor portability, and can only be used in laboratory environments, making it difficult to meet the long-term, low-cost and open-space usage requirements in actual scenarios.

[0003] With the development of computer vision and human pose recognition technology, researchers have begun to try to use cameras to obtain human motion images, and indirectly estimate the GRF during movement through pose estimation and model solving, to achieve "sensorless" and "force platform-free" ground reaction force acquisition. This method combines the advantages of visual perception and physical modeling, has good generality and scalability, and has become a hot research direction in the intersection of biomechanics and artificial intelligence in recent years.

[0004] However, current image-driven GRF estimation methods either rely on black box models or lack individual contact modeling capabilities, making it difficult to meet the demand for high-precision GRF estimation in open spaces and uncalibrated environments. SUMMARY

[0005] To solve the technical problems existing in the prior art, the present application provides an action analysis method and system based on image and foot contact modeling, and the technical solution is as follows:

[0006] On the one hand, an action analysis method based on image and foot contact modeling is provided, which comprises:

[0007] S1, using multiple cameras to collect multi-view image data of the subject's movement;

[0008] S2, using the collected multi-view image data to recognize human pose, reconstructing the three-dimensional pose of the subject's movement according to the recognized 2D key points, and scaling the standard skeletal model according to the height data of the subject, to generate a personalized skeletal model matching the subject, and outputting the joint position, pose sequence and scaled personalized skeletal model of the subject during the entire movement process;

[0009] S3, obtain the foot bottom image of the subject, extract the foot bottom contour through image recognition, and automatically generate a plurality of contact spheres in the foot bottom area according to the shape of the extracted foot bottom contour to approximately represent each area of the foot bottom in contact with the ground, the positions and radii of the contact spheres are optimized and set according to the foot type of the subject, forming a personalized foot bottom contact model, and outputting the center positions C of the contact spheres i and radii R i ;

[0010] S4, according to the personalized bone model, joint motion data and personalized foot bottom contact model, using a unified pressure model to solve the ground reaction force GRF that makes the dynamics consistent under given kinematics conditions;

[0011] S5, deeply analyze the ground reaction force GRF data, determine the moving path of the force action point with time according to the force condition of each contact sphere of the two feet, and calculate the trajectory of the foot bottom pressure center COP.

[0012] Optionally, the S3 specifically comprises:

[0013] S31, foot bottom contour extraction;

[0014] Obtain the foot bottom image of the subject, extract the foot bottom contour through OpenCV Canny algorithm, and output the edge point sequence on the foot bottom contour and the binary mask representing the contact and non-contact areas of the foot bottom with the ground

[0015] S32, gridding;

[0016] Grid the shoe sole mask to generate a two-dimensional grid (x i , z j );

[0017] S33, select N balls initial contact sphere center positions C i using the k-means++ clustering algorithm;

[0018] S34, calculate the radius of the contact sphere;

[0019] For each cluster center C i , calculate its maximum circumscribed circle, and the contact sphere radius R i The calculation formula is:

[0020]

[0021] S35, adaptively adjust the size and position of the contact sphere;

[0022] Calculate the total area of all contact spheres:

[0023] Ensure the total area is close to the actual area A of the sole contact area contact If not, adjust the radius R i :

[0024]

[0025] If some ball's radius exceeds R max , split it into two smaller balls and update the number of balls;

[0026] S36, output the contact ball distribution;

[0027] Output the final contact ball center position C i and radius R i as the contact ball parameters in subsequent simulation calculations, and for each contact ball C i , based on the material properties and physical model of the area where it is located, assign each contact ball a stiffness k i and damping ε i .

[0028] Optionally, the S33 specifically includes:

[0029] S331, sample valid pixels;

[0030] Collect all pixel coordinates in the white mask representing the contact area of the sole into a point set: P = {(x, z)};

[0031] S332, determine the first ball center;

[0032] Randomly draw 1 point from P as the first ball center C1;

[0033] S333, select new ball centers in turn;

[0034] For each point p in P, calculate its distance D(p) from the nearest selected center, and sample according to D(p) 2 , the drawn point is the new center, until N balls is selected;

[0035] S334, fine-tune the ball centers by classical k-means to make the ball centers located at the geometric center of gravity of their respective clusters;

[0036] Steps: clustering → mean → updating center, loop to convergence.

[0037] Optionally, the S4 specifically includes:

[0038] S41, kinematic analysis;

[0039] Input: frame number N;

[0040] Compute:

[0041] Initial inclination: initial inclination θ0 is obtained by preset or by initial inclination obtaining algorithm, representing the angle of the whole sole tilting upward when the heel just touches the ground;

[0042] Maximum indentation: d max ≈5~mm, representing the maximum deformation or compression depth of the plantar soft tissue that may occur during contact with the ground, preset 5mm;

[0043] Output: θ k = θ0(1-t k )(k = 1...N), d k = d max t k ;

[0044] S42, indentation displacement calculation;

[0045] Input: sole mask θ k ; d k ;

[0046] Compute:

[0047] Indentation field:

[0048] Output: indentation field δ k ;

[0049] S43, indentation velocity calculation;

[0050] Input: δ k , δ k-1 ;

[0051] Compute:

[0052] Indentation velocity:

[0053] Output: indentation velocity

[0054] S44, pressure calculation;

[0055] Input: subject mass m, vertical acceleration of subject's center of gravity a y , gravitational acceleration g, equivalent stiffness coefficient k, equivalent damping coefficient c, index p, q, indentation field δ k , indentation velocity Sole mask

[0056] Compute:

[0057] Pressure distribution:

[0058] Output: Pressure field p k (x, z), the integral of the pressure field p k (x, z) over the contact surface ∫∫ A p k (x, z) dxdz represents the ground reaction force GRF;

[0059] where p k (x, z): instantaneous pressure at grid (x, z) / Pa; m: subject mass / kg; a y : vertical acceleration of the subject's center of mass / m·s -2 -2; g: acceleration due to gravity; k: equivalent stiffness coefficient / N·m -p ; c: equivalent damping coefficient / N·s·m -(q+1) ; p, q: exponents; δ k (x, z): indentation displacement / m; indentation velocity / m·s -1 ; Sole mask, contact = 1, non-contact = 0; A c : current frame contact area; t k : current frame time ratio; θ k : current frame inclination angle; d k : current frame indentation depth; A: grid area / m 2 .

[0060] Optionally, the S5 specifically comprises:

[0061] Input: pressure field p k ;

[0062] Calculation:

[0063] COP coordinates:

[0064] Output: COP trajectory (x cop，k , z cop，k );

[0065] where x cop，k : x-coordinate of the current frame COP / m; z cop，k : z-coordinate of the current frame COP / m.

[0066] Optionally, the initial inclination angle θ0 is obtained by an initial inclination angle obtaining algorithm, specifically comprising:

[0067] Step 1: Extract 2D foot features by key point method or mask method;

[0068] Wherein the key point method: detecting multiple 2D foot points; the mask method: obtaining a shoe sole binary mask through semantic segmentation, performing minimum circumscribed rectangle on the mask, and extracting four corner points of the minimum circumscribed rectangle as feature points;

[0069] Step2: Multi-view geometric 3D reconstruction

[0070] Multi-view 2D coordinates (u, v) of the same physical point c , using the known camera transformation matrix to perform DLT / triangulation to obtain 3D point P i =(x i , y i , z i ); if the mask corner point is used, the known camera transformation matrix is used to perform DLT / triangulation on the four corner points of the circumscribed rectangle of the mask in each frame of image;

[0071] Step3: Shoe sole plane fitting;

[0072] Input: ≥3 3D points;

[0073] RANSAC iteration:

[0074] Randomly take 3 points to fit a plane → solve the plane equation n·(X-d)=0, where n=(n x , n y , n z ) is the normal vector of the plane, X=(x, y, z) represents any point in space, d is a known point on the plane, and “·” represents dot product operation;

[0075] Among all the three-dimensional points in the shoe sole mask area, count the number of points falling near the current plane, called inliers. These inliers are considered to meet the model if the distance between them and the fitted plane is within a set error threshold;

[0076] Repeat the iteration multiple times, each time fitting a candidate plane, record the corresponding number of inliers, and finally select the plane parameters with the most inliers as the best fitting result;

[0077] Output the unit normal vector n of the best plane;

[0078] Step4: Calculate the initial inclination θ0

[0079] Let the ground normal represent the Y-axis vertically upward, and the shoe sole unit normal vector n should be close to it;

[0080] Initial inclination = complement angle of the included angle between the two normals:

[0081] The smaller the initial inclination, the closer the shoe sole stands to the ground.

[0082] In another aspect, an action analysis system based on image and foot contact modeling is provided, the system comprising:

[0083] An acquisition module configured to acquire multi-view image data of a subject's movement using a plurality of cameras;

[0084] A first generation module configured to perform human pose recognition using the acquired multi-view image data, reconstruct a three-dimensional pose of the subject's movement based on the recognized 2D key points, scale a standard skeletal model based on the subject's height data, generate a personalized skeletal model matching the subject, and output joint positions, pose sequences, and the scaled personalized skeletal model of the subject during the entire movement process;

[0085] A second generation module configured to acquire a subject's foot image, extract a foot contour through image recognition, automatically generate a plurality of contact spheres in the foot region based on the extracted foot contour shape to approximately represent each region of the foot-ground contact, optimize the positions and radii of the contact spheres according to the subject's foot type to form a personalized foot contact model, and output the center positions C i and radii R i of the contact spheres;

[0086] A solving module configured to solve the ground reaction force GRF that makes the dynamics consistent under given kinematic conditions using a uniform pressure model based on the personalized skeletal model, joint movement data, and personalized foot contact model;

[0087] A calculation module configured to perform in-depth analysis on the ground reaction force GRF data, determine the moving path of the force action point over time based on the force conditions of each contact sphere of the two feet, and calculate the trajectory of the foot pressure center COP.

[0088] In another aspect, an electronic device is provided, comprising a processor and a memory, the memory having at least one instruction stored therein, the at least one instruction being loaded and executed by the processor to implement the above-mentioned action analysis method based on image and foot contact modeling.

[0089] In another aspect, a computer-readable storage medium is provided, the storage medium having at least one instruction stored therein, the at least one instruction being loaded and executed by a processor to implement the above-mentioned action analysis method based on image and foot contact modeling.

[0090] The technical solutions provided by the present application have at least the following beneficial effects:

[0091] The present application has significantly improved the accuracy of individualized modeling, GRF solving, and COP calculation, specifically in the following aspects:

[0092] 1) Higher individualized modeling capability: The present application automatically generates a plantar contact sphere model driven by images, enabling personalized modeling for different subjects' plantar shape and anatomical features. The plantar model of the present application can be adjusted according to individual foot type differences, greatly improving the model's adaptability to individual differences and ensuring that the contact model fits each subject more accurately.

[0093] 2) Improved GRF solving accuracy and robustness: The method of the present application does not rely on empirical fitting of specific data sets, but calculates GRF through a unified pressure model, thus maintaining high prediction accuracy in different motion types and individual cases. This means that regardless of the subject's body type, gait, or the action performed, the present application can robustly provide accurate GRF solving results, with better universality.

[0094] 3) More accurate contact point representation and COP calculation: Due to the use of a refined, multi-contact point plantar sphere division model, the ground reaction force acting area is more accurately represented, with each contact sphere only being subjected to force within its plantar region. This makes the GRF action point calculation (COP) more accurate, and the present application can more realistically reflect the position changes of the plantar pressure center, thus improving the inference accuracy of key indicators such as COP.

[0095] 4) High-precision dynamic indicator inference capability: With accurate individualized models and unified pressure model calculations, the present application can not only obtain three-dimensional GRF time curves, but also infer high-precision dynamic indicators such as torque (M y ) around the vertical axis. The results provided by the present application are closer to the true values in terms of force size, direction, and action point, making the calculated COP trajectory, torque M y , and other indicators more reliable. This high-precision dynamic analysis is of great significance for clinical gait analysis, motion injury assessment, and other applications, demonstrating the significant superiority of the present application in practical applications.

[0096] In summary, by integrating multi-camera pose recognition, individualized plantar contact geometry modeling, and solving based on a unified pressure model, the present application realizes high-precision estimation of ground reaction force, overcoming the shortcomings of simple plantar contact models and poor individual adaptability in existing technologies, and can provide more accurate and reliable analysis tools for biomechanics research and clinical applications, with significant benefits. BRIEF DESCRIPTION OF DRAWINGS

[0097] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without any creative effort.

[0098] Figure 1 is a flow chart of a motion analysis method based on image and foot contact modeling provided by an embodiment of the present application;

[0099] Figure 2 is a general block diagram of a motion analysis method based on image and foot contact modeling provided by an embodiment of the present application;

[0100] Figure 3 is a layout diagram of a collection device of a motion analysis method based on image and foot contact modeling provided by an embodiment of the present application;

[0101] Figure 4 is a schematic diagram of foot region contact ball division provided by an embodiment of the present application;

[0102] Figure 5 is a system block diagram of a motion analysis system based on image and foot contact modeling provided by an embodiment of the present application;

[0103] Figure 6 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0104] In order to make the technical problems, technical solutions and advantages of the present application more clear, the following will be described in detail with reference to the drawings and specific embodiments.

[0105] An embodiment of the present application provides a motion analysis method based on image and foot contact modeling, which can be realized by an electronic device, which can be a terminal or a server. Figure 1 is a flow chart of the method, Figure 2 is a general block diagram of the method, and the processing flow can include the following steps:

[0106] S1, using a plurality of cameras to collect multi-view image data of subject motion;

[0107] An embodiment of the present application collects image data of a subject motion process from multiple perspectives, synchronously acquires multi-view video or sequence images of the subject motion through three cameras (as shown in the accompanying drawings) arranged at a certain angle in space, and provides complete appearance information of the subject motion through the cooperative collection of the three cameras, thereby providing data support for subsequent posture recognition and modeling. Figure 3 ​

[0108] S2, human posture recognition is performed by using the collected multi-view image data, a three-dimensional posture of the subject motion is reconstructed according to the recognized 2D key points, a standard bone model is scaled according to the height data of the subject, a personalized bone model matched with the subject is generated, and joint positions, posture sequences of the subject in the whole motion process and the scaled personalized bone model are output;

[0109] The embodiment of the application can utilize the HRNet deep learning algorithm to perform human posture recognition, and a plurality of 2D key points (including shoulders, knees, hip joints, elbow joints, etc.) are recognized, and the specific recognition process is not described herein.

[0110] The embodiment of the application can utilize the 2D key points of a plurality of cameras to perform 3D key point reconstruction by triangulation and camera calibration, and a weighted least square method, and the specific reconstruction process is not described herein.

[0111] S3, a plantar image of the subject is acquired, a plantar contour is extracted through image recognition, a plurality of contact spheres are automatically generated in the plantar region according to the shape of the extracted plantar contour to approximately represent each region (such as a heel, an inner side of a forefoot, an outer side of a forefoot, toes, etc.) of the plantar surface in contact with the ground (see FIG. 2), Figure 4 The positions and radii of these contact spheres are optimized and set according to the foot type of the subject to form a personalized plantar contact model, and the center positions C i and radii R i of the contact spheres are output.

[0112] The embodiment of the application realizes individualized reconstruction of plantar contact geometry in an image-driven manner, so that each subject has a contact model matched with the shape of the foot, and the generated personalized plantar contact model will serve as a basis for subsequent foot-ground contact force calculation.

[0113] Optionally, the S3 specifically includes:

[0114] S31, plantar contour extraction;

[0115] A plantar image of the subject is acquired, a plantar contour is extracted through an OpenCV Canny algorithm, and a sequence of upper edge points of the plantar contour and a binary mask representing the contact and non-contact regions of the plantar surface with the ground

[0116] S32, gridding;

[0117] The shoe sole mask is subjected to gridding processing to generate a two-dimensional grid (x i , z j );

[0118] S33, selecting N initial contact ball centers C using k-means++ clustering algorithm balls i ;

[0119] S34, calculating the radius R of each contact ball

[0120] For each cluster center C i , calculate its maximum circumscribed circle, the contact ball radius R i The calculation formula is:

[0121]

[0122] S35, adaptively adjusting the size and position of the contact ball

[0123] Calculate the total area of all contact balls:

[0124] Ensure that the total area is close to the actual area A of the shoe sole contact area contact If it does not match, adjust the radius R i :

[0125]

[0126] If the radius of some balls exceeds R max , split it into two smaller balls, and update the number of balls;

[0127] S36, output the contact ball distribution

[0128] Output the final contact ball center position C i and radius R i as the contact ball parameters in subsequent simulation calculations, and for each contact ball C i , based on the material properties and physical model of the area where it is located, assign each contact ball a stiffness k i and damping ε i .

[0129] Optionally, the S33 specifically includes:

[0130] S331, sampling valid pixels

[0131] Collect all pixel coordinates in the white mask representing the contact area of the shoe sole into a point set: P = {(x, z)};

[0132] S332, determine the first ball center

[0133] Randomly draw 1 point from P as the first ball center C1;

[0134] S333, select new ball centers in turn​

[0135] For each point p in P, calculate its distance D(p) to the nearest selected center, and select the center with the smallest D(p) 2 Do probabilistic sampling, the sampled point is the new center, until N balls is selected;

[0136] S334, fine-tune the ball center by classical k-means, so that the ball center is located at the geometric center of gravity of the respective cluster;

[0137] Steps: clustering → mean → updating center, loop to convergence.

[0138] S4, according to the personalized skeletal model, joint motion data and personalized plantar contact model, using a uniform pressure model to solve the ground reaction force GRF that makes the dynamics consistent under given kinematics conditions;

[0139] Optionally, the S4 specifically comprises:

[0140] S41, kinematics analysis;

[0141] Input: frame number N;

[0142] Calculation:

[0143] Initial inclination: the initial inclination θ0 is obtained by presetting or by an initial inclination obtaining algorithm, indicating the angle of the overall shoe sole tilting upward when the heel just touches the ground;

[0144] Maximum indentation: d max ≈5~mm, indicating the maximum deformation or compression depth of the plantar soft tissue during contact with the ground, preset 5mm;

[0145] Output: θ k = θ0(1-t k )(k = 1...N), d k = d max t k ;

[0146] S42, indentation displacement calculation;

[0147] Input: shoe sole mask θ k ; d k ;

[0148] Calculation:

[0149] Indentation field:

[0150] Output: indentation field δ k ;

[0151] S43, calculate the indentation velocity

[0152] Input: δ k , δ k-1 ;

[0153] Calculate:

[0154] Indentation velocity:

[0155] Output: indentation velocity

[0156] S44, calculate the pressure

[0157] Input: subject mass m, vertical acceleration of the subject's center of gravity a y , gravitational acceleration g, equivalent stiffness coefficient k, equivalent damping coefficient c, index p, q, indentation field δ k , indentation velocity Sole mask

[0158] Calculate:

[0159] Pressure distribution:

[0160] Output: pressure field p k (x, z), the integral of the pressure field p k (x, z) on the contact surface ∫∫ A p k (x, z) dxdz represents the ground reaction force GRF;

[0161] where p k (x, z): instantaneous pressure at grid (x, z) / Pa; m: subject mass / kg; a y : vertical acceleration of the subject's center of gravity / m·s -2 ; g: gravitational acceleration; k: equivalent stiffness coefficient / N·m -p ; c: equivalent damping coefficient / N·s·m -(q+1) ; p, q: index; δ k (x, z): indentation displacement / m; Indentation velocity / m·s -1 ; Sole mask, contact = 1, non-contact = 0; A c : current frame contact area; t k : current frame time ratio; θ k : current frame inclination angle; d k : current frame indentation depth; A: grid area / m 2 .

[0162] Optionally, the initial inclination θ0 is obtained by an initial inclination acquisition algorithm, specifically comprising:

[0163] Step 1: Extract 2D foot features by key point method or mask method;

[0164] Wherein the key point method: detect multiple 2D foot points; the mask method: obtain a shoe sole binary mask by semantic segmentation, perform minimum circumscribed rectangle on the mask, and extract four corner points of the minimum circumscribed rectangle as feature points;

[0165] Step 2: Multi-view geometric 3D reconstruction

[0166] Multi-view 2D coordinates (u, v) of the same physical point c , using the known camera transformation matrix to perform DLT / triangulation to obtain 3D point P i = (x i , y i , z i ); if using mask corner points, using the known camera transformation matrix to perform DLT / triangulation on the four corner points of the circumscribed rectangle of the mask in each frame of image;

[0167] Step 3: Fitting of shoe sole plane

[0168] Input: ≥3 3D points

[0169] RANSAC iteration:

[0170] Randomly take 3 points to fit a plane → solve the plane equation n·(X-d)=0, where n=(n x , n y , n z ) is the normal vector of the plane, X=(x, y, z) represents any point in space, d is a known point on the plane, and "·" represents dot product operation;

[0171] Among all the three-dimensional points in the shoe sole mask area, count the number of points falling near the current plane, called inliers, which are within the set error threshold and considered to conform to the model;

[0172] Repeat the iteration multiple times, each time fitting a candidate plane, record the corresponding number of inliers, and finally select the plane parameters with the most inliers as the best fitting result;

[0173] Output the unit normal vector n of the best plane

[0174] Step 4: Calculate initial inclination θ0

[0175] Let the ground normal represent the Y-axis vertically upward, and the shoe sole unit normal vector n should be close to it;

[0176] Initial inclination angle = supplementary angle of two normal angles:

[0177] The smaller the initial inclination angle, the closer the shoe sole is to standing on the ground.

[0178] S5, in-depth analysis of the ground reaction force GRF data, according to the force of each contact ball of the two feet Determine the moving path of the force point with time, calculate the trajectory of the plantar pressure center COP.

[0179] Optionally, the S5 specifically comprises:

[0180] Input: pressure field p k ;

[0181] Calculation:

[0182] COP coordinates:

[0183] Output: COP trajectory (x cop，k , z cop，k );

[0184] Wherein, x cop，k : x coordinate of current frame COP / m; z cop，k : z coordinate of current frame COP / m.

[0185] The embodiment of the application can also calculate the torsional moment (such as M y ) around the vertical axis and other high-order indicators, which are used to evaluate the torsional stability in the gait of the subject, and output the above mechanical curves and indicators in the form of visual charts or data reports for the user to view and further apply, which can be used in the scenes of motor function evaluation, rehabilitation training feedback, etc.

[0186] As shown in Figure 5 , the embodiment of the application also provides a motion analysis system based on image and foot contact modeling, the system comprises:

[0187] The acquisition module 510 is used for collecting multi-view image data of the subject's motion using multiple cameras;

[0188] The first generation module 520 is used for human posture recognition using the collected multi-view image data, reconstructing the three-dimensional posture of the subject's motion according to the recognized 2D key points, and scaling the standard skeletal model according to the height data of the subject, generating a personalized skeletal model matched with the subject, and outputting the joint position, posture sequence and scaled personalized skeletal model of the subject in the entire motion process.

[0189] The second generation module 530 is configured to acquire a subject foot bottom image, extract a foot bottom contour through image recognition, automatically generate a plurality of contact spheres in the foot bottom area according to the extracted foot bottom contour shape to approximately represent each area of the foot bottom in contact with the ground, the positions and radii of the contact spheres are set according to the foot type of the subject to form a personalized foot bottom contact model, and output the contact sphere center position C i and the radius R i ;

[0190] The solving module 540 is configured to solve the ground reaction force GRF that makes the dynamics consistent under the given kinematic conditions by using a uniform pressure model according to the personalized bone model, joint motion data and the personalized foot bottom contact model.

[0191] The calculation module 550 is configured to deeply analyze the ground reaction force GRF data, determine the moving path of the force action point with time according to the force condition of each contact sphere of the double foot, and calculate the trajectory of the foot bottom pressure center COP.

[0192] The action analysis system based on image and foot bottom contact modeling provided by the embodiment of the application corresponds to the action analysis method based on image and foot bottom contact modeling provided by the embodiment of the application, and details are not repeated here.

[0193] Figure 6 FIG. 6 is a structural schematic diagram of an electronic device 600 provided by the embodiment of the application. The electronic device 600 can have great differences due to different configurations or performances, and can include one or more processors (central processing units, CPUs) 601 and one or more memories 602. The memory 602 stores at least one instruction, which is loaded and executed by the processor 601 to realize the steps of the above-mentioned action analysis method based on image and foot bottom contact modeling.

[0194] In the exemplary embodiment, a computer readable storage medium, for example, a memory including instructions, is also provided. The above-mentioned instructions can be executed by the processor in the terminal to complete the above-mentioned action analysis method based on image and foot bottom contact modeling. For example, the computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk and an optical data storage device, etc.

[0195] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by a program instructing related hardware, and the program can be stored in a computer readable storage medium, such as a read-only memory, a magnetic disk or an optical disk.

[0196] The above merely describes preferred embodiments of the present application, and is not used to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A motion analysis method based on image and foot contact modeling, characterized in that, The method includes: S1. Use multiple cameras to collect multi-view image data of the subject's movement; S2. Use the collected multi-view image data to perform human posture recognition, reconstruct the three-dimensional posture of the subject's movement based on the identified 2D key points, and scale the standard skeletal model according to the subject's height data to generate a personalized skeletal model that matches the subject. Output the subject's joint positions, posture sequence and scaled personalized skeletal model throughout the movement process. S3. Acquire images of the subject's soles, extract the sole contour through image recognition, and automatically generate multiple contact spheres in the sole area to approximate the various areas of contact between the sole and the ground based on the extracted sole contour shape. The positions and radii of these contact spheres are optimized according to the subject's foot shape to form a personalized sole contact model, and the center position C of the contact spheres is output. i and radius R i ; S4. Based on the personalized skeletal model, joint motion data and personalized plantar contact model, use the uniform pressure model to solve the ground reaction force (GRF) that makes the dynamics consistent under given kinematic conditions. S5. Conduct in-depth analysis of the ground reaction force (GRF) data, determine the movement path of the resultant force point over time based on the force conditions of each contact ball of both feet, and calculate the trajectory of the plantar pressure center (COP).

2. The method according to claim 1, characterized in that, S3 specifically includes: S31. Foot contour extraction; Acquire images of the subject's soles, extract the sole contour using the OpenCV Canny algorithm, and output the sequence of upper edge points of the sole contour. And a binary mask representing the areas of contact and non-contact between the sole of the foot and the ground. S32, Meshization; Masking the sole Perform meshing to generate a two-dimensional mesh (x i , z j ); S33. Use the k-means++ clustering algorithm to select N. balls The center position of the initial contact ball C i ; S34. Calculate the radius of the contact ball; For each cluster center C i Calculate its maximum circumcircle and the radius R of the contact sphere. i Calculation formula: S35. Adaptively adjust the size and position of the contact ball; Calculate the total area of ​​all contacting balls: Ensure the total area is close to the actual area A of the sole contact area. contact If they don't match, adjust the radius R. i : If the radius of some spheres exceeds R max If so, split it into two smaller balls and update the number of balls; S36, Output contact ball distribution; The final contact ball center position C i and radius R i The output is used as the contact ball parameters in subsequent simulation calculations, and for each contact ball C... i Based on the material properties and physical model of the region, a stiffness k is assigned to each contact ball. i and damping ε i .

3. The method according to claim 2, characterized in that, S33 specifically includes: S331, Sampled valid pixels; Collect all the pixel coordinates in the white mask representing the contact area on the sole of the shoe into a point set: P = {(x, z)}; S332, Determine the first center of the ball; Randomly select one point from P as the first center of the ball C1; S333, Select the new center ball in sequence; For each point p in P, calculate its distance D(p) to the nearest selected center, and then... 2 By performing probability sampling, the selected point becomes the new center, and this process continues until N points are selected. balls indivual; S334. Fine-tune the sphere center using the classic k-means algorithm so that the sphere center is located at the geometric centroid of its respective cluster; Steps: Clustering → Mean → Update center, repeat until convergence.

4. The method according to claim 2, characterized in that, S4 specifically includes: S41, Kinematic analysis; Input: Number of frames N; calculate: Initial tilt angle: The initial tilt angle θ0 is obtained by preset or by the initial tilt angle acquisition algorithm. It represents the angle at which the sole of the shoe tilts forward and upward when the heel just touches the ground. Maximum indentation: d max ≈5~mm, representing the maximum deformation or compression depth that the soft tissue of the foot may undergo during contact with the ground, preset to 5mm; output: i k =θ0(1-t k (k=1...N), d k =d max t k ; S42. Calculation of indentation displacement; Input: Shoe sole mask θ k ;d k ; calculate: Depression site: Output: Inclination field δ k ; S43. Calculation of indentation rate; Input: δ k δ k-1 ; calculate: Indentation speed: Output: Indentation speed S44, Pressure Calculation; Input: Subject's mass m, vertical acceleration a of the subject's center of gravity y Gravitational acceleration g, equivalent stiffness coefficient k, equivalent damping coefficient c, exponents p and q, indentation field δ k Indentation speed shoe sole mask calculate: Pressure distribution: Output: Pressure field p k (x, z), the pressure field p k The integral of (x, z) over the contact surface ∫∫ A p k (x, z)dx dz represents the ground reaction force GRF; Where, p k (x, z): Instantaneous pressure at grid (x, z) / Pa; m: Subject mass / kg; a y Vertical acceleration of the subject's center of gravity / m·s -2 g: gravitational acceleration; k: equivalent stiffness coefficient / N·m -p c: Equivalent damping coefficient / N·s·m -(q+1) p, q: exponents; δ k (x, z): indentation displacement / m; Indentation velocity / m·s -1 ; Shoe sole mask, contact = 1, non-contact = 0; A c : Current frame contact area; t k : Current frame time ratio; θ k : Current frame tilt angle; d k : Current frame indentation depth; A: Grid area / m 2 .

5. The method according to claim 4, characterized in that, S5 specifically includes: Input: pressure field p k ; calculate: COP coordinates: Output: COP trajectory (x cop，k , z cop，k ); Where, x cop，k : The x-coordinate / m of the current frame COP; z-coordinate cop，k : The z-coordinate / m of the current frame COP.

6. The method according to claim 5, characterized in that, The process of obtaining the initial tilt angle θ0 using the initial tilt angle acquisition algorithm specifically includes: Step 1: Extract 2D foot features using the keypoint method or masking method; The key point method detects multiple 2D foot points; the mask method obtains a binary mask of the sole through semantic segmentation, constructs a minimum bounding rectangle for the mask, and extracts the four corner points of the minimum bounding rectangle as feature points. Step 2: Multi-view geometric 3D reconstruction Multi-view 2D coordinates (u, v) of the same physical point c Using the known camera transformation matrix, DLT / triangulation is performed to obtain the 3D point P. i =(x i y i , z i If mask corner points are used, DLT / triangulation is performed using the known camera transformation matrix to triangulate the four corner points of the mask bounding rectangle in each frame of the image. Step 3: Fitting the sole plane; Input: ≥3 3D points; RANSAC iteration: Randomly select 3 points to fit a plane → find the plane equation n·(Xd)=0, where n=(n x n y n z ) is the normal vector of the plane, X = (x, y, z) represents any point in space, d is a known point on the plane, and "·" represents the dot product operation; The number of points that fall near the current plane among all three-dimensional points within the shoe sole mask area is called inliers. If the distance between these inliers and the fitted plane is within the set error threshold, they are considered to conform to the model. Repeat the iteration multiple times, fitting a candidate plane each time, recording the number of its corresponding interior points, and finally selecting the set of plane parameters with the most interior points as the best fitting result. Output the unit normal vector n of the optimal plane; Step 4: Calculate the initial tilt angle θ0 Assume ground normal This indicates that the Y-axis points vertically upwards, and the unit normal vector n of the shoe sole should be close to it; Initial tilt angle = supplementary angle of the two normals: The smaller the initial tilt angle, the closer the shoe sole is to standing on flat ground.

7. A motion analysis system based on image and foot contact modeling, characterized in that, The system includes: The acquisition module is used to acquire multi-view image data of the subject's movement using multiple cameras; The first generation module is used to perform human posture recognition using the collected multi-view image data, reconstruct the three-dimensional posture of the subject's movement based on the identified 2D key points, and scale the standard skeletal model according to the subject's height data to generate a personalized skeletal model that matches the subject, and output the subject's joint positions, posture sequence and scaled personalized skeletal model throughout the movement process. The second generation module acquires images of the subject's soles, extracts the sole contour through image recognition, and automatically generates multiple contact spheres in the sole area to approximate the various contact areas between the sole and the ground based on the extracted sole contour shape. The positions and radii of these contact spheres are optimized according to the subject's foot shape to form a personalized sole contact model, and the center position C of the contact spheres is output. i and radius R i ; The solver module is used to solve the ground reaction force (GRF) that makes the dynamics consistent under given kinematic conditions, based on the personalized skeletal model, joint motion data, and personalized plantar contact model, using a uniform pressure model. The calculation module is used to perform in-depth analysis of ground reaction force (GRF) data, determine the movement path of the resultant force point over time based on the force conditions of each contact ball of both feet, and calculate the trajectory of the plantar pressure center (COP).

8. An electronic device comprising a processor and a memory, wherein the memory stores at least one instruction, characterized in that, The processor loads and executes at least one instruction to implement the motion analysis method based on image and plantar contact modeling as described in any one of claims 1-6.

9. A computer-readable storage medium storing at least one instruction, characterized in that, The at least one instruction is loaded and executed by the processor to implement the motion analysis method based on image and plantar contact modeling as described in any one of claims 1-6.

Images