Method and system for acquiring three-dimensional coordinates of any gait model

Abstract

The invention relates to the technical field of gait reconstruction, in particular to a method and system for obtaining three-dimensional coordinates of any gait model, and the method comprises the steps: extracting two-dimensional key point data of a human body, and carrying out the multi-view synchronization and figure association; based on preset camera calibration parameters, converting the two-dimensional key point data into three-dimensional space coordinates through triangulation calculation; drawing self-defined mark points in the static skeleton model, loading and setting attributes of the self-defined mark points and an association relationship between the self-defined mark points and the parent class framework, and performing zooming adjustment on the skeleton model; based on the three-dimensional space coordinates, combining with an inverse kinematics model to drive the static skeleton to generate a dynamic skeleton; and calculating the three-dimensional dynamic coordinates of the self-defined mark point in the movement process in real time, and generating dynamic position time series data associated with the time point. According to the invention, the problems of high cost, complex deployment and strong environment dependence in the prior art are effectively solved, and the three-dimensional coordinate acquisition method which is low in cost, high in flexibility and suitable for diversified gait analysis is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of gait reconstruction, and in particular to a method and system for acquiring three-dimensional coordinates of an arbitrary gait model. Background Art

[0002] Gait analysis and 3D reconstruction technology are widely used in virtual reality, robotics, sports medicine, intelligent monitoring and other fields. Existing technologies usually rely on high-precision hardware devices, such as motion capture systems, depth cameras, etc. These devices can provide accurate 3D coordinate data, but their high cost, complex deployment and strong dependence on the environment and hardware limit their popularity in daily applications.

[0003] Nowadays, software developed by OpenSim and Qualisys can be used for sports performance research, but the former is open source and the built-in fixed data research lacks flexibility, while the latter market software is expensive and many intermediate data are difficult to provide. Therefore, it is still a technical challenge to obtain data of any gait model for open research. Existing methods generally lack efficient and low-cost solutions, especially in diversified gaits and large-scale application scenarios, and their performance still needs to be improved. Summary of the invention

[0004] The present invention provides a method and system for obtaining three-dimensional coordinates of an arbitrary gait model, thereby effectively solving the problems pointed out in the background technology.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: A method for obtaining three-dimensional coordinates of an arbitrary gait model, comprising: collecting and processing human motion information based on multi-view video data, extracting two-dimensional key point data of the human body by detecting key points of the human body on the collected video, and performing multi-view synchronization and character association; Based on the preset camera calibration parameters, the two-dimensional key point data is converted into three-dimensional space coordinates through triangulation calculation; Drawing custom markers in a static skeleton model, loading the static skeleton model, setting properties of the custom markers and the association between the custom markers and the parent framework, and scaling the static skeleton model according to the body shape characteristics of the subject; Based on the three-dimensional space coordinates and in combination with the inverse kinematics model, the static skeleton model is driven to generate a dynamic skeleton according to the actual movement of the subject, and the custom markers on the dynamic skeleton are updated in real time with the dynamic movement of the skeleton; The three-dimensional spatial dynamic coordinates of the custom marker point in the entire movement process are obtained, the three-dimensional dynamic coordinates of the custom marker point are calculated in real time, and the dynamic coordinates of the custom marker point are associated with the corresponding time point to generate dynamic position time series data of the custom marker point.

[0006] Furthermore, based on the multi-view video data, human motion information is collected and processed, human key point detection is performed on the collected video, two-dimensional key point data of the human body is extracted, and multi-view synchronization and character association are performed, including: Collecting motion video data of the subject through multiple synchronous video cameras, and performing time synchronization processing on the video data; Using the human key point detection algorithm, the two-dimensional key point coordinates of the subject are extracted from the video of each perspective; The two-dimensional key point data from different perspectives are paired and associated in time sequence, and the key point data at each moment corresponds to the body part of the same subject.

[0007] Furthermore, based on the preset camera calibration parameters, the two-dimensional key point data is converted into three-dimensional space coordinates through triangulation calculation, including: Determine the projection matrix of each camera device using a pre-calibrated camera intrinsic and extrinsic parameter model; Input the multi-view 2D key point data into the triangulation algorithm and calculate the position of each key point in 3D space; The calculated three-dimensional coordinate data is checked for consistency, points with large calculation errors are eliminated and the results are optimized to generate accurate three-dimensional space coordinate data.

[0008] Furthermore, custom markers are drawn in the static skeleton model, the static skeleton model is loaded, the attributes of the custom markers and the association between the custom markers and the parent framework are set, and the static skeleton model is scaled and adjusted according to the body shape characteristics of the subject, including: Defining the positions of custom markers in the static skeleton model and assigning a unique identifier to each marker; Setting the properties of the custom marker point, the properties including the name of the custom marker point, the associated bone part, and whether it is fixed to the bone node and joint center; Scaling and adjusting the static skeleton model according to the subject's body shape parameters to make it match the actual human body proportions; The adjusted static skeleton model is loaded through a visualization tool to check whether the position and attributes of the custom markers meet the requirements of gait analysis.

[0009] Furthermore, based on the three-dimensional space coordinates and in combination with the inverse kinematics model, the static skeleton model is driven to generate a dynamic skeleton according to the actual movement of the subject, and the custom markers on the dynamic skeleton are updated in real time with the dynamic movement of the skeleton, including: Using three-dimensional spatial coordinate data, extract the joint position information and motion trajectory at each time point; Based on the principle of inverse kinematics, the joint motion data is mapped onto the static skeleton model to generate a dynamic skeleton model that conforms to the actual motion of the subject; Following the dynamic movement of the skeleton model, the three-dimensional spatial position of the custom marker is updated in real time, so that the trajectory of the custom marker is consistent with the movement of the parent framework; By verifying and adjusting the dynamic marker point data, the complete motion data of the marker points in the dynamic skeleton model is finally output.

[0010] Furthermore, the three-dimensional spatial dynamic coordinates of the custom marker in the entire motion process are obtained, the three-dimensional dynamic coordinates of the custom marker are calculated in real time, and the dynamic coordinates of the custom marker are associated with the corresponding time points to generate dynamic position time series data of the custom marker, including: According to the dynamic movement of the skeleton model, the spatial coordinates of the custom markers at each time point are extracted; The three-dimensional spatial coordinates of the marking points are integrated into time series data in chronological order, and the time series data is smoothed to generate a complete dynamic position trajectory of the marking points.

[0011] Furthermore, the three-dimensional space coordinates obtained by the triangulation calculation are further enhanced, including: Correcting the three-dimensional spatial coordinates based on a human anatomical model until the three-dimensional spatial coordinates conform to the spatial relationship of the anatomical structure; Eliminate noise and outliers in data using smoothing and optimization algorithms; According to the physical characteristics of the bone nodes and the proportions of the human body, the positions of the joints and bone nodes are refined and adjusted.

[0012] Furthermore, the generation process of the dynamic skeleton is further optimized, and the abnormal motion trajectory is adjusted through an adaptive correction algorithm, including: Extracting historical motion trajectory data of the dynamic skeleton and custom markers, building a time series model based on the long short-term memory network method, and predicting the motion trajectory at future time points; Compare the predicted trajectory with the actual 3D coordinates in real time, and use anomaly detection algorithms based on statistical analysis and machine learning to identify anomalies in motion; An adaptive correction mechanism is introduced to make local adjustments to abnormal points by referring to the overall skeleton trend and the trajectory relationship of adjacent markers, and the trajectory is completed using data fusion method; Combined with the corrected marker point trajectory, the dynamic motion of the skeleton joints is recalculated, the matching of the corrected marker point trajectory and the global dynamic skeleton model is verified, and the final optimized dynamic skeleton and marker point motion data are output.

[0013] A system for acquiring three-dimensional coordinates of an arbitrary gait model, the system comprising: The two-dimensional key point extraction module collects and processes human motion information based on multi-view video data, detects human key points in the collected video, extracts the two-dimensional key point data of the human body, and performs multi-view synchronization and character association; The key point dimension conversion module converts the two-dimensional key point data into three-dimensional space coordinates through triangulation calculation based on the preset camera calibration parameters; A skeleton model adjustment module, which draws custom markers in a static skeleton model, loads the static skeleton model, sets the properties of the custom markers and the association between the custom markers and the parent framework, and scales the static skeleton model according to the body shape characteristics of the subject; A dynamic skeleton generation module, based on the three-dimensional space coordinates and in combination with an inverse kinematics model, drives the static skeleton model to generate a dynamic skeleton according to the actual movement of the subject, and the custom markers on the dynamic skeleton are updated in real time with the dynamic movement of the skeleton; The marking point dynamic calculation module obtains the three-dimensional spatial dynamic coordinates of the custom marking point during the entire movement process, calculates the three-dimensional dynamic coordinates of the custom marking point in real time, and associates the dynamic coordinates of the custom marking point with the corresponding time point to generate the dynamic position time series data of the custom marking point.

[0014] Furthermore, the two-dimensional key point extraction module includes: A video acquisition unit, which acquires motion video data of the subject through multiple synchronous camera devices and performs time synchronization processing on the video data; The key point detection unit uses a human key point detection algorithm to extract the two-dimensional key point coordinates of the subject from the video of each viewpoint; The key point association unit pairs and associates the two-dimensional key point data from different perspectives in chronological order, and the key point data at each moment corresponds to the body part of the same subject.

[0015] The technical solution of the present invention can achieve the following technical effects: It effectively solves the problems of high cost, complex deployment and strong environmental dependence in the existing technology, and realizes a low-cost, highly flexible three-dimensional coordinate acquisition method suitable for diversified gait analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A schematic diagram of a method for obtaining three-dimensional coordinates of an arbitrary gait model; Figure 2 A schematic diagram of the process of extracting two-dimensional key point data of the human body and performing multi-view synchronization and character association; Figure 3 A schematic diagram of the process of converting two-dimensional key point data into three-dimensional space coordinates; Figure 4 A schematic diagram of the process of drawing custom markers and scaling a static skeleton model; Figure 5 A schematic diagram of a process for generating a dynamic skeleton according to the actual movement of a subject by driving a static skeleton model in combination with an inverse kinematics model; Figure 6 A schematic diagram of the process of generating dynamic location time series data of custom markers; Figure 7 A schematic diagram of a process for further enhancing three-dimensional space coordinates; Figure 8 Schematic diagram of the process of adjusting abnormal motion trajectory through adaptive correction algorithm. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0020] Embodiment 1 like Figure 1 As shown, the present invention provides a method for obtaining three-dimensional coordinates of an arbitrary gait model, the method comprising: S1: Based on multi-view video data, human motion information is collected and processed. By detecting key points of the collected video, the two-dimensional key point data of the human body is extracted, and multi-view synchronization and character association are performed; Specifically, by synchronizing the recording time of multiple cameras, we ensure that the data of each perspective video is completely aligned on the timeline, eliminating the data inconsistency problem caused by time differences; at the same time, we use the human key point detection algorithm to process the video of each perspective and extract the key point coordinates of the subject's body, including the position information of joints and limbs. These two-dimensional key point data will be further correlated and matched under multiple perspectives to ensure that the key points of different perspectives correspond to the body parts of the same person, avoiding the confusion of multiple targets. Through this series of operations, a complete and accurate multi-perspective two-dimensional key point data set can be generated.

[0021] S2: Based on the preset camera calibration parameters, the two-dimensional key point data is converted into three-dimensional space coordinates through triangulation calculation; Specifically, using preset camera calibration parameters, the two-dimensional key point data extracted from multiple perspectives are accurately converted into three-dimensional space coordinates through triangulation calculation. This step is the key transformation process from plane data to spatial data. By pre-calibrating the internal and external parameters of each camera device (including focal length, optical axis, camera position and direction, etc.), the projection matrix of each camera device can be constructed to provide the necessary geometric constraints for triangulation calculation; then, based on the two-dimensional key point data from multiple perspectives, the triangulation algorithm is applied to calculate the position of each key point in three-dimensional space, thereby forming three-dimensional space coordinate data with depth information.

[0022] S3: drawing custom markers in the static skeleton model, loading the static skeleton model, setting the properties of the custom markers and the relationship between the custom markers and the parent framework, and scaling the static skeleton model according to the body shape characteristics of the subject; Specifically, by drawing custom markers in the static skeleton model and defining their properties and associations, the positions of the markers on the skeleton and their attachment to the skeletal framework are clarified to ensure that the markers can accurately change with the skeleton during movement. At the same time, the skeleton model is scaled according to the subject's body characteristics (such as height, limb length, and joint proportions) so that the proportions of the skeleton model are highly consistent with the actual human body structure, providing accurate and reliable basic support for subsequent dynamic skeleton generation and gait analysis.

[0023] S4: Based on the three-dimensional space coordinates and combined with the inverse kinematics model, the static skeleton model is driven to generate a dynamic skeleton according to the actual movement of the subject. The custom markers on the dynamic skeleton are updated in real time with the dynamic movement of the skeleton. Specifically, by combining the three-dimensional spatial coordinates and the inverse kinematics model, the static skeleton model is converted into a dynamic skeleton that can reflect the actual movement of the subject. The rotation angles and displacement paths of the skeleton joints are calculated through the inverse kinematics model, so that the skeleton can be dynamically updated according to the actual motion trajectory of the subject. At the same time, the custom marker points are kept consistent with the dynamic movement of the skeleton, and their three-dimensional spatial positions are updated in real time to ensure that the motion trajectory of the marker points accurately reflects the actual motion characteristics of the subject.

[0024] S5: Obtain the three-dimensional spatial dynamic coordinates of the custom marker during the entire movement process, calculate the three-dimensional dynamic coordinates of the custom marker in real time, and associate the dynamic coordinates of the custom marker with the corresponding time point to generate dynamic position time series data of the custom marker.

[0025] Specifically, the three-dimensional dynamic coordinates of the custom markers during the entire movement process are extracted from the dynamic skeleton model, and these coordinates are associated with the corresponding time points to generate complete dynamic position time series data. This time series data can accurately describe the motion trajectory of the markers. At the same time, through real-time calculation and time point association, the continuity and timing consistency of the marker trajectory can be ensured, thereby improving the integrity and practicality of the motion data.

[0026] The present invention effectively solves the problems of high cost, complex deployment and strong environmental dependence in the prior art, and realizes a three-dimensional coordinate acquisition method with low cost, high flexibility and applicability to diversified gait analysis.

[0027] As a preferred embodiment of the above, Figure 2 As shown, step S1, based on multi-view video data, collects and processes human motion information, detects human key points on the collected video, extracts two-dimensional key point data of the human body, and performs multi-view synchronization and character association, including: S11: collecting motion video data of the subject through multiple synchronous camera devices, and performing time synchronization processing on the video data; S12: Using the human key point detection algorithm, extract the two-dimensional key point coordinates of the subject from the video of each perspective; S13: Pair and associate the two-dimensional key point data from different perspectives in time sequence, and the key point data at each moment corresponds to the body part of the same subject.

[0028] Specifically, the motion video data of the subject is collected through multiple synchronized cameras, and the video data is time-synchronized to ensure that the time axes of the videos of each perspective are aligned. The cameras are arranged around the subject's motion area to cover all motion perspectives, and a global synchronization signal (such as NTP time synchronization or hardware trigger signal) is used to synchronize all cameras. An accurate timestamp is added to each frame of the collected video to ensure that it corresponds to the same motion time point on the timeline. Subsequently, the human key point detection algorithm (such as OpenPose or HRNet) is used to process the video frames one by one to extract the subject's two-dimensional key point data. The key points are represented in the form of two-dimensional coordinates (x, y) and are numbered according to the body parts, such as head, shoulder, elbow, etc. The detected two-dimensional key point data are stored in a structured data format (such as JSON or CSV) by frame. Each record contains a timestamp, a key point number and its two-dimensional coordinates. Finally, the two-dimensional key point data from different perspectives are paired and associated in chronological order. Based on multi-perspective geometric constraints and projection matrices, the matching probability of key points from different perspectives at each moment is calculated, and the identity association algorithm is used to ensure that the key points correspond to the same body part of the subject, generating a two-dimensional key point data set with consistent time and accurate matching.

[0029] As a preferred embodiment of the above, Figure 3 As shown, step S2, based on the preset camera calibration parameters, converts the two-dimensional key point data into three-dimensional space coordinates through triangulation calculation, including: S21: Determine the projection matrix of each camera device using a pre-calibrated camera internal and external parameter model; S22: input the multi-view two-dimensional key point data into a triangulation algorithm to calculate the position of each key point in the three-dimensional space; S23: Perform consistency check on the calculated three-dimensional coordinate data, remove points with large calculation errors and optimize the results to generate accurate three-dimensional space coordinate data.

[0030] First, in the experimental environment, all cameras are accurately calibrated to obtain the intrinsic parameters (such as focal length, optical center) and extrinsic parameters (such as the rotation matrix and displacement vector of the camera) of each device. These parameters are used to construct the projection matrix of the camera device, and the projection mapping relationship of the three-dimensional point in the camera is expressed as a mathematical model to ensure that the spatial geometric relationship of each camera device is accurately described; then, for each time frame, the two-dimensional key point data from multiple cameras are mapped to the three-dimensional space coordinates through the projection matrix, and the three-dimensional position of each key point is calculated through the geometric intersection of multiple perspectives using the triangulation principle, and the possible redundant data and perspective difference problems are solved by the least squares method or other optimization algorithms, thereby generating preliminary three-dimensional coordinate data; finally, by performing geometric and temporal continuity analysis on the three-dimensional coordinate data, noise points and data with large errors (such as abnormal points caused by occlusion or detection errors) are eliminated, and the three-dimensional coordinates of continuous frames are further optimized using a motion smoothing algorithm to ensure the temporal and spatial continuity of the coordinates and generate high-precision three-dimensional space coordinate data.

[0031] As a preferred embodiment of the above, Figure 4 As shown, step S3, drawing custom markers in the static skeleton model, loading the static skeleton model, setting the attributes of the custom markers and the association between the custom markers and the parent framework, and scaling the static skeleton model according to the body shape characteristics of the subject, including: S31: define the position of the custom marker points in the static skeleton model, and assign a unique identifier to each marker point; S32: Setting the properties of the custom marker point, including the name of the custom marker point, the associated bone part, and whether it is fixed at the bone node and joint center; S33: scaling and adjusting the static skeleton model according to the subject's body shape parameters to match the actual human body proportions; S34: Load the adjusted static skeleton model through the visualization tool to check whether the position and attributes of the custom markers meet the requirements of gait analysis.

[0032] Specifically, first, according to the human anatomical structure, the positions of the markers are specified at key parts of the skeleton model (such as joints, limb centers, or other points required for analysis). The position of the marker point is expressed in the form of coordinates, and a unique identifier is assigned to each marker point to ensure that the marker point can be clearly identified and referenced in the model; then, the properties of the custom marker point are set. The properties of each marker point include its name (such as "left knee joint marker point"), the associated bone part (such as the joint center connecting the femur and tibia), and whether it is fixed to the bone node or joint center. By setting these properties, the function and dynamic association relationship of the marker point on the skeleton can be clarified, providing the necessary logical constraints for the subsequent generation of the dynamic skeleton; then, the subject's body parameters (such as height, limb length, joint proportion, etc.) are collected, and the overall and local sizes of the static skeleton model are scaled according to these parameters so that the proportion of the skeleton model is highly consistent with the subject's actual body structure. This step ensures that the skeleton model can accurately reflect the subject's body characteristics; finally, the modified skeleton model is loaded using 3D visualization software to verify the position and properties of the custom marker points in the model, check whether the marker point is located at the expected position, and whether its association with the parent class framework is correct. If it is found that the position or attribute setting of the marker point does not meet the requirements of gait analysis, the position or attribute of the marker point can be further modified through the adjustment tool.

[0033] As a preferred embodiment of the above, Figure 5 As shown, step S4, based on the three-dimensional space coordinates and in combination with the inverse kinematics model, drives the static skeleton model to generate a dynamic skeleton according to the actual movement of the subject, and the custom markers on the dynamic skeleton are updated in real time with the dynamic movement of the skeleton, including: S41: extracting joint position information and motion trajectory at each time point using the three-dimensional space coordinate data; S42: Based on the principle of inverse kinematics, the joint motion data is mapped onto the static skeleton model to generate a dynamic skeleton model that conforms to the actual motion of the subject; S43: Following the dynamic movement of the skeleton model, the three-dimensional spatial position of the custom marker is updated in real time, so that the trajectory of the custom marker is consistent with the movement of the parent frame; S44: By verifying and adjusting the dynamic marker point data, the complete motion data of the marker points in the dynamic skeleton model is finally output.

[0034] Specifically, first, the generated three-dimensional spatial coordinate data is used as input to parse the three-dimensional position information of each joint node in the skeleton model, and the motion trajectory of each joint is extracted according to the time series. By analyzing the three-dimensional coordinate changes in consecutive frames, the dynamic motion path of the joint nodes is obtained to provide basic data for subsequent skeleton driving; then, the inverse kinematics algorithm is used to calculate the rotation angle and displacement path of the skeleton joints to ensure that the movement of the skeleton conforms to the human anatomy and physical constraints. By adjusting the joint angles in the skeleton model, the bone structure can accurately simulate the actual movement of the subject. This step converts the static skeleton model into a dynamic skeleton model so that it is consistent with the three-dimensional motion data. Then, during the dynamic skeleton movement, the custom marker points on the skeleton will update their positions synchronously with their associated bone nodes, and the three-dimensional dynamic coordinates of each marker point will be calculated in real time to ensure that its trajectory is consistent with the motion state of the parent frame, thereby reflecting the accurate dynamic position of the marker point in the actual movement of the subject; finally, the generated dynamic marker point trajectory is verified for data consistency and integrity, possible outliers are removed and smoothed to optimize the motion path of the marker point. The final output dynamic skeleton model and its marker point data contain complete time series information and three-dimensional space trajectory, which can be directly used for gait analysis, motion feature extraction and other application scenarios.

[0035] As a preferred embodiment of the above, Figure 6 As shown, step S5, obtaining the three-dimensional spatial dynamic coordinates of the custom marker during the entire movement process, calculating the three-dimensional dynamic coordinates of the custom marker in real time, and associating the dynamic coordinates of the custom marker with the corresponding time point to generate dynamic position time series data of the custom marker, including: S51: extracting the spatial coordinates of the custom markers at each time point according to the dynamic motion of the skeleton model; S52: Integrate the three-dimensional spatial coordinates of the marker points into time series data in chronological order, and perform smoothing on the time series data to generate a complete dynamic position trajectory of the marker points.

[0036] Specifically, first, the three-dimensional spatial coordinates of the custom markers are calculated in real time at each time point through the motion trajectory of the dynamic skeleton model. The coordinates of the markers are determined by their associated parent skeleton framework and dynamic motion state, such as joint angles, bone node positions, etc. As the skeleton moves dynamically, the system extracts the three-dimensional coordinates of the markers frame by frame to ensure that the position information of the markers accurately reflects the actual motion state of the subject; then, the extracted three-dimensional coordinate data is sorted according to the timestamp to form a continuous time series. In order to eliminate possible noise or mutations, a data smoothing algorithm (such as low-pass filtering or moving average) is used to process the time series to optimize the continuity and accuracy of the marker trajectory. After smoothing, the generated time series data can fully reflect the dynamic position changes of the markers during the entire motion process.

[0037] As a preferred embodiment of the above, Figure 7 As shown, the three-dimensional space coordinates obtained by triangulation calculation are further enhanced, including: A10: Correct the three-dimensional spatial coordinates based on the human anatomical model until they conform to the spatial relationship of the anatomical structure; A20: Eliminate noise and outliers in data using smoothing and optimization algorithms; A30: Make detailed adjustments to the positions of joints and bone nodes based on the physical properties of the bone nodes and the proportions of the human body.

[0038] Specifically, first, using the pre-built standard human anatomical model, the three-dimensional spatial coordinates generated by triangulation calculation are compared with the anatomical model to identify abnormal coordinate points that do not conform to the human body structure. For example, check whether the relative position and distance between joints are within the anatomically allowed range and whether the connection direction of the bones is reasonable. For coordinate points that deviate from anatomical rationality, correction is performed based on anatomical constraints to ensure the authenticity and consistency of the three-dimensional coordinate data with the human body structure. Then, smoothing processing (such as Kalman filtering, low-pass filtering or spline interpolation algorithm) is applied to the corrected three-dimensional coordinate data to eliminate the noise introduced by occlusion, equipment error or calculation deviation during movement. At the same time, combined with time series analysis, obvious abnormal points (such as sudden jumps or stationary points) are detected and removed, so that the temporal and spatial continuity of the coordinate data can be improved, thereby more realistically reflecting the movement trajectory. Then, based on the subject's body shape parameters (such as height, limb length and proportion), the positions of joints and bone nodes in the three-dimensional coordinates are adjusted to make them more compatible with the individualized human body model. At the same time, the kinematic constraints of the bones (such as joint angle range and rotational freedom) are used to optimize the spatial position of the joints in the three-dimensional coordinates to ensure the connection and movement rationality of the joints.

[0039] As a preferred embodiment of the above, Figure 8As shown in the figure, the generation process of the dynamic skeleton is further optimized, and the abnormal motion trajectory is adjusted through an adaptive correction algorithm, including: B10: Extract historical motion trajectory data of dynamic skeletons and custom markers, build a time series model based on the long short-term memory network method, and predict the motion trajectory at future time points; B20: Real-time comparison of predicted trajectory with actual 3D coordinates, using anomaly detection algorithms based on statistical analysis and machine learning to identify anomalies in motion; B30: Introducing an adaptive correction mechanism, by referring to the overall skeleton trend and the trajectory relationship of adjacent markers, local adjustments are made to abnormal points, and the trajectory is completed using a data fusion method; B40: Combine the corrected marker point trajectory to recalculate the dynamic motion of the skeleton joints, verify the matching of the corrected marker point trajectory with the global dynamic skeleton model, and output the final optimized dynamic skeleton and marker point motion data.

[0040] Specifically, firstly, the historical motion trajectory data of custom markers and joints are obtained from the dynamic skeleton model, and the trajectory is modeled as a time series. The long-term dependency capture capability of the LSTM network is used to analyze the time dependency and regularity of the motion trajectory, and the motion trend at future time points is predicted. The prediction result provides a reference path for the dynamic skeleton motion trajectory, which is helpful for correction when the motion is abnormal. Then, the LSTM predicted trajectory is compared point by point with the actually generated three-dimensional coordinates, and the deviation value between the two is calculated. The statistical analysis method (such as z-score analysis) or the machine learning anomaly detection algorithm (such as the isolation forest algorithm) is used to identify abnormal points in the trajectory that are sudden, discontinuous, or deviate from a reasonable range. The detected abnormal points The mark is marked as a target that needs to be corrected; then, the adaptive correction mechanism combines the global skeleton motion trend and the trajectory smoothness characteristics of adjacent marker points, and corrects the abnormal points through weighted data fusion (such as Bayesian filtering or Kalman filtering). The completion of the abnormal points generates a reasonable alternative trajectory through interpolation methods (such as spline interpolation), while ensuring the continuity of the corrected trajectory with the surrounding points; finally, the corrected marker point data is used as input to update the dynamic motion state of the skeleton joints, and the overall consistency verification is performed to ensure that the corrected marker point trajectory is consistent with the kinematic and dynamic characteristics of the skeleton model. The final dynamic skeleton and marker point motion data output have high precision, continuity and reliability, and can be used for gait analysis, motion feature extraction and other high-precision applications.

[0041] Embodiment 2 Based on the same inventive concept as the method for obtaining the three-dimensional coordinates of an arbitrary gait model in the aforementioned embodiment, the present invention further provides a system for obtaining the three-dimensional coordinates of an arbitrary gait model, the system comprising: The two-dimensional key point extraction module collects and processes human motion information based on multi-view video data, detects human key points in the collected video, extracts the two-dimensional key point data of the human body, and performs multi-view synchronization and character association; The key point dimension conversion module converts the two-dimensional key point data into three-dimensional space coordinates through triangulation calculation based on the preset camera calibration parameters; The skeleton model adjustment module draws custom markers in the static skeleton model, loads the static skeleton model, sets the properties of the custom markers and the relationship between the custom markers and the parent framework, and scales the static skeleton model according to the subject's body shape characteristics; The dynamic skeleton generation module, based on the three-dimensional space coordinates and combined with the inverse kinematics model, drives the static skeleton model to generate a dynamic skeleton according to the actual movement of the subject. The custom markers on the dynamic skeleton are updated in real time with the dynamic movement of the skeleton. The marker point dynamic calculation module obtains the three-dimensional spatial dynamic coordinates of the custom marker point during the entire movement process, calculates the three-dimensional dynamic coordinates of the custom marker point in real time, and associates the dynamic coordinates of the custom marker point with the corresponding time point to generate the dynamic position time series data of the custom marker point.

[0042] The above system in the present invention can effectively implement a method for obtaining the three-dimensional coordinates of any gait model, and the technical effects that can be achieved are as described in the above embodiments, which will not be repeated here.

[0043] As a preferred embodiment of the above, the two-dimensional key point extraction module includes: A video acquisition unit, which acquires motion video data of the subject through multiple synchronous camera devices and performs time synchronization processing on the video data; The key point detection unit uses a human key point detection algorithm to extract the two-dimensional key point coordinates of the subject from the video of each viewpoint; The key point association unit pairs and associates the two-dimensional key point data from different perspectives in chronological order, and the key point data at each moment corresponds to the body part of the same subject.

[0044] Although the present application has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the accompanying drawings are merely exemplary illustrations of the present application as defined herein, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application.

[0045] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application is intended to include these modifications and variations.

Claims

1. A method for obtaining the three-dimensional coordinates of an arbitrary gait model, characterized in that: include: Based on multi-view video data, human motion information is collected and processed. By detecting key points of the collected video, the two-dimensional key point data of the human body is extracted, and multi-view synchronization and character association are performed. Based on the preset camera calibration parameters, the two-dimensional key point data is converted into three-dimensional space coordinates through triangulation calculation; Drawing custom markers in a static skeleton model, loading the static skeleton model, setting properties of the custom markers and the association between the custom markers and the parent framework, and scaling the static skeleton model according to the body shape characteristics of the subject; Based on the three-dimensional space coordinates and in combination with the inverse kinematics model, the static skeleton model is driven to generate a dynamic skeleton according to the actual movement of the subject, and the custom markers on the dynamic skeleton are updated in real time with the dynamic movement of the skeleton; The three-dimensional spatial dynamic coordinates of the custom marker point in the entire movement process are obtained, the three-dimensional dynamic coordinates of the custom marker point are calculated in real time, and the dynamic coordinates of the custom marker point are associated with the corresponding time point to generate dynamic position time series data of the custom marker point.

2. The method for obtaining three-dimensional coordinates of an arbitrary gait model according to claim 1, characterized in that: Based on multi-view video data, human motion information is collected and processed. By detecting key points of the collected video, the two-dimensional key point data of the human body is extracted, and multi-view synchronization and character association are performed, including: Collecting motion video data of the subject through multiple synchronous video cameras, and performing time synchronization processing on the video data; Using the human key point detection algorithm, the two-dimensional key point coordinates of the subject are extracted from the video of each perspective; The two-dimensional key point data from different perspectives are paired and associated in time sequence, and the key point data at each moment corresponds to the body part of the same subject.

3. The method for obtaining three-dimensional coordinates of an arbitrary gait model according to claim 1, characterized in that: Based on the preset camera calibration parameters, the two-dimensional key point data is converted into three-dimensional space coordinates through triangulation calculation, including: Determine the projection matrix of each camera device using a pre-calibrated camera intrinsic and extrinsic parameter model; Input the multi-view 2D key point data into the triangulation algorithm and calculate the position of each key point in 3D space; The calculated three-dimensional coordinate data is checked for consistency, points with large calculation errors are eliminated and the results are optimized to generate accurate three-dimensional space coordinate data.

4. The method for obtaining three-dimensional coordinates of an arbitrary gait model according to claim 1, characterized in that: Drawing custom markers in a static skeleton model, loading the static skeleton model, setting properties of the custom markers and the association between the custom markers and the parent framework, and scaling the static skeleton model according to the body shape characteristics of the subject, including: Defining the positions of custom markers in the static skeleton model and assigning a unique identifier to each marker; Setting the properties of the custom marker point, the properties including the name of the custom marker point, the associated bone part, and whether it is fixed to the bone node and joint center; Scaling and adjusting the static skeleton model according to the subject's body shape parameters to make it match the actual human body proportions; The adjusted static skeleton model is loaded through a visualization tool to check whether the position and attributes of the custom markers meet the requirements of gait analysis.

5. The method for obtaining three-dimensional coordinates of an arbitrary gait model according to claim 1, characterized in that: Based on the three-dimensional space coordinates and in combination with the inverse kinematics model, the static skeleton model is driven to generate a dynamic skeleton according to the actual movement of the subject, and the custom markers on the dynamic skeleton are updated in real time with the dynamic movement of the skeleton, including: Using three-dimensional spatial coordinate data, extract the joint position information and motion trajectory at each time point; Based on the principle of inverse kinematics, the joint motion data is mapped onto the static skeleton model to generate a dynamic skeleton model that conforms to the actual motion of the subject; Following the dynamic movement of the skeleton model, the three-dimensional spatial position of the custom marker is updated in real time, so that the trajectory of the custom marker is consistent with the movement of the parent framework; By verifying and adjusting the dynamic marker point data, the complete motion data of the marker points in the dynamic skeleton model is finally output.

6. The method for obtaining three-dimensional coordinates of an arbitrary gait model according to claim 1, characterized in that: Acquiring the three-dimensional spatial dynamic coordinates of the custom marker during the entire motion process, calculating the three-dimensional dynamic coordinates of the custom marker in real time, and associating the dynamic coordinates of the custom marker with the corresponding time point to generate dynamic position time series data of the custom marker, including: According to the dynamic movement of the skeleton model, the spatial coordinates of the custom markers at each time point are extracted; The three-dimensional spatial coordinates of the marking points are integrated into time series data in chronological order, and the time series data is smoothed to generate a complete dynamic position trajectory of the marking points.

7. The method for obtaining three-dimensional coordinates of an arbitrary gait model according to claim 1, characterized in that: Further enhancing the three-dimensional space coordinates obtained by the triangulation calculation, including: Correcting the three-dimensional spatial coordinates based on a human anatomical model until the three-dimensional spatial coordinates conform to the spatial relationship of the anatomical structure; Eliminate noise and outliers in data using smoothing and optimization algorithms; According to the physical characteristics of the bone nodes and the proportions of the human body, the positions of the joints and bone nodes are refined and adjusted.

8. The method for obtaining three-dimensional coordinates of an arbitrary gait model according to claim 1, characterized in that: The dynamic skeleton generation process is further optimized, and the abnormal motion trajectory is adjusted through an adaptive correction algorithm, including: Extracting historical motion trajectory data of the dynamic skeleton and custom markers, building a time series model based on the long short-term memory network method, and predicting the motion trajectory at future time points; Compare the predicted trajectory with the actual 3D coordinates in real time, and use anomaly detection algorithms based on statistical analysis and machine learning to identify anomalies in motion; An adaptive correction mechanism is introduced to make local adjustments to abnormal points by referring to the overall skeleton trend and the trajectory relationship of adjacent markers, and the trajectory is completed using data fusion method; Combined with the corrected marker point trajectory, the dynamic motion of the skeleton joints is recalculated, the matching of the corrected marker point trajectory and the global dynamic skeleton model is verified, and the final optimized dynamic skeleton and marker point motion data are output.

9. A system for obtaining three-dimensional coordinates of any gait model, characterized in that: The system comprises: The two-dimensional key point extraction module collects and processes human motion information based on multi-view video data, detects human key points in the collected video, extracts the two-dimensional key point data of the human body, and performs multi-view synchronization and character association; The key point dimension conversion module converts the two-dimensional key point data into three-dimensional space coordinates through triangulation calculation based on the preset camera calibration parameters; A skeleton model adjustment module, which draws custom markers in a static skeleton model, loads the static skeleton model, sets the properties of the custom markers and the association between the custom markers and the parent framework, and scales the static skeleton model according to the body shape characteristics of the subject; A dynamic skeleton generation module, based on the three-dimensional space coordinates and in combination with an inverse kinematics model, drives the static skeleton model to generate a dynamic skeleton according to the actual movement of the subject, and the custom markers on the dynamic skeleton are updated in real time with the dynamic movement of the skeleton; The marking point dynamic calculation module obtains the three-dimensional spatial dynamic coordinates of the custom marking point during the entire movement process, calculates the three-dimensional dynamic coordinates of the custom marking point in real time, and associates the dynamic coordinates of the custom marking point with the corresponding time point to generate the dynamic position time series data of the custom marking point.

10. The system for obtaining three-dimensional coordinates of any gait model according to claim 9, characterized in that: The two-dimensional key point extraction module comprises: A video acquisition unit, which acquires motion video data of the subject through multiple synchronous camera devices and performs time synchronization processing on the video data; The key point detection unit uses a human key point detection algorithm to extract the two-dimensional key point coordinates of the subject from the video of each viewpoint; The key point association unit pairs and associates the two-dimensional key point data from different perspectives in chronological order, and the key point data at each moment corresponds to the body part of the same subject.

Images