Three-dimensional human motion capture method and apparatus

Abstract

The application relates to a three-dimensional human body motion capture method and device, which comprises the following steps: multi-view synchronous shooting and inter-frame timestamp alignment are performed on a target human body to obtain a synchronous image frame group, a joint point detection is performed on a human body region in the synchronous image frame group to obtain two-dimensional joint point coordinates, multi-view triangulation is performed on the two-dimensional joint point coordinates based on camera equipment calibration parameters to obtain three-dimensional joint point coordinates, a limb segment connection relationship is established based on the three-dimensional joint point coordinates and a human body skeleton topological structure to obtain a limb segment connection graph, a rotation angle of each limb segment in the limb segment connection graph is solved to obtain limb segment posture data, and the limb segment posture data is time-sequentially arranged according to the acquisition time of the synchronous image frame group to obtain three-dimensional motion data, so that the technical problem of how to improve the accuracy of three-dimensional joint point reconstruction and the robustness of a limb segment connection relationship under the condition that the multi-view two-dimensional joint point detection result is uncertain is solved.

Description

Technical Field

[0001] This invention relates to the field of human motion technology, and in particular to a three-dimensional human motion capture method and apparatus. Background Technology

[0002] In recent years, with the development of computer vision and graphics technologies, methods for 3D human motion capture based on multi-view images have been widely applied. These methods typically involve deploying multiple calibrated cameras to simultaneously acquire video data of the target human body from different angles, and aligning image frames from each viewpoint using timestamps to form frame groups. Subsequently, a joint detection algorithm is applied to extract the two-dimensional coordinates of key human points from each viewpoint image. Combined with the intrinsic and extrinsic parameters of the camera equipment, the three-dimensional positions of the joints are reconstructed using triangulation principles from multi-view geometry. Finally, based on the topological structure of the human skeleton, these three-dimensional joints are connected to form limb links, achieving modeling of human posture and reconstruction of motion sequences. This technology has been applied in fields such as virtual reality, motion analysis, human-computer interaction, and film and animation production.

[0003] However, in practical implementation, the local false detections or occlusions caused by the independently detected 2D joint coordinates from different perspectives often lead to positioning deviations. Direct triangulation significantly affects the accuracy of 3D joint coordinate reconstruction, especially in scenarios involving limb intersections or multi-person interactions, easily causing joint misalignment and limb connection errors, which in turn leads to distortion in subsequent pose calculations. Therefore, how to improve the accuracy of 3D joint reconstruction and the robustness of limb connection relationships under the uncertainty of multi-view 2D joint detection results has become a key issue restricting the stable application of this method in complex scenarios. Summary of the Invention

[0004] The main technical problem addressed in this application is to provide a three-dimensional human motion capture method and apparatus, which solves the technical problem of how to improve the accuracy of three-dimensional joint reconstruction and the robustness of limb connection relationships when there is uncertainty in the multi-view two-dimensional joint detection results.

[0005] To solve the above-mentioned technical problems, this application adopts a three-dimensional human motion capture method, which includes the following steps: Multi-view synchronous shooting of the target human body and frame-to-frame timestamp alignment are performed to obtain a synchronous image frame group. Joint point detection is performed on the human body region in the synchronous image frame group to obtain two-dimensional joint point coordinates. Based on the preset camera equipment calibration parameters, the coordinates of the two-dimensional joint points are measured by multi-view triangulation to obtain the coordinates of the three-dimensional joint points. Based on the topological structure of the human skeleton, the limb connection relationship of the three-dimensional joint point coordinates is established to obtain the limb connection diagram. The rotation angle of each limb in the limb connection diagram is calculated to obtain limb posture data. The limb posture data is then arranged in time sequence according to the acquisition time of the synchronous image frame group to obtain three-dimensional motion data.

[0006] Furthermore, the target human body is simultaneously captured from multiple perspectives and the inter-frame timestamps are aligned to obtain a synchronized image frame group, including: Simultaneously start shooting the target human body using multiple preset camera devices and add timestamps to obtain time-stamped image frames; Timestamp comparison is performed on time-stamped image frames with the same frame number from different viewpoints. If the timestamp deviation of image frames with the same frame number from different viewpoints is within a preset threshold range, the time-stamped image frame is retained. If it exceeds the preset threshold range, the timestamps of the time-stamped image frames at the corresponding viewpoints and subsequent time-stamped image frames are linearly interpolated to adjust the timestamps so that the timestamp deviation of time-stamped image frames exceeding the preset threshold range from the timestamps of corresponding frames at other viewpoints is within the preset threshold range, thus obtaining a synchronized image frame group.

[0007] Furthermore, the step of performing joint point detection on the human body region in the synchronized image frame group to obtain two-dimensional joint point coordinates includes: By using pixel color differences and edge contour information, the human body region of each frame in the synchronized image frame group is separated from the background to obtain a human body segmentation image. The human body segmentation image is then grayscaled to obtain a grayscale human body image. The human joints in the grayscale human body image are located and marked. The marked joint image is obtained by using the positional features of the human joints in the grayscale human body image and the relative positional relationship between adjacent joints. Extract the pixel coordinates of each joint in the marked joint image, and then convert the pixel coordinates into two-dimensional joint coordinates in the image coordinate system.

[0008] Furthermore, the step of performing multi-view triangulation on the two-dimensional joint coordinates based on preset camera equipment calibration parameters to obtain three-dimensional joint coordinates includes: For the two-dimensional joint point coordinates, combined with the camera intrinsic parameters in the preset camera equipment calibration parameters, the two-dimensional joint point coordinates are transformed from the image coordinate system to the camera coordinate system corresponding to each viewpoint, so as to obtain the camera coordinate points under each viewpoint. Noise filtering is performed on the camera coordinate points under each viewpoint to obtain clean camera coordinate points under each viewpoint. Based on clean camera coordinates from each viewpoint, and according to the principle of spatial line of sight intersection, the intersection points of corresponding joints in the spatial line of sight under different viewpoints are calculated to obtain the initial three-dimensional joint coordinates. The initial three-dimensional joint coordinates are processed to unify the three-dimensional coordinates calculated from different perspectives into the same world coordinate system, thus obtaining the three-dimensional joint coordinates.

[0009] Furthermore, based on clean camera coordinates from each viewpoint, and according to the principle of spatial line-of-sight intersection, the intersection points of corresponding joint points in the spatial line of sight under different viewpoints are calculated to obtain the initial three-dimensional joint point coordinates, including: Based on the clean camera coordinates at each viewpoint, and combined with the optical center coordinates of the corresponding camera device at each viewpoint, the clean camera coordinates and the optical center coordinates are connected and extended to obtain the spatial rays at each viewpoint, including the ray origin coordinates and the ray direction vector. Based on the principle of spatial line of sight intersection, spatial rays from different perspectives are paired up in pairs, and the common perpendicular segment of the paired spatial rays is solved to obtain the ray pair common perpendicular segment including the coordinates of the two endpoints of the common perpendicular segment and the length value of the segment. The midpoints of the two endpoints of the common perpendicular segment of the ray pair are calculated to obtain the candidate spatial points of each ray pair; Assign confidence scores to the candidate spatial points according to the length of the line segment of the common perpendicular line segment of the ray, and obtain candidate points with confidence scores; Based on the confidence level of the candidate points with confidence, a weighted average is calculated for multiple candidate points with confidence corresponding to the same joint point to obtain the initial three-dimensional joint point coordinates.

[0010] Furthermore, the step of establishing limb connection relationships based on the three-dimensional joint coordinates according to the human skeletal topology to obtain a limb connection diagram includes: The coordinates of the three-dimensional joint points are labeled with joint point types based on human anatomical features to obtain a set of labeled joint points, and the distance data between each joint point in the set of labeled joint points is calculated. Based on the distance data between each joint and the preset limb length threshold in the human skeletal topology, the labeled joint set is initially connected and matched, and joints whose distance data is within the corresponding limb length threshold range are connected to obtain initial connected joint pairs. Based on the initial connection joint pairs, the connection order of each limb segment is determined according to the parent-child hierarchical relationship of limb segments in the human skeletal topology. Based on the connection order of each limb segment, a limb connection diagram containing each limb segment and its connection relationship is drawn.

[0011] Furthermore, the step of determining the connection order of each limb segment based on the initial connection joint pair and according to the parent-child hierarchy of limb segments in the human skeletal topology includes: The initial connected joint point pair is subjected to trunk joint point identification to obtain the root joint point position, and the hierarchical distance of the initial connected joint point pair is calculated based on the root joint point position to obtain the joint point hierarchical value. Based on the joint level value, the parent-child relationship of the initial connected joint pairs is marked. The joint with the smaller level value in each limb segment is marked as the parent joint and the joint with the larger level value is marked as the child joint, thus obtaining the limb segment parent-child marked pairs. The limb segment parent-child marked pairs are then divided into branch paths according to the human skeletal topology, and limb segments belonging to the same kinetic chain are grouped into the same transmission branch, thus obtaining branch limb group. Each segment in the branch segment group is sequentially numbered from the root joint to the terminal joint to obtain the segment number within the branch. Based on the segment number within the branch and the preset priority of each branch segment group, all segments are uniformly sorted to obtain the connection order of each segment.

[0012] Furthermore, the step of calculating the rotation angle of each limb segment in the limb connection diagram to obtain limb posture data includes: Define coordinate axes for each limb segment in the limb segment connection diagram, and determine local coordinate axes for each limb segment based on human anatomical orientation; Using vector calculation methods, the rotation angle between the local coordinate axes of adjacent limb segments is calculated at the joints between adjacent limb segments. Based on the rules of human joint kinematics, the rotation angle is constrained and checked, and the rotation angle that exceeds the preset range of motion of the joint is adjusted in order to determine the limb posture data of each limb segment in three-dimensional space.

[0013] Furthermore, the method of calculating the rotation angle between the local coordinate axes of adjacent limb segments at the joints using vector calculation includes: A reference plane is constructed for the local coordinate axes of the parent limb segment in the adjacent limb segments. Based on the three orthogonal axis vectors of the local coordinate axes of the parent limb segment, three mutually perpendicular reference planes are formed by combining them in pairs, thus obtaining the joint transition coordinate system. By using vector calculation methods, the principal direction vectors of the local coordinate axes of the sub-limb segments in the adjacent limb segments are orthogonally projected onto each reference plane of the joint transition coordinate system to obtain the projection component vectors on each reference plane. The angle between the projection component vector and the corresponding reference axis vector in the local coordinate axis of the parent limb segment is calculated to obtain the plane rotation angle in each reference plane, wherein the plane rotation angle includes flexion-extension angle, adduction-abduction angle and axial rotation angle; Based on the planar rotation angles in each reference plane, the flexion-extension angle, adduction-abduction angle, and axial rotation angle are ordered and combined according to the joint rotation sequence to obtain the rotation angle between the local coordinate axes of adjacent limb segments.

[0014] Furthermore, the step of arranging the limb posture data in time sequence according to the acquisition time of the synchronized image frame group to obtain three-dimensional motion data includes: The limb posture data is correlated frame by frame with the acquisition time of each frame in the synchronized image frame group to obtain a time-stamped posture frame sequence. Based on the acquisition time values ​​of each frame in the time-stamped attitude frame sequence, the time-stamped attitude frame sequence is arranged in ascending order according to the time sequence, and the limb attitude data of each frame after arrangement is connected end to end to form a continuous attitude change record. Based on the posture change records, three-dimensional motion data is obtained, including a sequence of rotation angle changes of each limb segment arranged in chronological order and the corresponding time axis information.

[0015] The present invention also provides a three-dimensional human motion capture device, comprising: The detection module is used to simultaneously capture the target human body from multiple perspectives and align the timestamps between frames to obtain a synchronized image frame group, and to perform joint point detection on the human body region in the synchronized image frame group to obtain two-dimensional joint point coordinates. The measurement module is used to perform multi-view triangulation on the coordinates of the two-dimensional joint points based on preset camera equipment calibration parameters to obtain the coordinates of the three-dimensional joint points, and to establish the limb connection relationship of the coordinates of the three-dimensional joint points based on the topology of the human skeleton to obtain the limb connection diagram. The arrangement module is used to calculate the rotation angle of each limb in the limb connection diagram to obtain limb posture data, and to arrange the limb posture data in time sequence according to the acquisition time of the synchronous image frame group to obtain three-dimensional motion data.

[0016] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above methods.

[0017] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the above methods.

[0018] The above scheme involves simultaneous multi-view shooting of the target human body and timestamp alignment between frames to obtain a synchronized image frame group. Joint point detection is performed on the human body region within the synchronized image frame group to obtain two-dimensional joint point coordinates. Based on the camera equipment calibration parameters, multi-view triangulation is performed on the two-dimensional joint point coordinates to obtain three-dimensional joint point coordinates. Based on the human skeletal topology, limb segment connections are established for the three-dimensional joint point coordinates to obtain a limb segment connection diagram. Rotation angles are calculated for each limb segment in the limb segment connection diagram to obtain limb posture data. This limb posture data is then arranged chronologically according to the acquisition time of the synchronized image frame group to obtain three-dimensional motion data. This solution addresses the technical challenge of improving the accuracy of three-dimensional joint point reconstruction and the robustness of limb segment connections when there is uncertainty in the results of multi-view two-dimensional joint point detection. It achieves rotation angle calculation for each limb segment in the limb segment connection diagram, converting spatial coordinates into joint motion parameters, which more closely reflects the actual human movement and facilitates subsequent applications in animation-driven and motion analysis scenarios. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the steps of a three-dimensional human motion capture method in one embodiment of the present invention; Figure 2 This is a structural block diagram of a three-dimensional human motion capture device according to an embodiment of the present invention; Figure 3 This is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0021] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] Specifically, the three-dimensional human motion capture method of this embodiment includes the following steps: like Figure 1As shown, Figure 1 This invention provides a three-dimensional human motion capture method, comprising the following steps: Step S1: Perform multi-view synchronous shooting and frame-to-frame timestamp alignment on the target human body to obtain a synchronous image frame group, and perform joint point detection on the human body region in the synchronous image frame group to obtain two-dimensional joint point coordinates.

[0024] Specifically, when performing multi-view synchronous shooting of a target human body, two or more camera devices need to be set up and positioned around the subject from different angles. Each camera is connected to the same trigger signal source or uses a hardware synchronization mechanism to ensure that the image acquisition time is strictly consistent. After the shooting is completed, the absolute timestamps or frame numbers recorded by each camera are matched with the pre-stored time alignment parameters to correct for minor offsets caused by transmission delays and other factors. This results in a time-aligned image set, i.e., a synchronized image frame group. Subsequently, in each frame, a convolutional neural network is used to locate the human body region and predict the pixel coordinates of several key points within that region, such as the two-dimensional positions of the shoulders, elbows, and knees. This process is called joint detection, and the output result is the two-dimensional joint coordinates. For example, in a motion capture laboratory environment, four 1080p cameras are used to record the subject's walking process at 60fps. After timestamp alignment, the OpenPose model is run frame by frame to extract the pixel positions of 18 joints per viewpoint, which serve as the basic data input for subsequent 3D reconstruction.

[0025] Step S2: Perform multi-view triangulation on the coordinates of the two-dimensional joints based on the preset camera equipment calibration parameters to obtain the coordinates of the three-dimensional joints, and establish the limb connection relationship of the coordinates of the three-dimensional joints based on the topology of the human skeleton to obtain the limb connection diagram.

[0026] Specifically, using the camera calibration parameters obtained beforehand through the checkerboard calibration method, including intrinsic and extrinsic rotation and translation matrices, the coordinates of two-dimensional joint points at the same moment from different perspectives are substituted into the collinearity equation to construct the projection rays of the corresponding spatial points. Then, the optimal intersection point of these rays in three-dimensional space is solved by minimizing the reprojection error. This process is called multi-view triangulation, and the output result is the three-dimensional joint point coordinates. Next, based on the inherent skeletal connection rules of the human body, such as the topological relationship of the elbow joint connecting the upper arm and forearm, and the hip joint connecting the torso and thigh, the three-dimensional joint points obtained above are paired and connected in anatomical order to form a graph structure composed of joint points and limb segments, that is, a limb segment connection graph. For example, after measuring the positions of the left and right shoulders, elbows, and wrists in three-dimensional space in a four-camera system, they are connected in the order of "shoulder-elbow-wrist" to form the upper limb link. The same method is used to process the lower limbs and torso, and finally, a complete human skeleton representation is spliced ​​together.

[0027] Step S3: Calculate the rotation angle of each limb in the limb connection diagram to obtain limb posture data, and arrange the limb posture data in time sequence according to the acquisition time of the synchronous image frame group to obtain three-dimensional motion data.

[0028] Specifically, using the joints defined in the limb connection diagram as a reference, the limb chain formed by three adjacent points is regarded as a rigid body motion unit. For example, the direction of the upper arm is determined by the shoulder and elbow joints, while that of the forearm is determined by the elbow and wrist. The rotation angle at the joint is calculated by the change of this vector between consecutive frames. The specific method is to first construct a local coordinate system, and then use quaternions or Euler angles to represent the orientation difference, thereby solving the rotation amount in each degree of freedom. This result is the limb posture data. Subsequently, the posture parameters obtained at each moment are arranged sequentially according to the time sequence corresponding to the previously synchronized image frame group to form a motion record that changes over time, which is three-dimensional motion data. In a practical case, in walking motion capture, after continuously acquiring the rotation angle of each joint of the lower limb in each frame, they are strung together at 60fps intervals to reconstruct a complete gait cycle curve, which can be used for subsequent analysis of motion characteristics or to drive virtual character animation.

[0029] In a specific embodiment, the target human body is simultaneously captured from multiple perspectives and the inter-frame timestamps are aligned to obtain a synchronized image frame group, including: Simultaneously start shooting the target human body using multiple preset camera devices and add timestamps to obtain time-stamped image frames; Timestamp comparison is performed on time-stamped image frames with the same frame number from different viewpoints. If the timestamp deviation of image frames with the same frame number from different viewpoints is within a preset threshold range, the time-stamped image frame is retained. If it exceeds the preset threshold range, the timestamps of the time-stamped image frames at the corresponding viewpoints and subsequent time-stamped image frames are linearly interpolated to adjust the timestamps so that the timestamp deviation of time-stamped image frames exceeding the preset threshold range from the timestamps of corresponding frames at other viewpoints is within the preset threshold range, thus obtaining a synchronized image frame group.

[0030] Specifically, in implementing multi-view synchronous shooting, multiple camera devices are first arranged in different spatial positions around the target human body according to experimental requirements. These positions are usually pre-set based on the capture range and occlusion avoidance principles. Each camera starts recording simultaneously after receiving a unified trigger signal, ensuring high consistency in the timing of the starting frames, thereby acquiring a sequence of raw images from different perspectives—the multi-view raw image sequence. To facilitate subsequent processing, the system numbers each frame captured by each camera according to its order in the local sequence; this process is called frame numbering, and the result is a numbered image frame. Next, simultaneously or after the image is written to the storage medium, the absolute time information of the actual time the frame was captured is appended to its header; this operation is called timestamp addition, resulting in a time-stamped image frame. The timestamp used here generally comes from a high-precision clock source, such as PTP protocol timing or a GPS synchronization module, to ensure a unified time base across devices.

[0031] After completing the above steps, the system proceeds to the inter-frame timestamp alignment stage. The system extracts time-stamped image frames with the same frame number from different viewpoints and compares their timestamp differences. If the time difference between one or more pairs of frames with the same frame number does not exceed a preset threshold (e.g., ±2 milliseconds), their synchronization is considered satisfactory, and these time-stamped image frames are directly retained for subsequent processing. However, in practice, transmission delays, buffering mechanisms, or uneven hardware response often lead to cumulative offsets in some viewpoints. Once a timestamp deviation corresponding to a frame number is found to exceed the allowable range, linear interpolation adjustment of the timestamps of that frame and all subsequent time-stamped image frames under that viewpoint is required. Specifically, based on known correctly aligned reference frame points, a time mapping function is constructed, and the original timestamp is re-estimated and replaced, so that the time deviation of the adjusted result with corresponding frames from other viewpoints falls back within the threshold. For example, in a four-camera gait analysis experiment, the third camera lagged behind by 5ms due to network card latency. At this time, the initial and last two sets of aligned frames were used as anchor points, and proportional time compensation was applied to all intermediate frames to finally obtain a strictly aligned synchronized image frame group, which served as the basic input for joint detection.

[0032] In a specific embodiment, the step of performing joint point detection on the human body region in the synchronized image frame group to obtain two-dimensional joint point coordinates includes: By using pixel color differences and edge contour information, the human body region of each frame in the synchronized image frame group is separated from the background to obtain a human body segmentation image. The human body segmentation image is then grayscaled to obtain a grayscale human body image. The human joints in the grayscale human body image are located and marked. The marked joint image is obtained by using the positional features of the human joints in the grayscale human body image and the relative positional relationship between adjacent joints. Extract the pixel coordinates of each joint in the marked joint image, and then convert the pixel coordinates into two-dimensional joint coordinates in the image coordinate system.

[0033] Specifically, after obtaining the synchronized image frame group, a joint detection operation needs to be performed on each frame to obtain the coordinates of two-dimensional joints that can be used for subsequent 3D reconstruction. This process begins with human body region segmentation. For a single frame image, the distribution characteristics of pixels in the color space, such as the obvious difference between human skin color and the background environment in RGB or HSV channels, are used. Combined with edge detection algorithms (such as the Canny operator), regions with continuous closed contours are extracted. These connected components that conform to the prior knowledge of human body shape are identified as the foreground human body parts, thereby achieving the separation of the human body region from the background. The output result is the human segmentation image. This step is particularly suitable for shooting scenes where the background is relatively static and has high contrast with the human clothing. For example, in a laboratory environment, using a light-colored wall as the background, the subject wears dark tight-fitting clothes for motion capture. In this case, the skin color and clothing color form a strong contrast with the background, which is conducive to accurate segmentation.

[0034] Next, the human segmentation image is converted to grayscale, that is, it is converted from color format to single-channel grayscale image. Specifically, the brightness value of each pixel can be calculated by weighted averaging, and the formula is generally I = 0.299R + 0.587G + 0.114B, to obtain a grayscale human image. This not only reduces the data dimensionality but also reduces the computational burden of subsequent processing, while retaining sufficient structural information for joint localization. Then, the joint marking stage begins. Based on known human anatomical structure patterns in the grayscale human image, combined with the local texture features of the joint location (such as brightness changes at joint bends) and the geometric constraints between adjacent joints (such as the elbow joint usually being located in the lower-middle region of the line connecting the shoulder and wrist), template matching or regression-based prediction models are used to mark the key point positions one by one, generating marked joint image. For example, in a top-down view, when the positions of the shoulder and wrist have been preliminarily determined, the system can infer the approximate area of ​​the elbow and search for gradient abrupt change points within this range for precise localization.

[0035] The final step is coordinate extraction and transformation. The pixel row and column values ​​(u, v) of each labeled joint are read from the marked joint image; these values ​​represent its discrete position in the image plane. These values ​​are then expressed in a unified image coordinate system, typically with the top-left corner of the image as the origin, the positive x-axis pointing horizontally to the right, and the positive y-axis pointing vertically downwards, forming the final two-dimensional joint coordinates. This coordinate system is consistent with the coordinate framework used in the camera equipment calibration process, ensuring data compatibility in subsequent triangulation steps.

[0036] In a specific embodiment, the step of performing multi-view triangulation on the two-dimensional joint coordinates based on preset camera equipment calibration parameters to obtain the three-dimensional joint coordinates includes: For the two-dimensional joint point coordinates, combined with the camera intrinsic parameters in the preset camera equipment calibration parameters, the two-dimensional joint point coordinates are transformed from the image coordinate system to the camera coordinate system corresponding to each viewpoint, so as to obtain the camera coordinate points under each viewpoint. Noise filtering is performed on the camera coordinate points under each viewpoint to obtain clean camera coordinate points under each viewpoint. Based on clean camera coordinates from each viewpoint, and according to the principle of spatial line of sight intersection, the intersection points of corresponding joints in the spatial line of sight under different viewpoints are calculated to obtain the initial three-dimensional joint coordinates. The initial three-dimensional joint coordinates are processed to unify the three-dimensional coordinates calculated from different perspectives into the same world coordinate system, thus obtaining the three-dimensional joint coordinates.

[0037] Specifically, after obtaining the coordinates of the two-dimensional joint points from various viewpoints, multi-view triangulation needs to be performed using preset camera equipment calibration parameters to reconstruct their three-dimensional positions. First, using the intrinsic parameter matrix obtained beforehand through calibration for each camera, including focal length, principal point coordinates, and distortion coefficients, the coordinates of the two-dimensional joint points in the original image are inversely mapped from the image coordinate system to the camera coordinate system of the corresponding viewpoint. Specifically, the pixel coordinates are first corrected for distortion, and then normalized according to the intrinsic parameters to obtain the spatial ray direction pointing from the camera's optical center to the joint point, forming the camera coordinate points for each viewpoint. Since false detections or edge blurring may occur in actual detection, these ray directions may carry noise. Therefore, noise filtering needs to be performed on the camera coordinate points output from different viewpoints at the same time. Common methods include setting a threshold based on reprojection error to remove outliers, or using the RANSAC algorithm to fit a spatial line and retain interior points, thereby obtaining more reliable and clean camera coordinate points for each viewpoint.

[0038] Based on this, triangulation is performed according to the principle of spatial line-of-sight intersection. For the same anatomical joint (e.g., the left elbow), the line-of-sight rays represented by clean camera coordinate points from at least two different viewpoints are taken, and their closest intersection point or the midpoint of their common perpendicular in 3D space is calculated as the preliminary 3D position estimate of the joint. Considering that ideally multiple rays should converge, but in reality, due to measurement errors, they often do not intersect perfectly, the least squares method is typically used to find the spatial point that minimizes the sum of the squared distances from all rays to that point, thereby improving positioning accuracy. After the initial reconstruction, the obtained 3D coordinates are still in the local coordinate systems of their respective cameras and must be unified under a common reference frame to form a complete representation of the human skeleton. At this point, the extrinsic parameters determined during calibration are introduced, that is, the rotation matrix and translation vector of each camera relative to the world coordinate system are applied to the above 3D points, transforming them to the same world coordinate system, ultimately outputting consistent and comparable 3D joint coordinates. For example, in a four-camera system, after key points such as the left and right shoulders and hips have completed cross-viewpoint fusion and coordinate alignment, they can be used to subsequently construct a globally coherent motion sequence.

[0039] In a specific embodiment, the step of calculating the intersection points of corresponding joint points in the spatial line of sight under different viewpoints based on clean camera coordinate points from each viewpoint, according to the principle of spatial line of sight intersection, to obtain the initial three-dimensional joint point coordinates, includes: Based on the clean camera coordinates at each viewpoint, and combined with the optical center coordinates of the corresponding camera device at each viewpoint, the clean camera coordinates and the optical center coordinates are connected and extended to obtain the spatial rays at each viewpoint, including the ray origin coordinates and the ray direction vector. Based on the principle of spatial line of sight intersection, spatial rays from different perspectives are paired up in pairs, and the common perpendicular segment of the paired spatial rays is solved to obtain the ray pair common perpendicular segment including the coordinates of the two endpoints of the common perpendicular segment and the length value of the segment. The midpoints of the two endpoints of the common perpendicular segment of the ray pair are calculated to obtain the candidate spatial points of each ray pair; Assign confidence scores to the candidate spatial points according to the length of the line segment of the common perpendicular line segment of the ray, and obtain candidate points with confidence scores; Based on the confidence level of the candidate points with confidence, a weighted average is calculated for multiple candidate points with confidence corresponding to the same joint point to obtain the initial three-dimensional joint point coordinates.

[0040] Specifically, after obtaining the clean camera coordinates from each viewpoint after noise filtering, it is necessary to further utilize spatial geometric relationships to solve for their corresponding three-dimensional spatial positions. In practice, this involves first combining the optical center coordinates determined during the calibration process for each camera device—that is, the three-dimensional coordinates of the lens's optical center in the world coordinate system—and connecting this optical center with the clean camera coordinates of a certain joint point at the current viewpoint. This connection is then extended forward along this direction, forming a spatial ray with a clear starting point and direction. The starting point of this ray is the optical center coordinate, and the direction vector is obtained by subtracting the optical center from the clean camera coordinates and then normalizing it, representing the possible distribution paths of that joint point on the current line of sight.

[0041] Next, based on the principle of spatial line-of-sight intersection, the aforementioned spatial rays from different perspectives are paired up. Due to errors in actual measurements, two rays corresponding to the same anatomical point often do not actually intersect, but rather appear as skew lines. In this case, it is necessary to solve for the shortest line segment, i.e., the common perpendicular segment, between each pair of rays. This segment intersects the two rays perpendicularly, and its two endpoints can be calculated using the vector projection formula. The length of this segment is also recorded. For example, in a four-camera system, if the rays pointing towards the "right wrist" extracted from the left and front cameras do not directly intersect, the common perpendicular segment between them is calculated mathematically to obtain the positions and distance between their endpoints.

[0042] Subsequently, the arithmetic mean of the coordinates of the two endpoints of the common perpendicular segment is taken to obtain the midpoint, which is used as the estimated candidate spatial point for this ray pair. Theoretically, if the two rays intersect completely, the length of the common perpendicular segment should be zero, and the midpoint is the true intersection point; however, the larger the length, the worse the consistency. Therefore, the system uses the reciprocal of the segment length or a weighted negative exponential function as a basis to assign a confidence weight to each candidate spatial point, forming a candidate point with confidence. Finally, for the same joint point (such as "left knee"), there may be multiple candidate points with confidence from different viewpoint combinations. These are weighted and averaged according to their respective confidence levels, with higher weights having a greater impact. Ultimately, an initial 3D joint point coordinate system that integrates multi-viewpoint information and has strong anti-interference capabilities is output.

[0043] In a specific embodiment, the step of establishing limb connection relationships based on the three-dimensional joint coordinates according to the human skeletal topology to obtain a limb connection diagram includes: The coordinates of the three-dimensional joint points are labeled with joint point types based on human anatomical features to obtain a set of labeled joint points, and the distance data between each joint point in the set of labeled joint points is calculated. Based on the distance data between each joint and the preset limb length threshold in the human skeletal topology, the labeled joint set is initially connected and matched, and joints whose distance data is within the corresponding limb length threshold range are connected to obtain initial connected joint pairs. Based on the initial connection joint pairs, the connection order of each limb segment is determined according to the parent-child hierarchical relationship of limb segments in the human skeletal topology. Based on the connection order of each limb segment, a limb connection diagram containing each limb segment and its connection relationship is drawn.

[0044] Specifically, after reconstructing the 3D joint coordinates, it is necessary to further establish the connection relationships between the joints to form a structurally meaningful limb representation, i.e., a limb connection diagram. This process first involves labeling the acquired 3D joint coordinates according to human anatomical features, such as labeling a spatial point as "left shoulder," "right knee," or "mid-hip point," thus forming a labeled joint set. This step relies on the joint naming rules output in the previous detection phase to ensure that joints at the same location across different frames have consistent labels. With the labeling results, the system traverses all joint pairs, calculates their Euclidean distances, and obtains the distance data between each joint. These values ​​reflect the actual spacing of each part in 3D space and are the basis for subsequent connection judgments.

[0045] Next, the initial connection matching phase begins. Based on preset limb length thresholds in the human skeletal topology—for example, the upper arm is typically between 25 and 40 centimeters long, and the lower leg is about 30 to 45 centimeters—two joints whose distances fall within the corresponding range are considered potential connection objects. For instance, if the spatial distance between the "left shoulder" and the "left elbow" is 32 centimeters, which falls within the upper arm length range in a normal adult proportion, they are initially connected to form an initial connection joint pair. This geometrically constrained screening method effectively eliminates cross-regional erroneous connections, such as avoiding incorrectly connecting the "left wrist" to the "right knee." However, it should be noted that when the human body undergoes significant bending or occlusion causing coordinate shifts, the distance of some actual limb segments may temporarily exceed the threshold. Therefore, the threshold setting needs to retain a certain tolerance, generally using the statistical mean ± standard deviation to set the upper and lower bounds.

[0046] Distance matching alone is insufficient to definitively determine the correct connection structure, especially in the presence of incomplete or noisy components, which can easily lead to ambiguity. Therefore, a parent-child hierarchical relationship from the human skeletal topology is introduced for logical correction. For example, given that "torso" is the parent node, "left shoulder" is its child node, and "left elbow" is a child node of "left shoulder," connections are expanded layer by layer in this order to avoid reverse or skipped connections. By analyzing whether the initial connection joint pairs conform to this hierarchical logic and eliminating combinations that do not conform to anatomical rules, the connection order information for each limb segment is finally determined. This process is similar to constructing a tree structure with the pelvis as the central root node and the limbs as branches, ensuring the rationality and connectivity of the overall skeleton.

[0047] Finally, based on the determined connection sequence information, a visual structure diagram is generated using graphical drawing tools or procedural methods. Each joint is represented by a point, and each effective connection is represented by a line segment. The overall structure constitutes a limb connection diagram that includes each limb segment and its interrelationships. For example, in gait analysis experiments, if the connection diagram of consecutive frames shows that the left thigh and lower leg maintain a stable connection with smooth angle changes, the connection can be considered correct, supporting subsequent posture calculation.

[0048] In a specific embodiment, determining the connection order of each limb segment based on the initial connection joint pair and according to the parent-child hierarchy of limb segments in the human skeletal topology includes: The initial connected joint point pair is subjected to trunk joint point identification to obtain the root joint point position, and the hierarchical distance of the initial connected joint point pair is calculated based on the root joint point position to obtain the joint point hierarchical value. Based on the joint level value, the parent-child relationship of the initial connected joint pairs is marked. The joint with the smaller level value in each limb segment is marked as the parent joint and the joint with the larger level value is marked as the child joint, thus obtaining the limb segment parent-child marked pairs. The limb segment parent-child marked pairs are then divided into branch paths according to the human skeletal topology, and limb segments belonging to the same kinetic chain are grouped into the same transmission branch, thus obtaining branch limb group. Each segment in the branch segment group is sequentially numbered from the root joint to the terminal joint to obtain the segment number within the branch. Based on the segment number within the branch and the preset priority of each branch segment group, all segments are uniformly sorted to obtain the connection order of each segment.

[0049] Specifically, after generating the initial connection joint pairs, it is necessary to further clarify the structural order between the limb segments to conform to the actual movement transmission laws of the human body. This process begins by identifying key points in the trunk region, selecting a stable joint point located at the center of the body as the root joint point, usually the "mid-hip" or "vertebral base point." These points are not easily obscured in most postures and are directly connected to multiple limb segments, making them suitable as the topological starting point. Once the root joint point is determined, it is used as a reference benchmark to perform hierarchical analysis on all joint pairs formed through the initial connections. Specifically, the analysis is performed by expanding outward layer by layer according to the connection relationships, calculating the number of connecting edges that each joint point passes through to the root node. This value is the joint point's level value. For example, the "mid-hip" itself has a level of 0, the "left shoulder" is connected to it via the "thoracic spine," so its level is 2, and the "left wrist" is connected to the "left elbow," so its level is 4.

[0050] Based on the aforementioned hierarchical values, the system assigns parent-child relationships to each pair of initial connection joints. The rule is set as follows: within the same connection, the joint with the smaller hierarchical value is considered the parent joint, while the larger one is marked as the child joint, thus forming groups of directional limb segment parent-child pairs. This division simulates the mechanism of proximal-to-distal movement in human motion, such as the thigh driving the lower leg, and the upper arm driving the forearm. Subsequently, according to the natural branching paths in the human skeletal topology, these parent-child pairs are categorized into different kinetic chains. For example, all upper limb connections extending from the "left shoulder" are grouped into the left upper limb branch, including "left shoulder—left elbow," "left elbow—left wrist," etc., forming an independent branch segment group; similarly, the right upper limb, left lower limb, right lower limb, and trunk links are divided.

[0051] Next, within each branch segment group, the segments are sequentially numbered according to the transmission direction from the root joint to the terminal joint, resulting in segment numbers within the branch. For example, in the left lower limb branch, "left hip—left knee" is segment 1, and "left knee—left ankle" is segment 2. Finally, combining the preset priorities of each branch segment group (e.g., trunk first, left upper limb second, right lower limb last), the numbering sequences within all branches are concatenated according to priority to form a complete global sorting scheme. This order determines the order in which joint angles are solved in subsequent posture calculations, ensuring that the data processing flow is consistent with biomechanical logic. The entire process relies on connection structures and hierarchical derivation to achieve the transformation from disordered connections to an ordered skeleton.

[0052] In a specific embodiment, the step of calculating the rotation angle of each limb segment in the limb segment connection diagram to obtain limb segment posture data includes: Define coordinate axes for each limb segment in the limb segment connection diagram, and determine local coordinate axes for each limb segment based on human anatomical orientation; Using vector calculation methods, the rotation angle between the local coordinate axes of adjacent limb segments is calculated at the joints between adjacent limb segments. Based on the rules of human joint kinematics, the rotation angle is constrained and checked, and the rotation angle that exceeds the preset range of motion of the joint is adjusted in order to determine the limb posture data of each limb segment in three-dimensional space.

[0053] Specifically, after obtaining the limb connection diagram, the rotation angles of each limb segment need to be calculated to obtain limb posture data that reflects the human body's motion state. This process begins with the definition of coordinate axes. For each rigid limb segment consisting of two joints, such as the upper arm, forearm, or thigh, a local coordinate system is established based on the standard directions in human anatomy. Usually, the direction from the proximal joint to the distal joint is taken as the principal axis of the limb segment (such as the x-axis). Then, combined with the body's anterior-posterior direction or the direction of gravity, vector operations such as cross product are used to complete the other two axes, forming a right-handed coordinate system. For example, in the left upper arm segment, the vector from "left shoulder" to "left elbow" is normalized and set as the positive direction of the local x-axis. The y-axis can be the projection of the upward direction perpendicular to the ground onto the plane of the limb segment, and the z-axis is determined by x×y, thus completing the construction of the local coordinate axes.

[0054] Once the local coordinate systems of each limb segment are established, the calculation of rotation angles between adjacent segments can begin. Taking the elbow joint as an example, its movement mainly manifests as flexion, extension, and abduction of the forearm relative to the upper arm. Therefore, it is necessary to analyze the spatial transformation relationship between the local coordinate axes of the two connected segments, the forearm and the upper arm. Specifically, this involves extracting the rotation matrices of both segments, i.e., using a 3×3 matrix composed of the three-axis unit vectors of each segment in the current frame, and then solving for the relative rotation from the coordinate system of the preceding segment to the coordinate system of the following segment. Common methods include Euler angle decomposition or quaternion representation. For example, when the upper arm remains basically vertical while the forearm bends downwards, the rotation angle around the local x-axis can be obtained as approximately 120 degrees through matrix transformation, corresponding to the degree of elbow flexion. This vector-based calculation method can accurately capture spatial posture changes at the joint, and is particularly suitable for describing the complex movements of multi-degree-of-freedom joints such as the shoulder and hip.

[0055] However, due to slight jitter or positioning drift during 3D reconstruction, the directly calculated rotation angle may sometimes deviate from physiological reality. Therefore, human joint kinematics rules are introduced for constraint checks to ensure the results are within a reasonable range. Each joint has its inherent limits of motion; for example, the knee joint can generally only flex and extend between 0° (extended) and 130° (squatting), exceeding this range is considered abnormal. The system pre-stores a set of empirical thresholds. When a knee joint rotation angle of 145° is detected in a frame, it is corrected to the maximum allowable value of 130°, or smoothed using interpolation between consecutive frames to prevent abrupt changes from interfering with subsequent analysis. This step not only improves data stability but also enhances the fit to real human movement capabilities.

[0056] The final output limb posture data is the corrected sequence of rotational parameters for each joint, which can be used to drive digital human models, assess rehabilitation progress, or identify specific movement patterns.

[0057] In a specific embodiment, the step of calculating the rotation angle between the local coordinate axes of adjacent limb segments at the joints using a vector calculation method includes: A reference plane is constructed for the local coordinate axes of the parent limb segment in the adjacent limb segments. Based on the three orthogonal axis vectors of the local coordinate axes of the parent limb segment, three mutually perpendicular reference planes are formed by combining them in pairs, thus obtaining the joint transition coordinate system. By using vector calculation methods, the principal direction vectors of the local coordinate axes of the sub-limb segments in the adjacent limb segments are orthogonally projected onto each reference plane of the joint transition coordinate system to obtain the projection component vectors on each reference plane. The angle between the projection component vector and the corresponding reference axis vector in the local coordinate axis of the parent limb segment is calculated to obtain the plane rotation angle in each reference plane, wherein the plane rotation angle includes flexion-extension angle, adduction-abduction angle and axial rotation angle; Based on the planar rotation angles in each reference plane, the flexion-extension angle, adduction-abduction angle, and axial rotation angle are ordered and combined according to the joint rotation sequence to obtain the rotation angle between the local coordinate axes of adjacent limb segments.

[0058] Specifically, after determining the parent-child relationship between adjacent limb segments, the relative rotation angle between them at the joint needs to be calculated to accurately represent the limb's movement state. This process first constructs a reference system based on the local coordinate axes of the parent limb segment. Specifically, three mutually orthogonal unit direction vectors (such as the x, y, and z axes) defined by the parent limb segment are taken, and three mutually perpendicular planes are generated by combining them in pairs: the horizontal plane spanned by x and y, the sagittal plane formed by x and z, and the coronal plane determined by y and z. These three planes together form a local framework around the joint, called the joint transition coordinate system. This structure does not change with the movement of the child limb segment, but only depends on the orientation of the parent limb segment, making it suitable as a static reference for rotation measurement.

[0059] Next, we proceed to the projection calculation. For the principal direction vector of the sub-limb segment—usually its local x-axis, i.e., the direction from the proximal joint to the distal joint—it is orthogonally projected onto the three reference planes mentioned above. For example, projecting the sub-limb segment's x-axis vector onto the sagittal plane of the parent segment (composed of the x and z axes) yields a two-dimensional vector within that plane; this is the projection component vector. Similar operations are applied to the other two planes to ensure that the directional information of the sub-limb segment is completely decomposed into each anatomical plane.

[0060] Subsequently, the angle between each projection component vector and its corresponding reference axis in the reference plane is calculated using the vector angle formula. For example, in the sagittal plane, if the x-axis of the parent limb segment is used as the forward reference, the angle between the projection of the child limb segment on this plane and the x-axis is the flexion-extension angle; while in the coronal plane, the angle between its projection and the y-axis or z-axis of the parent limb segment may correspond to the adduction-abduction angle; as for the degree of torsion about the principal axis of the parent limb segment itself, it can be obtained by analyzing the difference between the offset direction of the child limb segment axis in the cross-section and the initial orientation, and is denoted as the axial rotation angle. These angles are collectively called planar rotation angles, each reflecting the motion component in a certain degree of freedom.

[0061] Finally, to avoid Euler angle singularity and conform to the actual movement sequence of the human body, the system sequentially combines the three planar rotation angles according to a preset joint rotation sequence. For example, the shoulder joint often uses the Tait-Bryan sequence of "flexion-extension-abduction-rotation," while the hip joint may be slightly different. Through this sequential synthesis, a set of rotation angle data that can completely describe the posture changes between adjacent limb segments is finally output. The entire process is implemented based on vector algebra, without matrix decomposition, and exhibits good stability and physiological rationality in gait analysis or motion capture playback.

[0062] In a specific embodiment, the step of arranging the limb posture data in chronological order according to the acquisition time of the synchronized image frame group to obtain three-dimensional motion data includes: The limb posture data is correlated frame by frame with the acquisition time of each frame in the synchronized image frame group to obtain a time-stamped posture frame sequence. Based on the acquisition time values ​​of each frame in the time-stamped attitude frame sequence, the time-stamped attitude frame sequence is arranged in ascending order according to the time sequence, and the limb attitude data of each frame after arrangement is connected end to end to form a continuous attitude change record. Based on the posture change records, three-dimensional motion data is obtained, including a sequence of rotation angle changes of each limb segment arranged in chronological order and the corresponding time axis information.

[0063] Specifically, after processing the limb pose data in each frame, it needs to be correlated with the time dimension to reconstruct the complete motion process. To this end, the limb pose data corresponding to each frame is first matched with its original acquisition time in the synchronized image frame group. This time comes from the precise timestamp recorded during previous multi-view shooting, typically expressed in milliseconds UTC time or as an offset relative to the starting time. By establishing a one-to-one correspondence, each set of pose parameters carries a clear time stamp, forming a single-frame data unit with temporal information, thus obtaining a time-stamped pose frame sequence. This process requires that the generation of pose data strictly correspond to its source image frames, without misalignment or omission, to ensure the accuracy of subsequent arrangement.

[0064] Because the system may experience buffering delays or output order disruptions due to parallel computation during processing, even though the input images are aligned, the calculated attitude frames may not be output in natural chronological order. Therefore, it is necessary to perform an ascending sorting operation on the entire sequence based on the acquisition time value attached to each time-stamped attitude frame. Specifically, a quicksort algorithm can be used to rearrange the frames by timestamp, placing the earliest acquired frame at the beginning of the sequence and the latest at the end, ultimately obtaining a continuous sequence strictly arranged chronologically. For example, under 60fps acquisition conditions, the theoretical interval between adjacent frames is approximately 16.67ms. If the timestamp of a frame deviates significantly from this rhythm, it is necessary to check for packet loss or interpolation compensation, and mark or remove outliers if necessary.

[0065] Once the sequence is sorted, the limb rotation angle data from each frame are concatenated end-to-end to form a record of posture changes over time. This record not only includes the angle evolution curves of joints such as the shoulder, elbow, and knee, but also retains the corresponding timeline information, reflecting the speed, acceleration, and periodicity of the movement. For example, in a walking motion capture, the flexion-extension angle of the left hip joint exhibits regular peak-and-trough changes, and this record clearly identifies the various stages of the gait cycle. Ultimately, this integrated dataset constitutes three-dimensional motion data, typically organized as a multi-column array, with each row representing a frame and each column storing a timestamp and rotation parameters for each limb segment, facilitating import into analysis software or animation engines.

[0066] Please see Figure 2 , Figure 2 This is a schematic diagram of the framework of an embodiment of the three-dimensional human motion capture device of this application. Figure 2 As shown, the three-dimensional human motion capture device includes a detection module 1, used to perform multi-view synchronous shooting of the target human body and frame-to-frame timestamp alignment to obtain a synchronous image frame group, and to perform joint point detection on the human body region in the synchronous image frame group to obtain two-dimensional joint point coordinates; a measurement module 2, used to perform multi-view triangulation measurement on the two-dimensional joint point coordinates based on preset camera equipment calibration parameters to obtain three-dimensional joint point coordinates, and to establish limb connection relationships on the three-dimensional joint point coordinates based on the human skeleton topology to obtain a limb connection diagram; and an arrangement module 3, used to calculate the rotation angle of each limb in the limb connection diagram to obtain limb posture data, and to arrange the limb posture data in time sequence according to the acquisition time of the synchronous image frame group to obtain three-dimensional motion data.

[0067] Reference Figure 3 This invention also provides a computer device whose internal structure can be as follows: Figure 3As shown, the computer device includes a processor, memory, display screen, input device, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores the data corresponding to this embodiment. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the above-described method.

[0068] Those skilled in the art will understand that Figure 3 The structures shown are merely block diagrams of some structures related to the present invention and do not constitute a limitation on the computer devices on which the present invention is applied.

[0069] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0070] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the present invention and embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM, etc.

[0071] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0072] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0073] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0074] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0075] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0076] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0077] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. A three-dimensional human motion capture method, characterized in that, Includes the following steps: Multi-view synchronous shooting of the target human body and frame-to-frame timestamp alignment are performed to obtain a synchronous image frame group. Joint point detection is performed on the human body region in the synchronous image frame group to obtain two-dimensional joint point coordinates. Based on the preset camera equipment calibration parameters, the coordinates of the two-dimensional joint points are measured by multi-view triangulation to obtain the coordinates of the three-dimensional joint points. Based on the topological structure of the human skeleton, the limb connection relationship of the three-dimensional joint point coordinates is established to obtain the limb connection diagram. The rotation angle of each limb in the limb connection diagram is calculated to obtain limb posture data. The limb posture data is then arranged in time sequence according to the acquisition time of the synchronous image frame group to obtain three-dimensional motion data.

2. The three-dimensional human motion capture method according to claim 1, characterized in that, Multi-view synchronous shooting of the target human body and inter-frame timestamp alignment are performed to obtain a synchronized image frame group, including: Simultaneously start shooting the target human body using multiple preset camera devices and add timestamps to obtain time-stamped image frames; Timestamp comparison is performed on time-stamped image frames with the same frame number from different viewpoints. If the timestamp deviation of image frames with the same frame number from different viewpoints is within a preset threshold range, the time-stamped image frame is retained. If it exceeds the preset threshold range, the timestamps of the time-stamped image frames at the corresponding viewpoints and subsequent time-stamped image frames are linearly interpolated to adjust the timestamps so that the timestamp deviation of time-stamped image frames exceeding the preset threshold range from the timestamps of corresponding frames at other viewpoints is within the preset threshold range, thus obtaining a synchronized image frame group.

3. The three-dimensional human motion capture method according to claim 1, characterized in that, The step of performing joint point detection on the human body region in the synchronized image frame group to obtain two-dimensional joint point coordinates includes: By using pixel color differences and edge contour information, the human body region of each frame in the synchronized image frame group is separated from the background to obtain a human body segmentation image. The human body segmentation image is then grayscaled to obtain a grayscale human body image. The human joints in the grayscale human body image are located and marked. The marked joint image is obtained by using the positional features of the human joints in the grayscale human body image and the relative positional relationship between adjacent joints. Extract the pixel coordinates of each joint in the marked joint image, and then convert the pixel coordinates into two-dimensional joint coordinates in the image coordinate system.

4. The three-dimensional human motion capture method according to claim 1, characterized in that, The process of performing multi-view triangulation on the two-dimensional joint coordinates based on preset camera equipment calibration parameters to obtain three-dimensional joint coordinates includes: For the two-dimensional joint point coordinates, combined with the camera intrinsic parameters in the preset camera equipment calibration parameters, the two-dimensional joint point coordinates are transformed from the image coordinate system to the camera coordinate system corresponding to each viewpoint, so as to obtain the camera coordinate points under each viewpoint. Noise filtering is performed on the camera coordinate points under each viewpoint to obtain clean camera coordinate points under each viewpoint. Based on clean camera coordinates from each viewpoint, and according to the principle of spatial line of sight intersection, the intersection points of corresponding joints in the spatial line of sight under different viewpoints are calculated to obtain the initial three-dimensional joint coordinates. The initial three-dimensional joint coordinates are processed to unify the three-dimensional coordinates calculated from different perspectives into the same world coordinate system, thus obtaining the three-dimensional joint coordinates.

5. The three-dimensional human motion capture method according to claim 1, characterized in that, The process of establishing limb connection relationships based on the three-dimensional joint coordinates according to the human skeletal topology to obtain a limb connection diagram includes: The coordinates of the three-dimensional joint points are labeled with joint point types based on human anatomical features to obtain a set of labeled joint points, and the distance data between each joint point in the set of labeled joint points is calculated. Based on the distance data between each joint and the preset limb length threshold in the human skeletal topology, the labeled joint set is initially connected and matched, and joints whose distance data is within the corresponding limb length threshold range are connected to obtain initial connected joint pairs. Based on the initial connection joint pairs, the connection order of each limb segment is determined according to the parent-child hierarchical relationship of limb segments in the human skeletal topology. Based on the connection order of each limb segment, a limb connection diagram containing each limb segment and its connection relationship is drawn.

6. The three-dimensional human motion capture method according to claim 1, characterized in that, The step of calculating the rotation angle of each limb in the limb connection diagram to obtain limb posture data includes: Define coordinate axes for each limb segment in the limb segment connection diagram, and determine local coordinate axes for each limb segment based on human anatomical orientation; Using vector calculation methods, the rotation angle between the local coordinate axes of adjacent limb segments is calculated at the joints between adjacent limb segments. Based on the rules of human joint kinematics, the rotation angle is constrained and checked, and the rotation angle that exceeds the preset range of motion of the joint is adjusted in order to determine the limb posture data of each limb segment in three-dimensional space.

7. The three-dimensional human motion capture method according to claim 1, characterized in that, The limb posture data is sequentially arranged according to the acquisition time of the synchronized image frame group to obtain three-dimensional motion data, including: The limb posture data is correlated frame by frame with the acquisition time of each frame in the synchronized image frame group to obtain a time-stamped posture frame sequence. Based on the acquisition time values ​​of each frame in the time-stamped attitude frame sequence, the time-stamped attitude frame sequence is arranged in ascending order according to the time sequence, and the limb attitude data of each frame after arrangement is connected end to end to form a continuous attitude change record. Based on the posture change records, three-dimensional motion data is obtained, including a sequence of rotation angle changes of each limb segment arranged in chronological order and the corresponding time axis information.

8. A three-dimensional human motion capture device, characterized in that, include: The detection module is used to simultaneously capture the target human body from multiple perspectives and align the timestamps between frames to obtain a synchronized image frame group, and to perform joint point detection on the human body region in the synchronized image frame group to obtain two-dimensional joint point coordinates. The measurement module is used to perform multi-view triangulation on the coordinates of the two-dimensional joint points based on preset camera equipment calibration parameters to obtain the coordinates of the three-dimensional joint points, and to establish the limb connection relationship of the coordinates of the three-dimensional joint points based on the topology of the human skeleton to obtain the limb connection diagram. The arrangement module is used to calculate the rotation angle of each limb in the limb connection diagram to obtain limb posture data, and to arrange the limb posture data in time sequence according to the acquisition time of the synchronous image frame group to obtain three-dimensional motion data.

9. A computer device, characterized in that, The method includes a memory and a processor coupled to each other, wherein the memory stores program instructions and the processor executes the program instructions to implement the three-dimensional human motion capture method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The device stores program instructions that can be executed by a processor, the program instructions being used to implement the three-dimensional human motion capture method according to any one of claims 1 to 7.

Images