Motion capture system and method

Abstract

The application discloses a motion capture system and method, which comprises the following steps: firstly, acquiring an animation scene image containing human motion, setting a mark point at a key part of a human body, and performing frame processing on an animation sequence; then, calculating pixel motion between adjacent frames by using an optical flow field technology, accurately identifying start and end frames of human motion, and tracking and recording pixel coordinates of the mark point in the motion frames; based on the coordinate information, constructing a motion trajectory of the mark point, and associating and combining related motions, so as to provide a data basis for subsequent motion analysis; finally, applying a forward kinematics principle to solve 3D coordinates and spatial posture angles of the human body under a coordinate system, combining the information to calculate accurate 3D positions of the human body in the animation scene, and adjusting the result to a coordinate system used by animation software through coordinate system conversion, so that accurate restoration and positioning of human motion in animation are realized.

Description

Technical Field

[0001] This invention relates to the field of motion capture technology, and more particularly to a motion capture system and method. Background Technology

[0002] Motion capture is a technology used to record and process the movements of people or other objects. Using specific hardware, it captures the trajectory of moving objects in three-dimensional space and transforms this motion data into computer-understandable three-dimensional models, which are then applied to various fields such as animation, virtual reality, game development, gait analysis, and biomechanical research.

[0003] Chinese invention patent application number 202110786864.0 discloses a human motion capture method: it can accurately locate and calculate the human body's posture in a ground coordinate system by using at least two optical cameras. It can also ensure the accuracy of motion capture results even when indoor and outdoor shooting scenarios are limited, which greatly improves the practicality, convenience, accuracy, stability and applicability of motion capture technology.

[0004] However, in the field of animation production, in order to achieve accurate capture and reproduction of human movements, technicians have been committed to developing efficient motion capture technology. Traditional animation production often relies on hand drawing or keyframe animation, which have limitations when depicting complex or delicate human movements. With the rapid development of computer technology and image processing technology, image processing and motion capture-based technologies have gradually become a popular direction in animation production. Summary of the Invention

[0005] This application provides a motion capture system and method that introduces a starting motion frame to accurately identify the pixel coordinates of the motion, thereby improving capture accuracy.

[0006] This application provides a motion capture method, the method comprising:

[0007] S1, acquire scene images containing human body movements in the animation sequence, set marker points on key parts of the human body in the images, and perform frame-by-frame processing on the animation sequence;

[0008] S2, add motion frames to the human body after frame segmentation, mark the start and end motion frames for tracking human body movements, and track and record the pixel coordinates of the marked points in the motion frames.

[0009] S3, construct the motion trajectory of the marker point based on the collected pixel coordinates, and perform related action association combinations based on the motion trajectory information; wherein, the motion trajectory includes the marker point number, position, frame number and pixel coordinates (x,y);

[0010] S4, combine related actions to calculate the human body's 3D coordinates and spatial posture angles in the coordinate system;

[0011] S5 combines 3D coordinates and spatial pose angles to calculate the precise 3D position of the human body in the animation scene, and adjusts the result to the coordinate system used by the animation software through coordinate system transformation.

[0012] Preferably, S2 includes:

[0013] A1 uses the OpenCV computer vision library to read and preprocess the frame-by-frame animation sequence;

[0014] A2, based on the optical flow field, calculates the motion of pixels between adjacent frames to identify the start and end frames of human motion, thereby determining the motion frame; wherein, the start and end frames of tracking human motion include the start frame of motion analysis starting point and the end frame of motion defining the complete range of motion ending point, and the animation frames between the start and end frames are motion frames containing human motion.

[0015] A3, after identifying the starting action frame, tracks and records the pixel coordinates of each marker point in each frame of the action frame.

[0016] Preferably, S3 includes:

[0017] B1 organizes the pixel coordinates of each punctuation point in each frame of the action frame into a time-continuous data structure.

[0018] B2, for the data structure of each marker point, process its pixel coordinates in the action frame in sequence, and connect the pixel coordinates of the marker points in the order of the frame number to form the motion trajectory of the marker point;

[0019] B3 combines the motion trajectories between the marked points to obtain the associated motion combinations.

[0020] Preferably, A2, determining the action frame further includes:

[0021] A21, the identified motion frames are segmented into animation sequences, and the segmented motion frames of the segmented animation sequences are named separately;

[0022] A22, identify the motion trajectory of the human body in a separately named action frame, calculate the occlusion influence of overlapping action frames with unidentifiable marker points, and determine the influence of the occlusion influence on the motion trajectory.

[0023] A23, if the occlusion effect is less than the threshold, the effect of frame image overlap on motion trajectory is ignored;

[0024] A24. If the occlusion effect is greater than the threshold, calculate the feature point matching degree of the overlapping motion trajectory of the frame images.

[0025] Preferably, the calculation of the occlusion effect in A22 includes:

[0026]

[0027] A cover A represents the area of ​​the image covered, measured in pixels. total The number of covered markers. To cover the proportion of the image area occupied by the human body; N covered To cover the number of frames of the action, N total This represents the total number of marker points in the entire motion trajectory within the action frame. The proportion of the marked points that are obscured to the total number of marked points in the trajectory; F covered F is the number of frames of the action being covered. total This represents the total number of frames in the entire action. ΔD is the proportion of the number of frames covered to the total number of frames in the action; ΔD is the difference measure of the connection between the trajectories before and after the occlusion, defined as the average displacement difference between the trajectories before and after the occlusion at the connection point, which directly reflects the difference between the trajectories before and after the occlusion; ∝ is an adjustment coefficient that adjusts the relative importance of different factors in the influence calculation, used to reflect the relative importance of different factors in the influence calculation.

[0028] Preferably, A24, calculating the feature point matching degree of the overlapping motion trajectory of the frame images includes:

[0029] A241 highlights marker points on the motion trajectories before and after overlap, serving as feature points for tracking overlap.

[0030] A242, extract the motion trajectory T1 before overlap and the motion trajectory T2 after overlap, and hypothesize a possible motion trajectory T based on the occlusion influence using the difference method. s ;

[0031] A243, T s The feature points on the motion trajectory are matched with the feature points on the motion trajectory before and after the overlap. Based on the calculated matching degree, the degree of matching between the hypothetical motion trajectory and the trajectory before and after the overlap is evaluated.

[0032] S244, if T s If the matching degree of a single feature point is higher than P, then the motion trajectory T is considered to be... s It is reasonable; if T s If there are multiple feature points in the trajectory, and the number of feature points with a matching degree higher than P exceeds 70% of the number of feature points on the overlapping trajectory, then the trajectory T is considered to be... sIt is reasonable; if the trajectory T s If it is unreasonable, then re-assume the trajectory T. s ;

[0033] A245, T1, T s T2 and T2 are connected in the time sequence of the action frames to form a complete motion trajectory.

[0034] Preferably, the formula for calculating the matching degree in A243 includes:

[0035] The formula for calculating the matching degree is:

[0036] N is the number of feature points used to calculate the matching degree, S i D is the shape similarity of the i-th feature point (which can be calculated using shape descriptors such as Hough transform, edge detection, etc.). i V is the directional similarity of the i-th feature point (which can be calculated using the direction vector or angle difference). i α is the velocity similarity of the i-th feature point (which can be calculated using velocity vectors or velocity differences); α, β, and γ are weighting coefficients used to adjust the relative importance of different features in the matching degree calculation.

[0037] Preferably, A22 further includes:

[0038] A221, adjust the feature point acquisition strategy according to body shape, clothing type, image ratio and occlusion relationship;

[0039] A222 dynamically adjusts the occlusion impact threshold based on body shape, clothing type, image ratio, and occlusion relationship;

[0040] The dynamic adjustment of the occlusion impact threshold includes:

[0041] I dynamic =K body *K clothing *K ratio *K occlusion *I

[0042] K body This is a body shape and posture coefficient, adjusted according to the body shape and posture of the individual; K clothing This is a clothing coefficient, adjusted according to the type of clothing worn by the individual; K ratio This is the aspect ratio factor, adjusted according to the proportion of the human body in the image; K occlusion This is the occlusion coefficient, which is adjusted based on the occlusion relationship of the human body in the image.

[0043] Preferably, S4 includes:

[0044] Forward kinematics is used to clarify the joint chain structure and relative positional relationships of the human body. For each marker point in the human joint chain structure, the joint angle of its relative parent joint is determined. The human root coordinate system is selected as the reference point for all calculations. A local coordinate system is established for each joint marker point. The origin of this coordinate system is located at the joint marker point, and the direction of the coordinate axes is determined according to the rotation axis of the joint. The local coordinate system will rotate with the rotation of the joint marker point, but the position of the origin remains unchanged.

[0045] This application also provides a motion capture system, the system comprising:

[0046] The marker setting and tracking module is used to set markers on key parts of the human body in the animation sequence and track and record the pixel coordinates of these markers in the action frames;

[0047] The motion frame recognition and segmentation module is used to identify the start and end motion frames of human body movements based on optical flow field technology, segment the animation sequence, and name the segmented motion frames individually.

[0048] The motion trajectory construction and association module is used to construct the motion trajectory of the marker point based on the collected pixel coordinates and associate and combine related actions;

[0049] The attitude calculation and 3D positioning module is used to calculate the 3D coordinates and spatial attitude angles of the human body in the coordinate system using forward kinematics, and to calculate the precise 3D position of the human body in the animation scene by combining this information.

[0050] The occlusion impact calculation and threshold adjustment module is used to calculate the occlusion impact when there is overlapping or occlusion of marker points, and dynamically adjust the occlusion impact threshold according to body shape, clothing type, screen ratio and occlusion relationship.

[0051] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0052] By setting marker points on key parts of the animated human body and using optical flow technology to calculate pixel motion between adjacent frames, this scheme can accurately identify the start and end frames of human movements, thus effectively defining the complete range of the action. Based on this, the scheme further constructs the motion trajectories of the marker points and, by associating and combining related actions, provides a rich data foundation for subsequent motion analysis. Finally, through the application of forward kinematics, the scheme successfully calculates the 3D coordinates and spatial pose angles of the human body in the coordinate system, achieving precise 3D positioning of the human body in the animation scene.

[0053] By employing fine segmentation of animation sequences and individual naming of action frames, a foundation is laid for the recognition and analysis of human motion trajectories. To address the issue of unrecognizable marker points due to frame overlap, a method for calculating the occlusion impact is introduced. This method comprehensively considers multiple factors, including the area of ​​the occluded image, the proportion of occluded marker points, the proportion of occluded frames, and the differences in the trajectory transitions before and after occlusion, to fully assess the impact of occlusion on the motion trajectory. Possible motion trajectories are hypothesized using the difference method, and feature point matching is performed between these hypotheses and the actual trajectories before and after overlap. The reasonableness of the hypothesized trajectory is judged based on the matching degree. This process considers not only the shape, direction, and velocity similarity of feature points but also adjusts the relative importance of different features in the matching degree calculation through weighting coefficients. Finally, the trajectories before and after overlap are connected with the hypothesized trajectories in chronological order to form a complete motion trajectory, improving the accuracy and coherence of motion trajectory recognition.

[0054] By comprehensively considering body shape, clothing type, image proportion, and occlusion relationships, and dynamically adjusting the feature point acquisition strategy and occlusion impact threshold, this solution not only improves the accuracy of motion trajectory recognition but also allows for flexible adjustments based on different scenes and character characteristics, enhancing the robustness of recognition. By dynamically adjusting the occlusion impact threshold, the impact of occlusion on recognition performance is effectively reduced, maintaining high recognition accuracy even in complex scenes. Furthermore, this solution boasts high adaptability and scalability, providing a more efficient and accurate solution for fields such as animation production and motion analysis. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating a motion capture method according to an embodiment of the present invention;

[0056] Figure 2 This is a structural block diagram of a motion capture system according to an embodiment of the present invention. Detailed Implementation

[0057] To facilitate understanding of the present invention, a more complete description of this application will be given below with reference to the accompanying drawings, which illustrate preferred embodiments of the invention. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to enable a more thorough and complete understanding of the disclosure of the present invention.

[0058] It should be noted that the terms "vertical," "horizontal," "up," "down," "left," "right," and similar expressions used in this article are for illustrative purposes only and do not represent the only possible implementation.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention; the term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0060] Example 1: Figure 1 This is a flowchart illustrating the motion capture method according to an embodiment of the present invention.

[0061] like Figure 1 As shown, a motion capture method includes the following steps:

[0062] S1, acquire scene images containing human body movements in the animation sequence, set marker points on key parts of the human body in the images, and perform frame-by-frame processing on the animation sequence.

[0063] The animation sequence is processed by frame segmentation using the OpenCV open-source computer vision library. The locations of key human body parts marked in the animation frames are mainly joints and points that move when the human body moves. Typically, the head (top of the head and eyes), torso (both shoulders, waist and hips), upper limbs (elbows, wrists and finger joints), and lower limbs (knees, ankles, heels and toes) are selected, and these marked points are saved as coordinate points on the image.

[0064] S2 adds motion frames to the human body after frame segmentation, marks the start and end motion frames for tracking the human body's motion, and tracks and records the pixel coordinates of the marked points in the motion frames.

[0065] In some embodiments, step S2 includes:

[0066] A1 uses the OpenCV computer vision library to read and preprocess the frame-by-frame animation sequence.

[0067] Preprocessing includes grayscale conversion and noise reduction.

[0068] A2 uses optical flow field calculations to determine the motion of pixels between adjacent frames to identify the start and end frames of human actions, thereby determining the action frames.

[0069] Among them, the motion frames for tracking the start and end of human motion include the start frame of motion analysis and the end frame of motion that defines the complete range of motion. The animation frames between the start and end frames are motion frames that contain human motion.

[0070] Specifically, the system identifies the start and end frames of human actions by calculating the motion of pixels between adjacent frames based on optical flow field calculations. The image is Gaussian smoothed, and the gradients in the x and y directions of the smoothed image are calculated. The optical flow field is then calculated using an optical flow algorithm (Lucas-Kanade method). In the first frame of the animation sequence, a SIFT (Signal-Induced Free Rendering) algorithm is used to extract marker points of the human body. For each marker point, the optical flow algorithm is used to calculate its motion trajectory in subsequent frames. When the optical flow velocity of a marker point exceeds a preset threshold, the human action is considered to have started, and that frame is marked as the start frame. The motion trajectory of the marker points is tracked in subsequent frames. When the optical flow velocity of the marker points gradually decreases and stabilizes, the human action is considered to have ended, and that frame is marked as the end frame. All frames between the start and end frames are marked as action frames. The preset threshold for optical flow velocity is automatically adjusted by machine learning training on a large amount of animated human motion data.

[0071] A3, after identifying the starting action frame, tracks and records the pixel coordinates of each marker point in each frame of the action frame.

[0072] S3: Construct the motion trajectory of the marker point based on the collected pixel coordinates, and perform related action association combinations based on the motion trajectory information.

[0073] The motion trajectory includes marker point number, position (such as "left shoulder", "right knee", etc.), frame number (from frame i to frame n), and pixel coordinates (x, y).

[0074] In some embodiments, step S3 includes:

[0075] B1 organizes the pixel coordinates of each punctuation point in each frame of the action frame into a time-continuous data structure.

[0076] The data structure is a sequence of ordered points, where each point contains a frame number and its corresponding pixel coordinates.

[0077] B2. For the data structure of each marker point, process its pixel coordinates in the action frame in sequence, and connect the pixel coordinates of the marker points in the order of the frame number to form the motion trajectory of the marker point.

[0078] B3 combines the motion trajectories between the marked points to obtain the associated motion combinations.

[0079] The motion trajectory combination is as follows: when the action A is performed, the movement of marker point q will cause the movement of marker point w. Therefore, the action A includes marker point q and marker point w. The marker points q and w are combined and associated into a set of motion trajectory combinations.

[0080] S4, associate and combine related actions to calculate the 3D coordinates and spatial posture angles of the human body in the coordinate system.

[0081] Specifically, using positive kinematics, the joint chain structure and relative positional relationships of the human body are clearly defined. For the human arm, it can be simplified into a joint chain composed of markers in the shoulder, elbow and wrist joints. For each marker in the joint chain, its rotation angle (joint angle) relative to the parent joint is clearly defined.

[0082] Choose a base coordinate system (such as the coordinate system at the root of the human body) as the reference point for all calculations; establish a local coordinate system for each joint marker point, with the origin of the coordinate system located at the joint marker point, and the direction of the coordinate axes determined according to the rotation axis of the joint; the local coordinate system will rotate with the rotation of the joint marker point, but the position of the origin remains unchanged; for calculating the transformation matrix and recursively calculating the position and orientation of the end effector, please refer to the calculation steps of forward kinematics, which will not be elaborated in this article.

[0083] S5 combines 3D coordinates and spatial pose angles to calculate the precise 3D position of the human body in the animation scene, and adjusts the result to the coordinate system used by the animation software through coordinate system transformation.

[0084] Specifically, the 3D coordinates of the joint markers of the human body obtained in step S4, as well as the overall or local spatial posture angles, are used to construct a human skeleton model in three-dimensional space. The obtained spatial posture angle information is used to adjust the posture of the skeleton model. Based on the skeleton model and the adjusted posture, the precise 3D position of the human body in the animation scene is calculated. The coordinate system used by the animation software is identified, and the coordinate system of the human 3D model is converted to the coordinate system used by the animation software as needed. The converted human 3D model is imported into the animation software, and the adjusted human 3D model is output in a format that the animation software can recognize.

[0085] The technical solutions described in the embodiments of this application have at least the following technical effects or advantages:

[0086] By setting marker points on key parts of the animated human body and using optical flow technology to calculate pixel motion between adjacent frames, this scheme can accurately identify the start and end frames of human movements, thus effectively defining the complete range of the action. Based on this, the scheme further constructs the motion trajectories of the marker points and, by associating and combining related actions, provides a rich data foundation for subsequent motion analysis. Finally, through the application of forward kinematics, the scheme successfully calculates the 3D coordinates and spatial pose angles of the human body in the coordinate system, achieving precise 3D positioning of the human body in the animation scene.

[0087] Example 2: In actual animation sequences, the marker points of different human figures may overlap or the complex background may obscure the human figures, causing some human figure marker points to be covered by those of other human figures. In this case, if only visible marker points are relied upon to construct the motion trajectory, it is impossible to accurately capture the motion details of the covered human figures. Therefore, the relationship between marker points in adjacent frames and the changes in marker point overlap are extremely important for determining the motion trajectory of the human figure's movement at the start and end frames. Without using the impact value of overlap on the continuity of the motion trajectory to assist in identification, it is difficult to accurately confirm the human figure's motion trajectory.

[0088] Therefore, the embodiments of this application are optimized based on the above embodiments.

[0089] In some embodiments, determining the action frame in step A2 further includes:

[0090] A21 segments the identified motion frames into animation sequences and names each segmented motion frame in the resulting animation sequence separately.

[0091] A22 identifies the motion trajectory of the human body in individually named motion frames, calculates the occlusion influence of motion frames with overlapping frame images that cannot be identified by markers, and determines the impact of the occlusion influence on the motion trajectory.

[0092] The formula for calculating the occlusion impact is as follows:

[0093]

[0094] A cover A represents the area of ​​the image covered, measured in pixels. total The number of covered markers. To cover the proportion of the image area occupied by the human body; N covered To cover the number of frames of the action, N total This represents the total number of marker points in the entire motion trajectory within the action frame. The proportion of the marked points that are obscured to the total number of marked points in the trajectory; F covered F is the number of frames of the action being covered. total This represents the total number of frames in the entire action. ΔD is the proportion of the number of frames covered to the total number of frames in the action; ΔD is the difference measure of the connection between the trajectories before and after the occlusion, defined as the average displacement difference between the trajectories before and after the occlusion at the connection point, which directly reflects the difference between the trajectories before and after the occlusion; ∝ is an adjustment coefficient that adjusts the relative importance of different factors in the influence calculation, used to reflect the relative importance of different factors in the influence calculation. This coefficient is set according to the specific application scenario and data characteristics.

[0095] A23, if the occlusion effect is less than the threshold, the effect of frame image overlap on motion trajectory is ignored.

[0096] The process involves setting a threshold for the degree of occlusion impact. This threshold is used to define the influence of frame image overlap on the connection between motion trajectories before and after overlap. The threshold setting should be based on the specific application scenario and data characteristics. If the occlusion impact is less than the threshold, the influence of frame image overlap on the motion trajectory can be ignored, and it can be considered that occlusion has no significant impact on the recognition and analysis of the motion trajectory. In practical applications, an appropriate threshold range is determined through experiments. For example, a small threshold can be started, and the threshold can be gradually increased to observe the impact on the results of motion trajectory recognition and analysis, thereby determining a threshold range that meets both accuracy requirements and processing efficiency.

[0097] A24. If the occlusion effect is greater than the threshold, calculate the feature point matching degree of the overlapping motion trajectory of the frame images.

[0098] Specifically, calculating the feature point matching degree of the overlapping motion trajectories of frame images includes:

[0099] A241 highlights marker points on the motion trajectories before and after overlap, serving as feature points for tracking overlap.

[0100] A242, extract the motion trajectory T1 before overlap and the motion trajectory T2 after overlap, and hypothesize a possible motion trajectory T based on the occlusion influence using the difference method. s .

[0101] A243, T s The feature points on the motion trajectory are matched with the feature points on the motion trajectory before and after the overlap. Based on the calculated matching degree, the degree of matching between the hypothetical motion trajectory and the trajectory before and after the overlap is evaluated.

[0102] The matching degree is calculated using the following formula:

[0103]

[0104] N is the number of feature points used to calculate the matching degree, S i D is the shape similarity of the i-th feature point (which can be calculated using shape descriptors such as Hough transform, edge detection, etc.). i V is the directional similarity of the i-th feature point (which can be calculated using the direction vector or angle difference). i α is the velocity similarity of the i-th feature point (which can be calculated using velocity vectors or velocity differences); α, β, and γ are weighting coefficients used to adjust the relative importance of different features in the matching degree calculation.

[0105] S244, if T s If the matching degree of a single feature point is higher than P, then the motion trajectory T is considered to be... sIt is reasonable; if T s If there are multiple feature points in the trajectory, and the number of feature points with a matching degree higher than P exceeds 70% of the number of feature points on the overlapping trajectory, then the trajectory T is considered to be... s It is reasonable; if the trajectory T s If it is unreasonable, then re-assume the trajectory T. s .

[0106] Where P is the matching degree threshold, and in practical applications, a suitable threshold is simulated based on specific needs and data characteristics.

[0107] A245, T1, T s T2 and T2 are connected in the time sequence of the action frames to form a complete motion trajectory.

[0108] The technical solutions described in the embodiments of this application have at least the following technical effects or advantages:

[0109] By employing fine segmentation of animation sequences and individual naming of action frames, a foundation is laid for the recognition and analysis of human motion trajectories. To address the issue of unrecognizable marker points due to frame overlap, a method for calculating the occlusion impact is introduced. This method comprehensively considers multiple factors, including the area of ​​the occluded image, the proportion of occluded marker points, the proportion of occluded frames, and the differences in the trajectory transitions before and after occlusion, to fully assess the impact of occlusion on the motion trajectory. Possible motion trajectories are hypothesized using the difference method, and feature point matching is performed between these hypotheses and the actual trajectories before and after overlap. The reasonableness of the hypothesized trajectory is judged based on the matching degree. This process considers not only the shape, direction, and velocity similarity of feature points but also adjusts the relative importance of different features in the matching degree calculation through weighting coefficients. Finally, the trajectories before and after overlap are connected with the hypothesized trajectories in chronological order to form a complete motion trajectory, improving the accuracy and coherence of motion trajectory recognition.

[0110] Example 3: In Example 2, accurate identification of human motion trajectories was achieved by finely segmenting the animation sequence, calculating the occlusion influence, and hypothesizing and verifying motion trajectories. To apply this method to different human bodies, different outfits, different body shapes, and different proportions, the feature point acquisition strategy needs to be flexibly adjusted. For example, more stable marker points can be selected based on human body characteristics and outfits, and the motion trajectory recognition algorithm can be optimized to adapt to changes brought about by different body shapes and proportions, thereby ensuring the accuracy and robustness of motion trajectory recognition.

[0111] Therefore, the embodiments of this application are optimized based on the above embodiments.

[0112] In some embodiments, step A22 further includes:

[0113] A221 adjusts the feature point acquisition strategy based on body shape, clothing type, image ratio, and occlusion relationship.

[0114] Specifically, image processing technology is used to identify human body shape and posture, and a body shape coefficient K is set according to the body shape and posture type. body Among them, body types include slender and tall, short and stout, etc.; the clothing type of the human body is identified through image recognition technology, and a clothing coefficient K is set according to the clothing type. clothing The clothing types included tight-fitting clothes, loose-fitting clothes, and heavy coats; the proportion of the human body in the image was calculated, and the image proportion coefficient K was set according to the image proportion. ratio The proportion refers to the ratio of the human body area to the image area; the occlusion relationship of human bodies in the image is identified, and an occlusion relationship coefficient K is set based on the occlusion relationship. occlusion Among them, the occlusion relationship is used to determine which human figures are in front and which human figures are behind.

[0115] For example, for slender human bodies, the density of joint points can be increased; for loose clothing, more stable feature points can be selected, and the adjusted feature point acquisition strategy can be used for motion trajectory recognition.

[0116] A222 dynamically adjusts the occlusion impact threshold based on body shape, clothing type, image ratio, and occlusion relationship.

[0117] The formula for dynamically adjusting the occlusion impact threshold is as follows:

[0118] I dynamic =K body *K clothing *K ratio *K occlusion *I

[0119] K body This is a body shape coefficient, adjusted according to the individual's body shape (e.g., tall and thin, short and stout, etc.); K clothing This is the clothing coefficient, adjusted according to the type of clothing worn (e.g., tight-fitting clothes, loose clothes, etc.); K ratio This is the aspect ratio factor, adjusted according to the proportion of the human body in the image; K occlusion This is the occlusion coefficient, which is adjusted based on the occlusion relationship of the human body in the image.

[0120] For example, in an animated film, there are multiple characters interacting in a scene. Their body shapes, clothing, and positional relationships are all different. It is necessary to dynamically adjust the feature point acquisition strategy and influence based on this information to improve the accuracy of motion trajectory recognition.

[0121] Body shape and physique settings: Slender character: Setting K body=1.2, because slender characters may have a larger range of joint motion, requiring a higher feature point acquisition density; short and stout characters: set K body =1.0, because the body contour changes of short and stout characters may be more obvious, but no additional feature point acquisition density is needed; symmetrical characters: set K body =1.1, as an intermediate value;

[0122] Clothing coefficient setting: Bodysuit character: Setting K clothing =1.0, because feature points under tight-fitting clothes are easier to identify and track; Loose-fitting character: Set K clothing =1.2, because loose clothing may obscure some feature points, requiring a higher feature point acquisition density; Heavy coat character: set K clothing =1.4, because the heavy coat obscures the view more severely, requiring a higher feature point acquisition density and resolution;

[0123] Screen aspect ratio settings: Small-scale characters (occupying less than 10% of the screen): Set to K ratio =1.5, because higher feature point acquisition density and resolution are required; Medium-sized characters (occupying 10%-50% of the screen): Set K ratio =1.2, as an intermediate value; Large-scale characters (occupying more than 50% of the screen): Set K ratio =1.0, because the feature point acquisition density and resolution are already sufficient;

[0124] Occlusion Coefficient Setting: Unoccupied Characters: Setting K occlusion =1.0, because the feature points are fully visible; partially occluded characters (occlusion ratio less than 50%): set K occlusion =1.2, because an algorithm is needed to predict or estimate the location of occluded feature points; for completely occluded characters (occlusion ratio greater than 50%): set K... occlusion =1.5, because a higher feature point acquisition density and a more complex algorithm are needed to handle occlusion relationships;

[0125] Suppose in a certain frame, there is a tall, thin character wearing loose clothing, occupying 20% ​​of the frame, and partially obscured (30% obscuration). According to the formula, the dynamic adjustment impact is calculated as follows:

[0126] I dynamic =1.2×1.2×1.2×1.2×I. Through this method, the acquisition strategy and influence of animated human feature points can be dynamically adjusted according to different body shapes, clothing, screen size, and front and back occlusion relationships, thereby improving the accuracy and robustness of motion trajectory recognition.

[0127] The technical solutions described in the embodiments of this application have at least the following technical effects or advantages:

[0128] By comprehensively considering body shape, clothing type, image proportion, and occlusion relationships, and dynamically adjusting the feature point acquisition strategy and occlusion impact threshold, this solution not only improves the accuracy of motion trajectory recognition but also allows for flexible adjustments based on different scenarios and human characteristics, enhancing the robustness of recognition. By dynamically adjusting the occlusion impact threshold, the impact of occlusion on recognition performance is effectively reduced, maintaining high recognition accuracy even in complex scenarios. Furthermore, this solution exhibits high adaptability and scalability, providing a more efficient and accurate solution for fields such as animation production and motion analysis.

[0129] Furthermore, embodiments of the present invention also provide a motion capture system.

[0130] Figure 2 This is a schematic diagram of the motion capture system according to an embodiment of the present invention.

[0131] like Figure 2 As shown, a motion capture system includes: a marker point setting and tracking module, an action frame recognition and segmentation module, a motion trajectory construction and association combination module, and a posture calculation and 3D positioning module.

[0132] Marker point setting and tracking module: used to set marker points on key parts of the human body in the animation sequence and track and record the pixel coordinates of these marker points in the action frames.

[0133] Action frame recognition and segmentation module: Based on optical flow field technology, it identifies the start and end action frames of human body movements, segments the animation sequence, and names the segmented action frames individually.

[0134] Motion trajectory construction and association module: Constructs the motion trajectory of the marker point based on the collected pixel coordinates, and associates and combines related actions.

[0135] Attitude calculation and 3D positioning module: Uses forward kinematics to calculate the 3D coordinates and spatial attitude angles of the human body in the coordinate system, and combines this information to calculate the precise 3D position of the human body in the animation scene.

[0136] Occlusion impact calculation and threshold adjustment module: When there is overlap or occlusion of marker points, calculate the occlusion impact and dynamically adjust the occlusion impact threshold according to body shape, clothing type, screen ratio and occlusion relationship.

[0137] It should be noted that other specific implementations of the motion capture system in this embodiment of the invention can refer to the above-described motion capture method.

[0138] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A motion capture method, characterized in that, The method includes: S1, acquire scene images containing human body movements in the animation sequence, set marker points on key parts of the human body in the images, and perform frame-by-frame processing on the animation sequence; S2, add motion frames to the human body after frame segmentation, mark the start and end motion frames for tracking the human body, and track and record the pixel coordinates of the marked points in the motion frames; wherein, the animation frames between the start and end motion frames are motion frames containing human body movements; use the OpenCV computer vision library to read and preprocess the animation sequence after frame segmentation; use optical flow field to calculate the motion of pixels between adjacent frames to identify the start and end motion frames of human body movements, thereby determining the motion frames; wherein, the start and end motion frames for tracking human body movements include the start motion frame at the start point of motion analysis and the end motion frame at the end point that defines the complete range of the motion; The identified motion frames are segmented into animation sequences, and the segmented motion frames of the animation sequence are named individually. The motion trajectory of the human body in the individually named motion frames is identified. For motion frames with overlapping frames where the marker points cannot be identified, the occlusion influence of the overlap is calculated, and the influence of the occlusion influence on the motion trajectory is determined. If the occlusion influence is less than the threshold, the influence of the overlapping frames on the motion trajectory is ignored. The feature point acquisition strategy is adjusted according to body shape, clothing type, screen ratio, and occlusion relationship. If the occlusion impact is greater than the threshold, the feature point matching degree of the overlapping motion trajectory of the frame images is calculated; highlighted points are added to the motion trajectory before and after overlap as feature points for tracking the overlap; the motion trajectory T1 before overlap and the motion trajectory T2 after overlap are extracted, and a preset motion trajectory T is constructed based on the occlusion impact using the interpolation method. s ; T s The feature points on the T path are matched with the feature points on the motion trajectories before and after overlap. Based on the calculated matching degree, the degree of matching between the hypothetical motion trajectory and the trajectories before and after overlap is evaluated; if T s The matching degree of a single feature point is higher than that of a single feature point. Then the trajectory T is considered to be s It is reasonable; if T s There are multiple feature points in the data, and the matching degree among these feature points is higher than that of the other feature points. If the number of feature points exceeds 70% of the total number of feature points on the overlapping motion trajectory, then the motion trajectory T is considered to be... s It is reasonable; if the trajectory T s If it is unreasonable, then re-assume the trajectory T. s ;T1, T s T2 and T2 are connected in the time sequence of the action frames to form a complete motion trajectory; After identifying the starting frame, the pixel coordinates of each marker point in each frame of the action frame are tracked and recorded; S3, construct the motion trajectory of the marker point based on the collected pixel coordinates, and perform related action associations based on the motion trajectory; wherein, the motion trajectory includes the marker point number, position, frame number and pixel coordinates (x,y). S4, combine related actions to calculate the human body's 3D coordinates and spatial posture angles in the coordinate system; S5 combines 3D coordinates and spatial pose angles to calculate the precise 3D position of the human body in the animation scene, and adjusts the result to the coordinate system used by the animation software through coordinate system transformation.

2. The motion capture method as described in claim 1, characterized in that, The S3 includes: B1 organizes the pixel coordinates of each punctuation point in each frame of the action frame into a time-continuous data structure. B2, for the data structure of each marker point, process its pixel coordinates in the action frame in sequence, and connect the pixel coordinates of the marker points in the order of the frame number to form the motion trajectory of the marker point; B3 combines the motion trajectories between the marked points to obtain the associated motion combinations.

3. The motion capture method as described in claim 1, characterized in that, The calculation of the occlusion impact includes: The formula for calculating the occlusion impact is: The area of ​​the image covered is measured in pixels. The number of covered markers. To cover the proportion of the human body area in the image; To cover the number of frames of the action frame, This represents the total number of marker points in the entire motion trajectory within the action frame. The proportion of the marked points that are covered out of the total marked points in the trajectory; The number of frames in the occlusion action. This represents the total number of frames in the entire action. The proportion of the number of frames to be covered out of the total number of action frames; To measure the difference at the junction of the preceding and following trajectories; This is an adjustment factor used to adjust the relative importance of different factors in the influence calculation.

4. The motion capture method as described in claim 1, characterized in that, The formula for calculating the matching degree includes: The formula for calculating the matching degree is: N is the number of feature points used to calculate the matching degree. It is the shape similarity of the i-th feature point. It is the directional similarity of the i-th feature point. It is the velocity similarity of the i-th feature point; , , It is a weighting coefficient used to adjust the relative importance of different features in the matching degree calculation.

5. The motion capture method as described in claim 1, characterized in that, The method of identifying the motion trajectory of a human body in a individually named action frame also includes: Adjust the feature point acquisition strategy according to body shape, clothing type, image ratio, and occlusion relationship; The threshold for occlusion impact is dynamically adjusted based on body shape, clothing type, image proportion, and occlusion relationship. The dynamic adjustment of the occlusion impact threshold includes: This is a body shape and posture coefficient, adjusted according to the body shape and posture of the human body; This is a clothing coefficient, adjusted according to the type of clothing worn by the individual; This is the aspect ratio factor, adjusted according to the proportion of the human body in the image; This is the occlusion coefficient, which is adjusted based on the occlusion relationship of the human body in the image.

6. The motion capture method as described in claim 1, characterized in that, S4 includes: using forward kinematics to clarify the joint chain structure and relative positional relationships of the human body; clarifying the joint angle of the relative parent joint for each marker point in the human joint chain structure; selecting the human root coordinate system as the reference point for all calculations; establishing a local coordinate system for each joint marker point, with the origin of the coordinate system located at the joint marker point and the direction of the coordinate axes determined according to the rotation axis of the joint; the local coordinate system will rotate with the rotation of the joint marker point, but the position of the origin remains unchanged.

7. A motion capture method-based system, applied to a motion capture method as described in any one of claims 1 to 6, characterized in that, The system includes: The marker setting and tracking module is used to set markers on key parts of the human body in the animation sequence and track and record the pixel coordinates of these markers in the action frames; The motion frame recognition and segmentation module is used to identify the start and end motion frames of human body movements based on optical flow field technology, segment the animation sequence, and name the segmented motion frames individually. The motion trajectory construction and association module is used to construct the motion trajectory of the marker point based on the collected pixel coordinates and associate and combine related actions; The attitude calculation and 3D positioning module is used to calculate the 3D coordinates and spatial attitude angles of the human body in the coordinate system using forward kinematics, and to calculate the precise 3D position of the human body in the animation scene by combining this information. The occlusion impact calculation and threshold adjustment module is used to calculate the occlusion impact when there is overlapping or occlusion of marker points, and dynamically adjust the occlusion impact threshold according to body shape, clothing type, screen ratio and occlusion relationship.

Images