Rigid body linkage design method and system based on human body model

Abstract

The invention relates to the technical field of motion capture, in particular to a rigid body linkage design method and system based on a human body model.The method comprises the steps that three-dimensional key points on a user body are extracted from camera calibration data based on a two-dimensional key point detection algorithm and a triangularization technology; a Python interface of Blender is used, rigid bodies are drawn according to the three-dimensional key points, and each rigid body is defined by two mark points; generating a corresponding coordinate axis for each three-dimensional mark point, and linking the direction of the coordinate axis and the Euler angle of the mark point through a parent class inheritance mechanism to obtain each enhanced three-dimensional mark point; and writing the enhanced three-dimensional mark points into a human body model file, and carrying out real-time linkage of the rigid body and the human body model through binding with skeleton joints. According to the invention, the problems of tedious operation and poor user experience of the traditional mark point motion capture method are effectively solved, the defects of the existing mark point-free technology in rigid body rendering and linkage are overcome, and high-quality mark point-free human body motion capture and visualization are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of motion capture, and in particular to a rigid body linkage design method and system based on a human body model. Background Art

[0002] With the continuous development of human motion capture technology, its application in medicine, sports, animation and other fields is becoming more and more extensive. At present, the motion capture method based on marker points requires users to wear reflective balls on their bodies and capture the trajectory of marker points through cameras to achieve three-dimensional modeling. Although this method has accurate positioning, it is cumbersome to operate and has a poor user experience. The marker-free technology directly extracts the key points of the human body through algorithms, avoiding the inconvenience of wearing equipment.

[0003] At the same time, the existing markerless point technology has deficiencies in rigid body rendering and linkage, and it is difficult to meet the needs of high-quality visualization; in addition, although 3D modeling tools such as Blender have powerful rendering capabilities, their application is mainly limited to static modeling and lacks the ability to interact with human motion data in real time. Summary of the invention

[0004] The present invention provides a rigid body linkage design method and system based on a human body model, thereby effectively solving the problems pointed out in the background technology.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: A rigid body linkage design method based on a human body model, comprising: Based on the 2D key point detection algorithm and triangulation technology, the 3D key points on the user's body are extracted from the camera calibration data to generate real-time 3D posture data reflecting human body movement; Using Blender's Python interface, draw rigid bodies according to the three-dimensional key points, each of the rigid bodies is defined by two marker points; Generate a corresponding coordinate axis for each three-dimensional marker point, and through the parent class inheritance mechanism, link the direction of the coordinate axis with the Euler angle of the marker point to obtain enhanced three-dimensional marker points; The enhanced three-dimensional marking points are written into the human body model file, and the rigid body and the human body model are linked in real time by binding with the skeletal joints.

[0006] Furthermore, based on the 2D key point detection algorithm and triangulation technology, the 3D key points on the user's body are extracted from the camera calibration data to generate real-time 3D posture data reflecting human body movement, including: Collect video data of the user during movement, and use the camera calibration file to obtain the internal and external parameters of each camera; Process multi-view videos based on a two-dimensional key point detection algorithm to extract the two-dimensional joint coordinates of the human body; Using the camera calibration data and the coordinates of the two-dimensional key points in multiple perspectives, the three-dimensional coordinates of the key points of the human body are calculated through triangulation technology to generate the three-dimensional key points on the user's body; A filtering algorithm is applied to the calculated three-dimensional key point data for optimization, and real-time three-dimensional posture information reflecting the user's actions is generated based on the optimized three-dimensional key point data.

[0007] Furthermore, the rigid body is represented by the geometric shape of a cone, the bottom is composed of a circular base surface defined by a first marking point, the first marking point is the bottom center of the rigid cone, and the top is formed by the convergence of the tip defined by a second marking point, and the second marking point is the top cone tip of the rigid cone.

[0008] Furthermore, through the frame-by-frame update algorithm, the coordinates of the user's real-time 3D key points are input into the Blender script to dynamically generate a rigid body model for each frame.

[0009] Furthermore, the frame-by-frame update algorithm includes: Calculating the rotation Euler angle of each of the marking points according to the real-time three-dimensional posture data; Based on the rotation Euler angles, a corresponding rotation matrix is ​​calculated, and the direction of the coordinate axis is updated in real time according to the rotation matrix; The position and direction of the three-dimensional marker point are dynamically updated according to the three-dimensional coordinates and rotation matrix calculated for each frame.

[0010] Furthermore, a corresponding coordinate axis is generated for each three-dimensional marker point, and the direction of the coordinate axis and the Euler angle of the marker point are linked through a parent class inheritance mechanism to obtain enhanced three-dimensional marker points, including: Generate corresponding coordinate axes according to each three-dimensional marking point, wherein the coordinate axes include an X axis, a Y axis and a Z axis, wherein the X axis and the Y axis are parallel to the joint cross-sectional plane, and the Z axis is perpendicular to the joint cross-sectional plane; Through the parent class inheritance mechanism, the rotation Euler angle of the marker point is linked with the coordinate axis direction, and the coordinate axis direction is updated in real time with the marker point; The dynamically updated coordinate axis is bound to the marking point to form an enhanced three-dimensional marking point, wherein the enhanced marking point includes direction, rotation Euler angle and linkage information with the joint.

[0011] Furthermore, the enhanced three-dimensional markers are written into the human body model file, and the rigid body and the human body model are linked in real time by binding with the skeletal joints, including: Create a human model file that records the three-dimensional position, rotation center and range of motion of bone joints, and establish a hierarchical structure of bones; Writing the enhanced data of the three-dimensional marker points into a human body model file and establishing a dynamic dependency relationship with the skeletal joints; Define the skeleton joint as the parent class object of the rigid body, and bind the bottom and top coordinates of the rigid body to the skeleton joint through the enhanced three-dimensional marker points; As the position and rotation state of the skeletal joints change in real time, the three-dimensional position and orientation of the enhanced markers and rigid bodies are dynamically updated.

[0012] Furthermore, the triangulation algorithm is least squares triangulation.

[0013] A rigid body linkage design system based on a human body model, the system comprising: The pose data generation module extracts the 3D key points on the user's body from the camera calibration data based on the 2D key point detection algorithm and triangulation technology, and generates real-time 3D pose data reflecting human body movement; A rigid body definition drawing module uses Blender's Python interface to draw rigid bodies according to the three-dimensional key points, each of which is defined by two marking points; The marker point enhancement module generates a corresponding coordinate axis for each three-dimensional marker point, and through the parent class inheritance mechanism, the direction of the coordinate axis and the Euler angle of the marker point are linked to obtain the enhanced three-dimensional marker points; The rigid body model linkage module writes the enhanced three-dimensional marking points into the human body model file, and performs real-time linkage between the rigid body and the human body model by binding with the bone joints.

[0014] Furthermore, the posture data generation module includes: An internal and external parameter acquisition unit collects video data during the user's movement and uses a camera calibration file to obtain the internal and external parameters of each camera; A two-dimensional coordinate extraction unit processes multi-view videos based on a two-dimensional key point detection algorithm to extract the two-dimensional joint point coordinates of the human body; A three-dimensional key point generation unit, which uses the camera calibration data and the two-dimensional key point coordinates of multiple perspectives to calculate the three-dimensional coordinates of the human body key points through triangulation technology to generate the three-dimensional key points on the user's body; The data filtering and optimization unit applies a filtering algorithm to optimize the calculated three-dimensional key point data, and generates real-time three-dimensional posture information reflecting the user's actions based on the optimized three-dimensional key point data.

[0015] The technical solution of the present invention can achieve the following technical effects: It effectively solves the problems of cumbersome operation and poor user experience of traditional marker point motion capture methods, overcomes the shortcomings of existing markerless point technology in rigid body rendering and linkage, and realizes high-quality markerless human motion capture and visualization. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flowchart of a rigid body linkage design method based on a human body model; Figure 2 A schematic diagram of the process for generating real-time three-dimensional pose data reflecting human body motion; Figure 3 It is a flowchart of the frame-by-frame update algorithm; Figure 4 A schematic diagram of the process of obtaining enhanced three-dimensional marker points; Figure 5 Schematic diagram of the process for real-time linkage between rigid body and human body model. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0020] Embodiment 1 like Figure 1 As shown, the present invention provides a rigid body linkage design method based on a human body model, the method comprising: S1: Based on the 2D key point detection algorithm and triangulation technology, the 3D key points on the user's body are extracted from the camera calibration data to generate real-time 3D pose data reflecting human body movement; Specifically, based on the two-dimensional key point detection algorithm and triangulation technology, three-dimensional key points are extracted from the two-dimensional image data of the user's body captured by the multi-view camera, and then real-time three-dimensional pose data reflecting the human body's motion state is generated. The core purpose of this process is to realize marker-free human motion capture, avoiding the tedious operation of wearing markers or reflective balls in traditional methods. At the same time, accurate three-dimensional pose data is obtained through calculation, providing a reliable foundation for subsequent rigid body modeling, rendering and linkage.

[0021] S2: Use Blender's Python interface to draw rigid bodies based on 3D key points, each of which is defined by two marker points; Specifically, the purpose of using Blender's Python interface is to convert the 3D key point data extracted from the camera into a virtual rigid body model in real time, and dynamically update the position and rotation angle of the rigid body based on these data, so as to achieve instant visualization of human movement. In this way, the virtual rigid body can be synchronized with human movement, improving the movement performance and interactivity in the virtual environment. Blender's Python interface also automates the rigid body creation and update process, reduces manual operations, improves efficiency, and ensures high-precision control. By writing scripts, the position and rotation of the rigid body can be adjusted in real time, and the motion data can be accurately applied to the virtual model.

[0022] S3: Generate corresponding coordinate axes for each 3D marker point, and through the parent class inheritance mechanism, link the direction of the coordinate axis with the Euler angle of the marker point to obtain enhanced 3D marker points; Specifically, by associating each 3D marker point with the corresponding coordinate axis and using the parent class inheritance mechanism to link the direction of the coordinate axis with the rotation angle (Euler angle) of the marker point, the expressiveness of the 3D marker point is enhanced. The direction of the coordinate axis will be updated in real time as the marker point rotates, ensuring that each 3D marker point not only contains position data, but also reflects its rotation and direction information. In this way, the representation of the 3D marker point is more comprehensive and can more accurately express the movement state of various parts of the human body. Using the parent class inheritance mechanism, the rotation changes of the marker point can be synchronously updated to the coordinate axis to ensure that the linkage between the marker point and the rigid body remains consistent. This not only improves the accuracy of motion capture, but also makes the virtual human model more natural and realistic when moving.

[0023] S4: Write the enhanced three-dimensional marker points into the human body model file, and perform real-time linkage between the rigid body and the human body model by binding with the bone joints.

[0024] Specifically, the enhanced three-dimensional marker points provide more comprehensive motion information, including rotation angle and direction, so that each marker point can accurately reflect the dynamic changes of human movement. By binding these marker points to the human skeletal joints, it can ensure that the movement of the rigid body model is updated synchronously with the movement of the human joints. When the position or rotation state of the skeletal joints changes, the rigid body model will adjust its position and direction in real time to maintain a high degree of consistency with the human body's movements.

[0025] The present invention effectively solves the problems of cumbersome operation and poor user experience of traditional marker point motion capture methods, overcomes the deficiencies of existing marker-free technologies in rigid body rendering and linkage, and realizes high-quality marker-free human motion capture and visualization.

[0026] As a preferred embodiment of the above, Figure 2 As shown, step S2, based on a two-dimensional key point detection algorithm and triangulation technology, extracts three-dimensional key points on the user's body from the camera calibration data to generate real-time three-dimensional posture data reflecting human body movement, including: S11: Collect video data of the user during movement, and use the camera calibration file to obtain the internal and external parameters of each camera; S12: Process the multi-view video based on the two-dimensional key point detection algorithm to extract the two-dimensional joint coordinates of the human body; S13: using the camera calibration data and the coordinates of the two-dimensional key points of the multi-view, the three-dimensional coordinates of the key points of the human body are calculated through triangulation technology to generate the three-dimensional key points on the user's body; S14: Apply a filtering algorithm to optimize the calculated three-dimensional key point data, and generate real-time three-dimensional posture information reflecting the user's actions based on the optimized three-dimensional key point data.

[0027] Specifically, the user's movements need to be captured by the camera system to generate multi-view video data. In this process, the camera's internal and external parameters (such as focal length, optical center, camera rotation and displacement, etc.) need to be obtained through camera calibration. These parameters are the key to calculating the transformation relationship from the world coordinate system to the image coordinate system for each camera; use a two-dimensional key point detection algorithm (such as OpenPose or other deep learning methods) to process each frame of video images, and extract the two-dimensional joint point coordinates of key parts of the human body (such as head, shoulders, elbows, knees, etc.) from different perspectives. These two-dimensional coordinates are the basic data for subsequent three-dimensional reconstruction; through triangulation technology, the two-dimensional key point coordinates under different perspectives and the camera's internal and external parameters are combined to map the two-dimensional coordinates into three-dimensional space, and calculate the three-dimensional coordinates of each key point. The triangulation technology uses the different positions of the same human body point seen from multiple camera perspectives to infer the three-dimensional spatial position of the point through geometric principles; the obtained three-dimensional key point data may be affected by noise, occlusion and other factors, so it is necessary to apply a filtering algorithm (such as Kalman filtering or smoothing algorithm) to optimize these data. The optimized data will generate real-time three-dimensional pose information to ensure the accurate and smooth performance of human body movements in three-dimensional space.

[0028] As a preferred embodiment of the above embodiment, the rigid body is represented by the geometric shape of a cone, the bottom is composed of a circular base surface defined by a first marking point, the first marking point is the bottom center of the rigid cone, and the top is formed by the convergence of the tip defined by the second marking point, and the second marking point is the top cone tip of the rigid cone.

[0029] Specifically, the rigid body is represented by the geometric shape of a cone, and the bottom is composed of a circular base surface defined by the first marking point, where the first marking point is the center of the bottom circle of the cone and determines the position of the bottom surface of the cone. The size of this circular base surface can be set as needed, determined by the position of the first marking point and the defined radius; the top of the cone is defined by the second marking point, which represents the top tip position of the cone and determines the height and shape of the cone. According to the positional relationship between the two marking points, the height and angle of the cone can be accurately calculated, thereby completely defining the geometric shape of the rigid body. This cone model is not only a geometric representation of the rigid body, but also can adjust its position, rotation angle and shape in real time with the dynamic update of the two marking points. When the motion data of the human body changes, the geometric shape and orientation of the cone will respond and update in real time, thereby ensuring the precise linkage between the rigid body model and the human body model.

[0030] As a preferred embodiment of the above, the coordinates of the user's real-time three-dimensional key points are input into the Blender script through a frame-by-frame update algorithm to dynamically generate a rigid body model for each frame.

[0031] Specifically, first, the frame-by-frame update algorithm will obtain the three-dimensional key point coordinates of each frame of the user's movement. These coordinates have been calculated and optimized by the previous algorithm, reflecting the real-time position and posture of various parts of the human body; next, the Blender script will pass these real-time three-dimensional key point coordinates to Blender's model generation system. In Blender, the script will automatically create and adjust the position, rotation and other related properties of the rigid body model based on the input three-dimensional key point data to ensure that the rigid body and the human body model are synchronized in movement. The data of each frame will be input into Blender to update the shape and direction of the model in real time. In this way, the rigid body model can be dynamically generated as the human body's movement changes, and the rigid body of each frame can accurately reflect the real-time posture and movement of the human body.

[0032] As a preferred embodiment of the above, Figure 3 As shown, the frame-by-frame update algorithm includes: A10: Calculate the rotation Euler angle of each marker point based on the real-time 3D pose data; A20: Based on the Euler angle rotation, the corresponding rotation matrix is ​​calculated, and the direction of the coordinate axis is updated in real time according to the rotation matrix; A30: Dynamically update the position and orientation of the 3D marker based on the 3D coordinates and rotation matrix calculated for each frame.

[0033] Specifically, in each frame of data, the rotation Euler angles of each marker point are first calculated based on the user's three-dimensional posture data. The Euler angles represent the rotation state of the marker point and are usually represented by rotation angles around three axes. Once the rotation Euler angles are obtained, the rotation matrix needs to be calculated using these Euler angles. The rotation matrix is ​​a mathematical tool used to describe the rotation of objects in space. The direction of the coordinate axes of the marker points can be updated using the rotation matrix. Finally, based on the calculated rotation matrix and three-dimensional coordinates, the position and direction of the three-dimensional marker points of each frame are dynamically updated. In each frame, the position and rotation of the marker points will be adjusted in real time according to the new data to ensure that the position and direction of the marker points in three-dimensional space are consistent with the current state of motion.

[0034] As a preferred embodiment of the above, Figure 4 As shown, step S3 generates corresponding coordinate axes for each 3D marker point, and through the parent class inheritance mechanism, the direction of the coordinate axis and the Euler angle of the marker point are linked to obtain enhanced 3D marker points, including: S31: Generate corresponding coordinate axes according to each three-dimensional marker point, the coordinate axes include an X axis, a Y axis and a Z axis, the X axis and the Y axis are parallel to the joint section plane, and the Z axis is perpendicular to the joint section plane; S32: Through the parent class inheritance mechanism, the rotation Euler angle of the marker point is linked with the coordinate axis direction, and the coordinate axis direction is updated in real time with the marker point; S33: Bind the dynamically updated coordinate axis to the marker point to form an enhanced three-dimensional marker point, where the enhanced marker point includes direction, rotation Euler angle and linkage information with the joint.

[0035] Specifically, on the basis of each three-dimensional marker point, a corresponding coordinate axis is generated. These coordinate axes include X-axis, Y-axis and Z-axis. The X-axis and Y-axis are parallel to the cross-sectional plane of the human joint, while the Z-axis is perpendicular to the cross-sectional plane of the joint. In this way, each marker point will have an independent coordinate system to represent the direction and rotation information of the marker point. The definition of the X-axis, Y-axis and Z-axis provides a reference frame for subsequent rotation and direction updates. Through the parent class inheritance mechanism, the rotation Euler angle of the marker point is linked with the direction of the coordinate axis. The parent class is defined as a bone joint, and the child class objects include three-dimensional marker points and coordinate axes. The rotation Euler angle of the marker point (indicating the angle information of the rotation) will update the direction of the coordinate axis in real time. Through the inheritance mechanism, the rotation state of the coordinate axis will be updated synchronously with the rotation angle of the marker point to ensure that the marker point and its coordinate axis direction are always consistent. In each frame update, the dynamically updated coordinate axis is bound to the corresponding marker point to form an enhanced three-dimensional marker point. These enhanced marker points contain three aspects of information: position (coordinates), direction (coordinate axis direction) and rotation Euler angle.

[0036] As a preferred embodiment of the above, Figure 5 As shown, step S4, writing the enhanced three-dimensional marker points into the human body model file, and performing real-time linkage between the rigid body and the human body model by binding with the skeletal joints, including: S41: Create a human model file that records the three-dimensional position, rotation center and range of motion of bone joints, and establish a hierarchical structure of bones; S42: writing the enhanced three-dimensional marker point data into the human body model file and establishing a dynamic dependency relationship with the skeletal joints; S43: Define the skeleton joint as the parent class object of the rigid body, and bind the bottom and top coordinates of the rigid body to the skeleton joint through the enhanced 3D marker points; S44: Dynamically update the 3D position and orientation of the enhanced markers and rigid bodies as the position and rotation of the skeletal joints change in real time.

[0037] Specifically, first, a 3D model file containing bone joints needs to be created. The file records the position, rotation center, and range of motion of each bone joint. The bone system usually adopts a hierarchical structure, in which each bone joint (such as shoulder, elbow, knee, etc.) is connected to other joints through a parent-child relationship to form a complete skeleton structure. This hierarchical structure ensures the relative motion relationship between each joint, and can automatically affect the motion of other joints when the bone rotates; Next, write the enhanced 3D marker points (including position, rotation angle, and direction information) into the human body model file, and establish a dynamic dependency relationship between these marker points and the corresponding bone joints. Each enhanced 3D marker point corresponds to a bone joint and is dynamically updated as the bone rotates and changes in position. This dependency ensures that the movement of the marker point is always synchronized with the movement of the bone joint; in this part, the bone joint is defined as the parent class object of the rigid body. Through the parent class inheritance mechanism, the bottom and top coordinates of the rigid body are bound to the bone joints, and the enhanced three-dimensional markers provide accurate spatial position and direction for the bottom and top of the rigid body. The bottom coordinate is defined by the first marker (such as the base position of the joint), and the top coordinate is defined by the second marker (such as the end position of the joint), ensuring that the rigid body model can accurately reflect the dynamic movement of the bone joints; finally, with the real-time changes in the position and rotation state of the bone joints, the enhanced three-dimensional markers and the three-dimensional position and direction of the rigid body will be dynamically updated. Whenever the bone joints rotate or displace, the coordinates and direction of the rigid body model will be synchronously updated according to the movement of the bone joints, ensuring real-time linkage between the human body model and the rigid body.

[0038] As a preferred embodiment of the above embodiment, the triangulation algorithm is least squares triangulation.

[0039] Specifically, in the process of using least squares triangulation to calculate three-dimensional coordinates, the two-dimensional key point coordinates of each joint of the human body are first extracted from the video data of multiple perspectives. These coordinates are captured by different cameras at different angles, and the intrinsic and extrinsic parameters of each camera (such as focal length, rotation matrix and displacement vector, etc.) are obtained through camera calibration; then, the core step of least squares triangulation is to establish a reprojection error model through these two-dimensional coordinates and the calibration parameters of the camera. The reprojection error represents the difference between the two-dimensional projection coordinates calculated by the intrinsic and extrinsic parameters of a point in three-dimensional space and the actual observed two-dimensional coordinates. The goal of the least squares algorithm is to minimize the sum of squares of these errors to ensure that the calculated three-dimensional coordinates best match the actual observed data; in the optimization process, the least squares method iteratively adjusts the position of each three-dimensional point so that the error between the two-dimensional coordinates projected from multiple perspectives and the true observation value is minimized, and finally the accurate three-dimensional coordinates are obtained. In order to improve the calculation accuracy, filtering algorithms (such as Kalman filtering) are usually introduced to smooth the data, remove sensor noise and other interference factors, thereby further improving the accuracy of three-dimensional reconstruction. Through this method, the three-dimensional coordinates of each joint of the human body can be accurately restored from the two-dimensional data of multiple perspectives, ensuring the accuracy of the motion capture data.

[0040] Embodiment 2 Based on the same inventive concept as the rigid body linkage design method based on a human body model in the aforementioned embodiment, the present invention also provides a rigid body linkage design system based on a human body model, the system comprising: The pose data generation module extracts the 3D key points on the user's body from the camera calibration data based on the 2D key point detection algorithm and triangulation technology, and generates real-time 3D pose data reflecting human body movement; The rigid body definition drawing module uses Blender's Python interface to draw rigid bodies based on 3D key points. Each rigid body is defined by two marker points. The marker enhancement module generates the corresponding coordinate axis for each 3D marker point. Through the parent class inheritance mechanism, the direction of the coordinate axis and the Euler angle of the marker point are linked to obtain the enhanced 3D marker points. The rigid body model linkage module writes the enhanced three-dimensional marker points into the human body model file, and performs real-time linkage between the rigid body and the human body model by binding with the bone joints.

[0041] As a preferred embodiment of the above, the posture data generation module includes: An internal and external parameter acquisition unit collects video data during the user's movement and uses a camera calibration file to obtain the internal and external parameters of each camera; A two-dimensional coordinate extraction unit processes multi-view videos based on a two-dimensional key point detection algorithm to extract the two-dimensional joint point coordinates of the human body; The 3D key point generation unit uses the camera calibration data and the 2D key point coordinates of multiple perspectives to calculate the 3D coordinates of the key points of the human body through triangulation technology to generate the 3D key points on the user's body; The data filtering and optimization unit applies a filtering algorithm to optimize the calculated three-dimensional key point data, and generates real-time three-dimensional posture information reflecting the user's actions based on the optimized three-dimensional key point data.

[0042] The above-mentioned design system in the present invention can effectively implement the rigid body linkage design method based on the human body model, and the technical effects that can be achieved are as described in the above-mentioned embodiments, which will not be repeated here.

[0043] Similarly, the above-mentioned optimization schemes for the system can also respectively achieve the optimization effects corresponding to the method in Example 1, which will not be repeated here.

[0044] Although the present application has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the accompanying drawings are merely exemplary illustrations of the present application as defined herein, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application is intended to include these modifications and variations.

Claims

1. A rigid body linkage design method based on a human body model, characterized in that: include: Based on the 2D key point detection algorithm and triangulation technology, the 3D key points on the user's body are extracted from the camera calibration data to generate real-time 3D posture data reflecting human body movement; Using Blender's Python interface, draw rigid bodies according to the three-dimensional key points, each of the rigid bodies is defined by two marker points; Generate a corresponding coordinate axis for each three-dimensional marker point, and through the parent class inheritance mechanism, link the direction of the coordinate axis with the Euler angle of the marker point to obtain enhanced three-dimensional marker points; The enhanced three-dimensional marking points are written into the human body model file, and the rigid body and the human body model are linked in real time by binding with the skeletal joints.

2. The rigid body linkage design method based on the human body model according to claim 1, characterized in that: Based on the 2D key point detection algorithm and triangulation technology, the 3D key points on the user's body are extracted from the camera calibration data to generate real-time 3D posture data reflecting human body movement, including: Collect video data of the user during movement, and use the camera calibration file to obtain the internal and external parameters of each camera; Process multi-view videos based on a two-dimensional key point detection algorithm to extract the two-dimensional joint coordinates of the human body; Using the camera calibration data and the coordinates of the two-dimensional key points in multiple perspectives, the three-dimensional coordinates of the key points of the human body are calculated through triangulation technology to generate the three-dimensional key points on the user's body; A filtering algorithm is applied to the calculated three-dimensional key point data for optimization, and real-time three-dimensional posture information reflecting the user's actions is generated based on the optimized three-dimensional key point data.

3. The rigid body linkage design method based on human body model according to claim 1, characterized in that: The rigid body is represented by the geometric shape of a cone, the bottom of which is composed of a circular base surface defined by a first marking point, the first marking point is the bottom center of the rigid cone, and the top is formed by the convergence of the tip defined by a second marking point, and the second marking point is the top cone tip of the rigid cone.

4. The rigid body linkage design method based on human body model according to claim 1, characterized in that: Through the frame-by-frame update algorithm, the coordinates of the user's real-time 3D key points are input into the Blender script to dynamically generate a rigid body model for each frame.

5. The rigid body linkage design method based on human body model according to claim 4, characterized in that: The frame-by-frame update algorithm includes: Calculating the rotation Euler angle of each of the marking points according to the real-time three-dimensional posture data; Based on the rotation Euler angles, a corresponding rotation matrix is ​​calculated, and the direction of the coordinate axis is updated in real time according to the rotation matrix; The position and direction of the three-dimensional marker point are dynamically updated according to the three-dimensional coordinates and rotation matrix calculated for each frame.

6. The rigid body linkage design method based on human body model according to claim 1, characterized in that: Generate a corresponding coordinate axis for each three-dimensional marker point, and through the parent class inheritance mechanism, link the direction of the coordinate axis with the Euler angle of the marker point to obtain the enhanced three-dimensional marker point, including: Generate corresponding coordinate axes according to each three-dimensional marking point, wherein the coordinate axes include an X axis, a Y axis and a Z axis, wherein the X axis and the Y axis are parallel to the joint cross-sectional plane, and the Z axis is perpendicular to the joint cross-sectional plane; Through the parent class inheritance mechanism, the rotation Euler angle of the marker point is linked with the coordinate axis direction, and the coordinate axis direction is updated in real time with the marker point; The dynamically updated coordinate axis is bound to the marking point to form an enhanced three-dimensional marking point, wherein the enhanced marking point includes direction, rotation Euler angle and linkage information with the joint.

7. The rigid body linkage design method based on human body model according to claim 1, characterized in that: The enhanced three-dimensional marker points are written into the human body model file, and the rigid body and the human body model are linked in real time by binding with the skeletal joints, including: Create a human model file that records the three-dimensional position, rotation center and range of motion of bone joints, and establish a hierarchical structure of bones; Writing the enhanced data of the three-dimensional marker points into a human body model file and establishing a dynamic dependency relationship with the skeletal joints; Define the skeleton joint as the parent class object of the rigid body, and bind the bottom and top coordinates of the rigid body to the skeleton joint through the enhanced three-dimensional marker points; As the position and rotation state of the skeletal joints change in real time, the three-dimensional position and orientation of the enhanced markers and rigid bodies are dynamically updated.

8. The rigid body linkage design method based on human body model according to claim 1, characterized in that: The triangulation algorithm is least squares triangulation.

9. A rigid body linkage design system based on a human body model, characterized in that: The system comprises: The pose data generation module extracts the 3D key points on the user's body from the camera calibration data based on the 2D key point detection algorithm and triangulation technology, and generates real-time 3D pose data reflecting human body movement; A rigid body definition drawing module uses Blender's Python interface to draw rigid bodies according to the three-dimensional key points, each of which is defined by two marking points; The marker point enhancement module generates a corresponding coordinate axis for each three-dimensional marker point, and through the parent class inheritance mechanism, the direction of the coordinate axis and the Euler angle of the marker point are linked to obtain the enhanced three-dimensional marker points; The rigid body model linkage module writes the enhanced three-dimensional marking points into the human body model file, and performs real-time linkage between the rigid body and the human body model by binding with the bone joints.

10. The rigid body linkage design system based on human body model according to claim 9, characterized in that: The posture data generation module comprises: An internal and external parameter acquisition unit collects video data during the user's movement and uses a camera calibration file to obtain the internal and external parameters of each camera; A two-dimensional coordinate extraction unit processes multi-view videos based on a two-dimensional key point detection algorithm to extract the two-dimensional joint point coordinates of the human body; A three-dimensional key point generation unit, which uses the camera calibration data and the two-dimensional key point coordinates of multiple perspectives to calculate the three-dimensional coordinates of the human body key points through triangulation technology to generate the three-dimensional key points on the user's body; The data filtering and optimization unit applies a filtering algorithm to optimize the calculated three-dimensional key point data, and generates real-time three-dimensional posture information reflecting the user's actions based on the optimized three-dimensional key point data.

Images