Cell model automatic adjustment method and device, electronic equipment and storage medium

Abstract

This application relates to the fields of motion capture technology and computer human body model technology, and particularly to a method, device, electronic device, and storage medium for automatic adjustment of a surface model. The method includes: acquiring the posture data of a reference human body at the current moment; adjusting the current posture of the surface model of the reference human body based on the posture data to obtain the target posture of the surface model, and acquiring posture representation data of a model human body using an inertial motion capture system; determining the mapping relationship between the reference human body and the model human body, and adjusting the target posture of the surface model based on the mapping relationship and the posture representation data to obtain a dynamic surface model that changes over time. This solves the problems in related technologies where posture human body model modeling requires manual adjustment, has a low degree of automation, and where automatic adjustment of human body models based on optical motion capture is expensive, complex to operate, and has a narrow range of applications.

Description

Technical Field

[0001] This application relates to the fields of motion capture technology and computer human body model technology, and in particular to a method, device, electronic device and storage medium for automatic adjustment of a surface model. Background Technology

[0002] Monte Carlo simulation based on computer human models is the most important means of human dose assessment. Different application fields require human models with different characteristics. Since it takes a lot of time to build a completely new phantom, the reference phantom can be deformed and adjusted according to different needs.

[0003] In related technologies, a human body surface model can be used to define the contours of human tissues and organs, and to adjust the posture of the phantom based on an optical motion capture system. The human body surface model uses NURBS (Non-Uniform Rational B-Spline) or PM (Polygonal Mesh) to construct a closed geometry. Due to the mathematical smoothness and continuous deformability of NURBS or PM, the surface model has the advantages of both posture adjustment and high resolution. Optical motion capture equipment usually uses infrared emitters and infrared cameras in conjunction with wearable devices for motion capture, which has high accuracy.

[0004] However, optical motion capture methods in related technologies do not perform well in challenging lighting conditions; at the same time, their vision-based methods are affected by occlusion, and adding cameras to existing equipment makes the system bulky and expensive, with a narrow range of practical applications and poor adaptability to different scenarios, failing to meet the needs of some scenarios. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for automatic adjustment of surface model, in order to solve the problems in related technologies such as the need for manual adjustment of pose human body model modeling, low degree of automation, high cost, complex operation, and narrow scope of application of automatic adjustment of human body model based on optical motion capture.

[0006] The first aspect of this application provides a method for automatically adjusting a surface model, comprising the following steps: acquiring the posture data of a reference human body at the current moment; adjusting the current posture of the surface model of the reference human body based on the posture data to obtain the target posture of the surface model, and acquiring the posture representation data of a model human body using an inertial motion capture system; determining the mapping relationship between the reference human body and the model human body, and adjusting the target posture of the surface model based on the mapping relationship and the posture representation data to obtain a dynamic surface model that changes over time.

[0007] Optionally, the posture representation data includes the three-dimensional rotation vectors of the joints and the global position of the model's body; the step of acquiring the posture representation data of the body at the current moment using an inertial motion capture system includes: setting skeletal points as key points; and using the inertial motion capture system to capture the three-dimensional rotation vectors of the joints at the key points and the global position of the model's body.

[0008] Optionally, determining the mapping relationship between the reference human body and the model human body includes: identifying the reference point movement of the model human body in the posture representation data; calculating the movement of the human motion anatomy center based on the reference point movement, wherein the human motion anatomy center is located inside each joint of the human body; and determining the correspondence between the joint positions of the reference human body and the model human body, as well as the respective mapping ratios of the range of motion and amplitude of each joint, based on the movement of the human motion anatomy center.

[0009] Optionally, adjusting the target pose of the surface model based on the mapping relationship and the pose representation data includes: identifying the deformation type of the reference human body; if the deformation type is a first type, then reading the pose representation data using an interface file, and adjusting the target pose of the surface model based on the pose representation data and the mapping relationship; if the deformation type is a second type, then connecting to the motion capture device of the inertial motion capture system via a plug-in, inputting the pose representation data acquired by the motion capture device into the deformation software to obtain a dynamic model of the human body, and adjusting the target pose of the surface model based on the dynamic model and the mapping relationship.

[0010] Optionally, the posture data includes the posture of the whole skeleton, the posture of soft tissue, and the posture of internal organs; adjusting the current posture of the surface model of the reference human body based on the posture data includes: rotating each part of the skeleton around the joints based on the posture of the whole skeleton and the rotation matrix transformation; deforming the soft tissue based on the posture of the soft tissue and the volumetric Laplacian operator algorithm; and deforming the internal organs based on the posture of the internal organs and the approximate rigid body transformation algorithm.

[0011] A second aspect of this application provides an automatic adjustment device for a surface model, comprising: an acquisition module for acquiring posture data of a reference human body at a current moment; a capture module for adjusting the current posture of the surface model of the reference human body based on the posture data to obtain a target posture of the surface model, and acquiring posture representation data of a model human body using an inertial motion capture system; and a determination module for determining the mapping relationship between the reference human body and the model human body, and adjusting the target posture of the surface model based on the mapping relationship and the posture representation data to obtain a dynamic surface model that changes over time.

[0012] Optionally, the posture representation data includes the three-dimensional rotation vectors of the joints and the global position of the model's body; the capture module is further used to: set skeletal points as key points; and use the inertial motion capture system to capture the three-dimensional rotation vectors of the joints at the key points and the global position of the model's body.

[0013] Optionally, the determining module is further configured to: identify the reference point movement of the model's body in the posture representation data; calculate the movement of the human motion anatomy center based on the reference point movement, wherein the human motion anatomy center is located inside each joint of the human body; and determine the correspondence between the joint positions of the reference human body and the model's body, as well as the respective mapping ratios of the range of motion and amplitude of each joint, based on the movement of the human motion anatomy center.

[0014] Optionally, the determining module is further configured to: identify the deformation type of the reference human body; if the deformation type is a first type, then read the posture representation data using an interface file, and adjust the target posture of the surface model based on the posture representation data and the mapping relationship; if the deformation type is a second type, then connect to the motion capture device of the inertial motion capture system through a plug-in, input the posture representation data obtained by the motion capture device into the deformation software to obtain the dynamic body model of the model human body, and adjust the target posture of the surface model based on the dynamic body model and the mapping relationship.

[0015] Optionally, the posture data includes the posture of the whole skeleton, the posture of soft tissue, and the posture of internal organs; the capture module is further used to: rotate each part of the skeleton around the joints based on the posture of the whole skeleton and the rotation matrix transformation; deform the soft tissue based on the posture of the soft tissue and the volumetric Laplacian operator algorithm; and deform the internal organs based on the posture of the internal organs and the approximate rigid body transformation algorithm.

[0016] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the automatic adjustment method for surface model as described in the above embodiments.

[0017] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the automatic adjustment method for surface model as described in the above embodiments.

[0018] Therefore, this application has at least the following beneficial effects:

[0019] This application embodiment can acquire human model posture representation data based on inertial motion capture, adjust the posture of the facet model, and establish a dynamic temporal facet model. Therefore, this application embodiment can reproduce or simulate real posture changes by referring to the mapping relationship between the human body and the model's human body and posture representation data, and obtain a real-time dynamic facet model, realizing automatic adjustment of model posture changes without manual body model adjustment, thus improving the intelligence and convenience of the solution. At the same time, by using the inertial motion capture system and its application programming interface, the cost of dynamic capture is reduced, the applicability of the solution is increased, the operation is simple and convenient, and it meets the needs of practical use.

[0020] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0021] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0022] Figure 1 This is a flowchart of the automatic adjustment method for surface element models according to an embodiment of this application;

[0023] Figure 2 This is a flowchart illustrating an embodiment of the automatic adjustment method for surface model according to this application;

[0024] Figure 3 This is an example diagram of an automatic adjustment device for surface model according to an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0026] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0027] Monte Carlo simulation based on computer human models is the most important means of human dose assessment. Different application fields require human models with different features (height, weight, posture, organ size and shape, etc.), but building a completely new human model requires a lot of time.

[0028] To address this issue, related technologies employ deformation and adjustment techniques to deform and adjust reference phantoms based on varying needs. For instance, while manual correction of posture-based human models is possible, posture adjustment relies heavily on human experience and skill, leading to significant variations in results and low efficiency and automation. Another approach involves iterative scaling along tissue surface normals using an RPI reference model to alter body dimensions and individual organ volumes to match statistical volumes. Individualized phantoms can also be created by registering patient CT images with a reference anatomical phantom. However, due to the inherent structural mechanisms of human voxel models, these methods cannot easily adjust posture and are limited to dose evaluation in upright positions. The calculation results are static and cannot account for changes in human posture over time.

[0029] In related technologies, mathematical phantoms are composed of geometric entities, which have a simple structure, making posture changes easy and computationally fast. However, this sacrifices anatomical realism and may cause significant systematic errors in simulations. Human body element models use NURBS or PM to construct closed geometric bodies to define the contours of human tissues and organs. Due to the mathematical smoothness and continuous deformability of NURBS or PM, element models possess the advantages of both posture adjustment and high resolution. Therefore, research on changing phantom postures can focus on element-based phantoms.

[0030] In practical applications, using human models in different postures significantly impacts dose calculation results. For example, reports indicate that using a single posture of the human model for accident dose reconstruction resulted in organ dose underestimation of up to 78% and effective dose underestimation of 19%. Furthermore, studies calculated the external radiation dose conversion coefficients for five postures (walking, sitting, bending, kneeling, and squatting) under six irradiation conditions (AP, PA, LLAT, RLAT, ROT, and ISO). The results showed significant differences in dose to organs such as red bone marrow, lungs, stomach, colon, breast, and gonads under different postures, with the gonads showing the largest difference, exceeding two orders of magnitude in extreme cases, and effective dose differences reaching up to 40%. Therefore, adjusting the posture of the human model to reflect actual conditions can yield more accurate dose results.

[0031] Motion capture equipment can capture human posture in real time and reproduce complete temporal posture changes. Therefore, in scenarios such as accident dose reconstruction and work simulation, motion capture equipment can be used to reproduce or simulate real posture changes.

[0032] The phantom posture adjustment technology in related technologies can be carried out based on optical motion capture systems. For example, a human face model and motion capture system can be used to conduct a dose reconstruction study of irradiated personnel in a criticality accident at a nuclear fuel plant. Specifically, firstly, a related optical motion capture system consisting of 12 optical cameras is used to record the movements of the actors. The actors' movements are set based on inferences about the movements of irradiated personnel at the time of the accident in relevant literature. Then, these movements are reproduced by deforming the face model. Finally, based on the environmental and radiation source information reported in relevant literature, the effective dose and organ dose of the irradiated personnel are calculated. However, this method involves too many approximations, loses anatomical structure, and significantly alters the volume of soft tissue and bone ends, resulting in certain errors in the calculated structure. For example, in related technologies, optical motion capture technology can be used to develop a real-time dose calculation method based on Geant4. Among them, the facet model posture adjustment technology is based on the rigid deformation algorithm and the fixed volume mesh deformation algorithm in the field of computer vision. The model is divided into organless model and internal organ model for deformation. This method is mainly used for TM (Tetrahedral Mesh) models, and has low applicability to established facet volume models.

[0033] In related technologies, optical motion capture devices typically utilize infrared emitters and infrared cameras in conjunction with wearable devices for motion capture. For example, the VICON Motion Capture System consists of a set of infrared cameras, markers (wearable reflective balls), and control software. The reflective balls can reflect the infrared light emitted by the infrared cameras back to the cameras at the same wavelength, thereby determining the two-dimensional coordinates of the reflective balls from the perspective of each camera. By using the control software to reconstruct the camera positions, the three-dimensional coordinates of each reflective ball can be obtained, and the time series of the three-dimensional positions of the reference points can be recorded. Based on this data, a time series of motion capture skeleton postures can be established.

[0034] However, optical methods for motion capture often perform poorly in challenging lighting conditions, and all vision-based methods are susceptible to occlusion. While this can sometimes be addressed by deploying more and denser cameras, this makes the system bulkier and more expensive, and is often impractical in certain applications—for example, placing cameras in a fully furnished common room is very difficult. Therefore, the limitation of vision-based methods lies in their difficulty in applications involving large-scale walking, running, and other everyday activities confined to a fixed spatial volume. This requires carefully controlled moving cameras to record sufficient motion information, making implementation challenging. These drawbacks significantly impact many applications, thus limiting the usability of vision-based solutions.

[0035] Compared to vision-based systems, motion capture using wearable inertial sensors is environment-independent, unaffected by occlusion, requires no complex setup, and has no limitations on the range of motion. Related work has shown that reconstructing human motion using data from six IMUs is feasible; however, as an optimization-based method, it requires access to the entire sequence, resulting in a long processing time. Further research has utilized TransPose to achieve 90fps global position and body posture motion capture using six IMUs. Using IMUs to record motion inertia is more economical, convenient, and reliable. Therefore, motion capture based on inertial sensors has attracted widespread attention.

[0036] To address the issues of high manual labor costs, significant reliance on experience leading to large discrepancies in model quality, and deviations from the target pose during real-time dynamic phantom creation, as well as the high cost and complexity of optical motion capture, this application provides an automatic adjustment method for surface model. This method utilizes inertial motion capture to acquire human model pose representation data, which can then be used for surface model pose adjustment and dynamic temporal surface model modeling. Thus, dynamic human surface models that change over time can be established without manual phantom pose adjustment. Furthermore, compared to optical motion capture, this method is simpler to operate, cheaper, more portable, and not limited by spatial constraints, making it more widely applicable.

[0037] The following description, with reference to the accompanying drawings, outlines an automatic adjustment method, apparatus, electronic device, and storage medium for surface modeling according to embodiments of this application.

[0038] like Figure 1 As shown, the automatic adjustment method for the surface element model includes the following steps:

[0039] In step S101, the posture data of the reference human body at the current moment is obtained.

[0040] It is understood that the embodiments of this application can acquire the posture data of a reference human body at the current moment; wherein, the human posture involves the position and shape of the whole skeleton, soft tissues (muscles, skin, lymphatic tissue) and internal organs; the posture data includes the posture of the whole skeleton, the posture of soft tissues and the posture of internal organs; the embodiments of this application can use an inertial motion capture system to extract the posture data.

[0041] It should be noted that the measurement device of the inertial motion capture equipment based on inertial sensors can consist of multiple measurement nodes. Each measurement node has a nine-axis IMU. A nine-axis IMU can contain three components: an accelerometer for measuring acceleration, a gyroscope for measuring angular velocity, and a magnetometer for measuring direction. It can return the angular velocity, acceleration, and magnetometer measurement information during motion. The core of attitude solving lies in rotation. The gyroscope can measure the three-axis rotational angular rate of the node relative to the body coordinate system. Through this angular rate and other information, the angle and position information can be updated. Therefore, the embodiments of this application can obtain the current attitude under the condition of a known initial state.

[0042] In step S102, the current pose of the reference human body's surface model is adjusted based on the pose data to obtain the target pose of the surface model, and the pose representation data of the model's human body is obtained using an inertial motion capture system.

[0043] It is understood that the embodiments of this application can use the acquired posture data to adjust the surface model of the reference human body, adjust the current posture of the surface model to obtain the target posture, and at the same time use an inertial motion capture system to acquire the posture representation data of the human body; wherein, the posture representation data includes the three-dimensional rotation vectors of the joints and the global position of the model's human body; the process of obtaining the target posture and acquiring the posture representation data in the embodiments of this application can be specifically as follows:

[0044] (I) Obtaining the target pose of the surface element model

[0045] In this embodiment of the application, adjusting the current posture of the reference human body's surface model based on posture data includes: rotating each part of the skeleton around the joints based on the posture of the whole skeleton and the rotation matrix transformation; deforming the soft tissue based on the soft tissue posture and the volumetric graph Laplacian operator algorithm; and deforming the internal organs based on the internal organ posture and the approximate rigid body transformation algorithm.

[0046] It is understandable that bones, soft tissues, and organs have certain differences in physical and geometric characteristics. In this embodiment, the above three parts can be adjusted separately for the facet model. In principle, bones drive the deformation of soft tissues, and soft tissues drive the deformation of organs. Therefore, in this embodiment, the facet model can be deformed in the order of bones, soft tissues, and organs. At the same time, in order to reduce the complexity of deformation, this embodiment can assume that the deformation is carried out in a time sequence, that is, it is assumed that when a part is deformed, the shape of the remaining parts is not changed.

[0047] Specifically, (1) Deformation of the skeleton: The pose adjustment of the surface model in this application embodiment can first adjust the skeleton, wherein the rotation matrix transformation can be used to rotate each part of the skeleton around the joint;

[0048] (2) Deformation of soft tissue: In this embodiment, the VGL (volumetric graph Laplacian) algorithm can be used to deform the soft tissue around the joint; wherein, for the mesh to be deformed, a volume graph that fills the inside of the mesh and a volume graph that covers the outside of the mesh can be constructed to prevent the shrinkage of the volume inside the surface and the self-intersection of the surface. Then, the transformation of the control curve is explicitly propagated to the region of interest through a propagation method similar to Poisson deformation, and the coordinates of the deformed network are solved by linear variation.

[0049] (3) Deformation of organs: The embodiments of this application can use the ARAP (as-rigid-as-possible) algorithm to deform internal organs. The core of this algorithm is to minimize the ARAP energy (characterizing the local deviation of the differential mapping between the original shape and the target shape). By minimizing this energy function, the model can be deformed as rigidly as possible, and then spatial overlap and other issues can be repaired.

[0050] Therefore, by deforming the surface model in the order of bones, soft tissues, and organs as described above, the embodiments of this application can adjust the surface model to obtain the target pose.

[0051] (II) Acquisition of Attitude Representation Data

[0052] In this embodiment of the application, the inertial motion capture system is used to obtain the posture representation data of the human body at the current moment, including: setting skeletal points as key points; and using the inertial motion capture system to capture the three-dimensional rotation vectors of the joints at the key points and the global position of the model's human body.

[0053] It is understandable that skeletal deformation is the basis of posture adjustment. The deformation is essentially the rotation of each bone around the joint. Therefore, in this embodiment, skeletal points can be set as key points, and an inertial motion capture system can be used to capture the key points to obtain the three-dimensional rotation vectors of the joints at the key points and the global position of the model's body, so as to realize the rotation transformation of the bones in the digital model based on the rotation matrix. Among them, the inertial motion capture system needs to capture the three-dimensional rotation vectors (axis angles) of multiple joints.

[0054] For example, in this embodiment, the number of joints can be reduced to 15 main joints: shoulder joints (2), elbow joints (2), wrist-palm joints (2), hip joints (2), knee joints (2), ankle joints (2), and spinal joints (simplified to 3: cervical joint, lumbar joint, and sacral joint). The motion capture system needs to acquire at least the three-dimensional rotation vectors (axis-angles) of these 15 joints and the global position of the model. This embodiment can also use a method based on SMPL (Skinned Multi-Person...) The Linear (human parametric model) model's skeleton includes 24 joints (Pelvis, L_Hip, R_Hip, Spine1, L_Knee, R_Knee, Spine2, L_Ankle, R_Ankle, Spine3, L_Foot, R_Foot, Neck, L_Collar, R_Collar, Head, L_Shoulder, R_Shoulder, L_Elbow, R_Elbow, L_Wrist, R_Wrist, L_Hand, R_Hand), making deformation more realistic.

[0055] It should be noted that the posture representation data format may include BVH, MBX, C3D, etc. The relevant format can be named according to the motion capture system or 3D (three dimensions) modeling software used in the application. Therefore, the embodiments of this application also need to call the interface of the required platform to write cross-platform programs.

[0056] In step S103, the mapping relationship between the reference human body and the model human body is determined, and the target pose of the surface model is adjusted based on the mapping relationship and pose representation data to obtain a dynamic surface model that changes over time.

[0057] It is understandable that the automatic adjustment of the surface model based on inertial motion capture mainly involves posture adjustment, acquisition of key data by the motion capture system, and data interoperability between the two. This application embodiment can confirm the mapping relationship between the reference human body and the model human body, and then adjust the target posture of the surface model based on the confirmed mapping relationship and the acquired posture representation data, thereby obtaining a dynamic surface model that changes over time. The process of determining the mapping relationship and adjusting the target posture in this application embodiment can be specifically described as follows:

[0058] (a) Determining the mapping relationship between the reference human body and the model human body

[0059] In this embodiment of the application, determining the mapping relationship between the reference human body and the model human body includes: identifying the reference point movement of the model human body in the posture representation data; calculating the movement of the human body motion anatomy center based on the reference point movement, wherein the human body motion anatomy center is located inside each joint of the human body; determining the correspondence between the joint positions of the reference human body and the model human body, as well as the respective mapping ratios of the range of motion and amplitude of each joint, based on the movement of the human body motion anatomy center.

[0060] It is understandable that the model body shape used by the motion capture system to acquire posture representation data may be different from the reference human face model for planned deformation (the height, joint position and the proportion of different parts are different). Therefore, the embodiments of this application can calculate the movement of the human movement anatomy center based on knowledge of movement anatomy.

[0061] Specifically, in this embodiment, the motion of the human movement anatomy center can be calculated from the motion (displacement and rotation) of the reference point motion of the posture representation data of the motion capture system. The posture representation data is scaled as a whole and mapped one-to-one for the joint positions. The range of motion of each joint is limited and the amplitude of motion is mapped proportionally. Then, the position and direction of the limb vector in each time frame are inferred from the motion of the movement anatomy center, thereby obtaining the position of the limb.

[0062] (ii) Adjusting the pose of the target in the metamodel

[0063] In this embodiment, adjusting the target pose of the surface model based on the mapping relationship and pose representation data includes: identifying the deformation type of the reference human body; if the deformation type is the first type, then using the interface file to read the pose representation data, and adjusting the target pose of the surface model based on the pose representation data and the mapping relationship; if the deformation type is the second type, then by connecting to the motion capture device of the inertial motion capture system through a plug-in, inputting the pose representation data obtained by the motion capture device into the deformation software to obtain the dynamic body model of the model human body, and adjusting the target pose of the surface model based on the dynamic body model and the mapping relationship.

[0064] The first type is non-real-time deformation, and the second type is real-time deformation.

[0065] It is understandable that, such as Figure 2 As shown, the embodiments of this application can identify the deformation type of the reference human body. When the identified deformation type is non-real-time deformation, the motion capture system outputs an interface file, uses the interface file to read posture representation data, and adjusts the posture of the surface model according to the read posture representation data and mapping relationship. When the identified type is real-time deformation, the motion capture device is directly connected through a plug-in to obtain a real-time dynamic model that reflects the posture of the model's human body.

[0066] Therefore, the embodiments of this application can obtain a real-time dynamic temporal posture model based on the posture adjustment of the surface model and combined with the temporal action frame, thereby restoring the accident scene or work scene and realizing the reconstruction of the real dosage scene.

[0067] The automatic adjustment method for the surface model of this application is illustrated below through a specific embodiment. In the following embodiment, the posture adjustment can be based on Blender software. By writing a program, the interface file output by the motion capture system is read to adjust the posture of the surface model; for example, the global position and the three-dimensional rotation angle of the joints output by the motion capture device are directly read, and then the skeleton is translated and rotated through a script to adjust the deformation of the organs. Alternatively, the motion capture device and Blender can be directly connected through relevant plugins to obtain a real-time dynamic model reflecting the model's posture in the Blender environment. Blender software is an open-source software that can realize modeling, simulation, animation, and rendering. It has an open Python API, allowing users to control the software by writing Python scripts or directly using the Python command line.

[0068] Specifically, in this embodiment of the application, an inertial motion capture system can be used to obtain 24 joint three-dimensional rotation angles and the global position of the phantom, that is, a total of 75 = 24 × 3 + 3 data for each posture. Among them, the rotation angle is represented by the axis angle (rotation vector), the axis angle direction is the rotation axis direction, and the phantom length is the rotation angle (relative to the reference shape).

[0069] In this application embodiment, a script can be written directly using the Blender Python API to read the data output by the interface. The main methods of representing rotation in Blender are Euler rotation and quaternion. Quaternion rotation requires 4 parameters (w, x, y, z), which means: with e = (x, y, z) as the axis of rotation, the rotation angle is w, and the unit is radians.

[0070] In this embodiment, the axis angle can be converted into a quaternion angle by the above definition, the skeleton member function rotation_quaternion.rotate is used for rotation, the global position is set by location, and the rotation angle information and the global 3D position are inserted into the keyframe by keyframe_insert. Finally, the pose capture sequence can be displayed and the individual deformable model and the temporal deformable model file can be output in the animation photo table in Blender.

[0071] In summary, the automatic adjustment method for surface model proposed in this application can acquire human body model posture representation data based on inertial motion capture, adjust the posture of the surface model, and establish a dynamic temporal surface model. Therefore, this application embodiment can reproduce or simulate real posture changes by referring to the mapping relationship between the human body and the model's body, and the posture representation data, to obtain a real-time dynamic surface model, achieving automatic adjustment of model posture changes without manual body model adjustment, thus improving the intelligence and convenience of the solution. Simultaneously, by using the inertial motion capture system and its application programming interface, the cost of dynamic capture is reduced, the applicability of the solution is expanded, and the operation is simple and convenient, meeting the needs of practical use.

[0072] Next, the automatic adjustment device for the surface model proposed according to the embodiments of this application is described with reference to the accompanying drawings.

[0073] Figure 3 This is a block diagram of an automatic adjustment device for surface model according to an embodiment of this application.

[0074] like Figure 3 As shown, the automatic adjustment device 10 for the surface model includes: an acquisition module 100, a capture module 200, and a determination module 300.

[0075] The module 100 is used to acquire the posture data of the reference human body at the current moment; the capture module 200 is used to adjust the current posture of the surface model of the reference human body based on the posture data to obtain the target posture of the surface model, and to acquire the posture representation data of the model human body using an inertial motion capture system; the determination module 300 is used to determine the mapping relationship between the reference human body and the model human body, and to adjust the target posture of the surface model based on the mapping relationship and the posture representation data to obtain a dynamic surface model that changes over time.

[0076] In this embodiment of the application, the posture representation data includes the three-dimensional rotation vectors of the joints and the global position of the model's body; the capture module 200 is further used to: set skeletal points as key points; and use an inertial motion capture system to capture the three-dimensional rotation vectors of the joints at the key points and the global position of the model's body.

[0077] In this embodiment of the application, the determining module 300 is further used to: identify the reference point movement of the model's human body in the posture representation data; calculate the movement of the human body's kinetic anatomy center based on the reference point movement, wherein the human body's kinetic anatomy center is located inside each joint of the human body; determine the correspondence between the joint positions of the reference human body and the model's human body, as well as the respective mapping ratios of the range of motion and amplitude of each joint, based on the movement of the human body's kinetic anatomy center.

[0078] In this embodiment of the application, the determining module 300 is further used to: identify the deformation type of the reference human body; if the deformation type is the first type, then use the interface file to read the posture representation data, and adjust the target posture of the surface model based on the posture representation data and the mapping relationship; if the deformation type is the second type, then connect to the motion capture device of the inertial motion capture system through the plug-in, input the posture representation data obtained by the motion capture device into the deformation software to obtain the dynamic body model of the model human body, and adjust the target posture of the surface model based on the dynamic body model and the mapping relationship.

[0079] In this embodiment of the application, the posture data includes the posture of the whole skeleton, the posture of soft tissue, and the posture of internal organs; the capture module 200 is further used to: rotate each part of the skeleton around the joint based on the posture of the whole skeleton and the rotation matrix transformation; deform the soft tissue based on the posture of the soft tissue and the volume graph Laplacian operator algorithm; and deform the internal organs based on the posture of the internal organs and the approximate rigid body transformation algorithm.

[0080] It should be noted that the explanation of the aforementioned embodiment of the automatic adjustment method for facet models also applies to the automatic adjustment device for facet models in this embodiment, and will not be repeated here.

[0081] The automatic adjustment device for surface model proposed in this application can acquire human body model posture representation data based on inertial motion capture, adjust the posture of the surface model, and establish a dynamic temporal surface model. Therefore, this application embodiment can reproduce or simulate real posture changes by referring to the mapping relationship between the human body and the model's body, and the posture representation data, to obtain a real-time dynamic surface model, achieving automatic adjustment of model posture changes without manual body model adjustment, thus improving the intelligence and convenience of the solution. Simultaneously, by using the inertial motion capture system and its application programming interface, the cost of dynamic capture is reduced, the applicability of the solution is expanded, and the operation is simple and convenient, meeting the needs of practical use.

[0082] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0083] The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0084] When the processor 402 executes the program, it implements the automatic adjustment method for the surface model provided in the above embodiments.

[0085] Furthermore, electronic devices also include:

[0086] Communication interface 403 is used for communication between memory 401 and processor 402.

[0087] The memory 401 is used to store computer programs that can run on the processor 402.

[0088] The memory 401 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0089] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Etended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0090] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0091] Processor 402 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of this application.

[0092] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for automatically adjusting the surface model.

[0093] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0094] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0095] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0096] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0097] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0098] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for automatically adjusting a surface element model, characterized in that, Includes the following steps: Obtain the current pose data of a reference human body; Based on the posture data, the current posture of the reference human body's surface model is adjusted to obtain the target posture of the surface model, and the posture representation data of the model's human body is obtained using an inertial motion capture system. The mapping relationship between the reference human body and the model human body is determined, and the target pose of the facet model is adjusted based on the mapping relationship and the pose representation data to obtain a dynamic facet model that changes over time.

2. The automatic adjustment method for surface element models according to claim 1, characterized in that, The posture representation data includes the three-dimensional rotation vectors of the joints and the global position of the model's body; The process of acquiring the current posture representation data of the human body using an inertial motion capture system includes: Set the skeletal points as key points; The inertial motion capture system is used to capture the three-dimensional rotation vectors of the joints at the key points and the global position of the model's body.

3. The automatic adjustment method for surface element models according to claim 1, characterized in that, Determining the mapping relationship between the reference human body and the model human body includes: Identify the movement of reference points of the model's body in the posture representation data; The motion of the human motion anatomy center is calculated based on the motion of the reference point, wherein the human motion anatomy center is located inside each joint of the human body; The correspondence between the joint positions of the reference human body and the model human body is determined based on the movement of the human movement anatomy center, as well as the respective mapping ratios of the range of motion and amplitude of each joint.

4. The automatic adjustment method for surface element models according to any one of claims 1-3, characterized in that, Adjusting the target pose of the surface element model based on the mapping relationship and the pose representation data includes: Identify the deformation type of the reference human body; If the deformation type is the first type, then the posture representation data is read using the interface file, and the target posture of the surface element model is adjusted based on the posture representation data and the mapping relationship; If the deformation type is the second type, the motion capture device of the inertial motion capture system is connected through a plug-in, and the posture representation data acquired by the motion capture device is input into the deformation software to obtain the dynamic body model of the model's human body. The target posture of the surface model is adjusted based on the dynamic body model and the mapping relationship.

5. The automatic adjustment method for surface element models according to claim 1, characterized in that, The posture data includes the posture of the whole skeletal system, soft tissue posture, and internal organ posture; Adjusting the current pose of the facet model of the reference human body based on the pose data includes: Based on the whole-body skeletal posture and rotation matrix transformation, each part of the skeleton is rotated around the joint; The soft tissue is deformed based on the soft tissue pose and volume map Laplacian operator algorithm. The internal organs are deformed based on the internal organ posture and the approximate rigid body transformation algorithm.

6. An automatic adjustment device for a surface element model, characterized in that, include: The acquisition module is used to acquire the posture data of the reference human body at the current moment; The capture module is used to adjust the current posture of the reference human body's surface model based on the posture data, obtain the target posture of the surface model, and use an inertial motion capture system to acquire the posture representation data of the model's human body. The determination module is used to determine the mapping relationship between the reference human body and the model human body, and adjust the target pose of the facet model based on the mapping relationship and the pose representation data to obtain a dynamic facet model that changes over time.

7. The automatic adjustment device for surface element models according to claim 6, characterized in that, The posture representation data includes the three-dimensional rotation vectors of the joints and the global position of the model's body; The capture module is further used for: Set the skeletal points as key points; The inertial motion capture system is used to capture the three-dimensional rotation vectors of the joints at the key points and the global position of the model's body.

8. The automatic adjustment device for surface element models according to claim 6, characterized in that, The determining module is further used for: Identify the movement of reference points of the model's body in the posture representation data; The motion of the human motion anatomy center is calculated based on the motion of the reference point, wherein the human motion anatomy center is located inside each joint of the human body; The correspondence between the joint positions of the reference human body and the model human body is determined based on the movement of the human movement anatomy center, as well as the respective mapping ratios of the range of motion and amplitude of each joint.

9. The automatic adjustment device for surface element models according to any one of claims 6-8, characterized in that, The determining module is further used for: Identify the deformation type of the reference human body; If the deformation type is the first type, then the posture representation data is read using the interface file, and the target posture of the surface element model is adjusted based on the posture representation data and the mapping relationship; If the deformation type is the second type, the motion capture device of the inertial motion capture system is connected through a plug-in, and the posture representation data acquired by the motion capture device is input into the deformation software to obtain the dynamic body model of the model's human body. The target posture of the surface model is adjusted based on the dynamic body model and the mapping relationship.

10. The automatic adjustment device for surface element models according to claim 6, characterized in that, The posture data includes the posture of the whole skeletal system, soft tissue posture, and internal organ posture; The capture module is further used for: Based on the whole-body skeletal posture and rotation matrix transformation, each part of the skeleton is rotated around the joint; The soft tissue is deformed based on the soft tissue pose and volume map Laplacian operator algorithm. The internal organs are deformed based on the internal organ posture and the approximate rigid body transformation algorithm.

11. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the automatic adjustment method for a surface model as described in any one of claims 1-5.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the automatic adjustment method for the surface model as described in any one of claims 1-5.

Images