A Method for Constructing a Virtual-Real Training Device Based on Digital Twins

Abstract

This invention discloses a method for constructing a virtual-real training device based on digital twins, belonging to the field of digital twin technology. The method includes: building a digital twin system; constructing a digital 3D twin of a moving object and a twin model of a moving scene respectively; fusing the digital 3D twin of the moving object and the twin model of the moving scene to obtain a static training model, and importing it into the digital twin system; acquiring real-time motion data during the movement of the moving object, and transmitting the real-time motion data to the digital twin system; iteratively optimizing the static training model based on the real-time motion data to obtain a real-time training model, achieving synchronous virtual-real training. This invention solves the problem of the inability to digitally analyze the training process of moving objects, provides a digital basis for discovering problems during sports training, and helps moving objects effectively improve their training performance.

Description

Technical Field

[0001] This invention belongs to the field of digital twin technology, and particularly relates to a method for constructing a virtual-real training device based on digital twins. Background Technology

[0002] Wearable devices such as fitness trackers and smartwatches can record steps, monitor heart rate, and record exercise data through inertial sensors. However, due to incomplete data collection, many training and fitness plans are mechanically generated according to fixed programs, failing to be precise and personalized for each individual, nor can they reflect the individual's exercise status, physiological state, and exercise needs in real time. Sports injuries are common illnesses that occur during exercise, frequently seen in gymnasts, ball game players, long-distance runners, new recruits in the military, and various other sports enthusiasts. Tennis elbow, runner's knee, and various soft tissue injuries are increasingly affecting younger people and are often overlooked, significantly shortening athletes' athletic lifespans and causing various health problems for sports enthusiasts.

[0003] With the development of new-generation information technologies (such as artificial intelligence, cloud computing, industrial IoT, intelligent manufacturing, Industry 4.0, and smart cities), digital twin technology has emerged and become a new technology attracting widespread attention. As people's understanding of digital twins deepens, research on digital twins of human motion data has become a new research topic. The application of digital twin technology in the field of sports training will also become a new development trend. Digital twin technology integrates real-time motion data, historical data, fused derived data, service data, and simulation data to construct a virtual digital twin motion body. It fully utilizes physical models, sensors, and historical motion data, integrating multi-disciplinary, multi-physical quantity, multi-scale, and multi-probability simulation processes to map the real physical world in virtual space. This achieves virtual-real mapping, real-time data interaction between the virtual and real worlds, and allows for simulation analysis of physical objects, detecting and predicting their different motion states.

[0004] Currently, research on human motion digital twins is still in its early stages. Overall, research on the theory and methodology of human motion digital twins is still superficial, and human motion training digital twin systems have not yet been seen in the sports field.

[0005] With the rapid development of the Internet of Things and information sensing technology, textile sensors are evolving towards greater flexibility, miniaturization, intelligence, and multifunctionality. Flexible wearable devices are core devices for collecting various physiological information from the human body and have become a research hotspot in the textile and apparel industry, biomedicine, and national defense. Currently, research on flexible wearable devices by scholars both domestically and internationally is still in the experimental and exploratory stage, with limited theoretical and technical research.

[0006] Flexible electronics technology deposits organic, inorganic, or organic-inorganic composite materials onto flexible substrates to form electronic components and integrated systems, exemplified by circuits. This emerging interdisciplinary technology possesses characteristics such as lightness, flexibility, thinness, and transparency, allowing it to conform to other materials and greatly expanding the applicability of electronic devices. Utilizing flexible wearable technology, sensors can be integrated into T-shirts without metal components; these textile sensors can monitor multiple important vital signs.

[0007] The construction of virtual motion objects is the foundation for building a motion digital twin's motion lifecycle in a virtual environment. Currently, some professional modeling software such as Open Sim, Poser, 3DS MAX, and Anybody perform well in 3D environment construction and 3D human body modeling. Because digital twins involve multiple interdisciplinary fields, human motion modeling involves a large amount of data, and the motion effects are diversified due to different motion environments and individual differences, the complexity of the simulation is greatly increased. Summary of the Invention

[0008] In view of the above-mentioned shortcomings in the prior art, the present invention provides a method for constructing a virtual and real training device based on digital twins, which solves the problem that the training process of moving objects cannot be digitally analyzed.

[0009] To achieve the above-mentioned objectives, the technical solution adopted by this invention is: a method for constructing a virtual-real training device based on digital twins, comprising the following steps:

[0010] S1. Build a digital twin system;

[0011] S2. Construct digital 3D twins of moving objects and twin models of moving scenes respectively;

[0012] S3. Merge the digital 3D twin of the moving object and the twin model of the moving scene to obtain a static training model, and import it into the digital twin system;

[0013] S4. Acquire real-time motion data of the moving object during its movement, and transmit the real-time motion data to the digital twin system;

[0014] S5. Based on the real-time motion data, iteratively optimize the static training model to obtain the real-time training model, thereby achieving synchronous training between virtual and real systems.

[0015] The beneficial effects of this invention are as follows: By using digital twin technology to construct digital twin models of the moving object and the moving scene, real-time motion data of the moving object's movement process is collected and reflected in the digital twin model, so that the digital three-dimensional twin of the moving object forms a mapping of the moving object in the real space in the moving scene twin model. This allows for real-time monitoring and analysis of the data of the moving object's training process, realizing the digitization of the moving object's training process, providing a digital basis for the discovery of problems in the sports training process, and helping the moving object effectively improve its training performance.

[0016] Furthermore, the digital twin system in step S1 includes a physical layer, a virtual layer, a service layer, and a data layer;

[0017] The physical layer is used to acquire motion data of moving objects, landmark data of real-world motion scenes, and climate data; obtain real-time motion data based on the motion data, landmark data, and climate data; and transmit the real-time motion data to the data layer and the virtual layer respectively.

[0018] The virtual layer is used to iteratively optimize the static training model based on the real-time motion data and positioning information to obtain a simulation model and simulation data, and to construct a real-time training model; and to transmit the simulation data and simulation model to the data layer and service layer respectively.

[0019] The service layer is used to display the actual movement process of the moving object and the virtual movement process of the digital 3D twin based on the real-time motion data and simulation model, respectively, and to support users in simulating and training the digital 3D twin to obtain simulation data; it is used to acquire the positioning information of the moving object; it is used to monitor the vital signs of the moving object based on the real-time motion data and obtain monitoring results; it is used to determine whether to activate the alarm module based on the monitoring results; it is used to request help from rescue personnel when the vital signs of the moving object are abnormal, and to send the positioning information and monitoring results to the rescue personnel; it is used to obtain service data based on the monitoring results, positioning information and simulation data, and to transmit the service data to the data layer;

[0020] The data layer is used to determine the timeliness of real-time motion data. If the time exceeds a preset value, historical data is obtained. It stores real-time motion data, simulation data, service data, historical data, and fused derived data. The real-time motion data and historical data are transmitted to the service layer.

[0021] The beneficial effects of the above-mentioned further scheme are as follows: the data of the motion process of the moving object is collected by the physical layer and transmitted to the virtual layer, data layer and service layer of the digital twin system. Model simulation is performed in the virtual layer, data processing and analysis are performed in the data layer, and information interaction is performed in the service layer. This enables the digital twin system to respond in real time to the data changes of the moving object during the training process, ensuring the accuracy and real-time nature of the moving object's motion data.

[0022] Furthermore, the service layer includes a motion data monitoring service module, a positioning module, a motion posture display service module, an alarm module, and a real-time control service module;

[0023] The motion data monitoring service module is used to monitor the vital signs of the moving object based on real-time motion data, obtain monitoring results, and feed the monitoring results back to the real-time control service module and the alarm module respectively.

[0024] The positioning module is used to acquire the positioning information of the moving object and transmit the positioning information to the real-time control service module, the alarm module and the virtual layer respectively;

[0025] The motion posture display module is used to display the actual motion process of the moving object and the virtual motion process of the digital three-dimensional twin of the moving object based on the real-time motion data and simulation model. At the same time, it supports users to simulate and train the digital three-dimensional twin of the moving object based on historical data to obtain simulation data, and transmit the simulation data to the real-time control service module.

[0026] The real-time control service module is used to determine whether to activate the alarm module based on the monitoring results; to obtain the simulation model and real-time motion data from the data layer, and to transmit the real-time motion data to the motion data monitoring service module and the motion posture display module respectively, and to transmit the simulation model to the motion posture display module; to control the posture of the digital 3D twin during simulation training; and to obtain service data based on the monitoring results, positioning information and simulation data.

[0027] The alarm module is used to request help from rescuers when the vital signs of a moving object are abnormal, and to send the location information and monitoring results to the rescuers.

[0028] The beneficial effects of the above-mentioned further solutions are as follows: training experts can view the movement of the moving object in real time through the motion posture display module, and can use historical data to simulate and train the digital 3D twin of the moving object, formulate training plans, and the activation of the positioning system also provides coordinate positioning for the simulation training of the digital 3D twin of the moving object in the digital twin model of the motion scene. The setting of the alarm module enables the moving object to call for help in time when an accident occurs during training, providing safety protection for the moving object. The construction of the service layer provides a good interactive foundation for the entire virtual and real training device.

[0029] Furthermore, the data layer includes a data transmission module, a database module, and a data processing module;

[0030] The data transmission module is used to transmit the real-time motion data, simulation data, service data, and historical data and fused derived data generated during the operation of the data processing module to the database module for storage; transmit the real-time motion data, simulation data, service data, historical data, and fused derived data to the data processing module for processing respectively; and transmit both the real-time motion data and historical data to the service layer.

[0031] The database module is used to store real-time motion data, simulation data, service data, historical data, and fused derived data;

[0032] The data processing module is used to determine the timeliness of real-time motion data. If the time is greater than a preset value, historical data is obtained and transmitted to the database module through the data transmission module; otherwise, no processing is performed.

[0033] The beneficial effects of the above-mentioned further solutions are as follows: the data layer processes and transmits the data fed back from the physical layer, virtual layer, and service layer in a timely manner, providing data support for the virtual and real training device; the database module is responsible for storing the data generated during the training of the moving object; the storage and support of a large amount of data ensures the accuracy of the digital three-dimensional twin of the moving object and the digital twin model of the moving scene, enabling the virtual space to form a high-precision mapping to the real space.

[0034] Furthermore, the method for constructing a digital 3D twin of the moving object in step S2 includes the following steps:

[0035] A1. Capture human motion data over a specific time period;

[0036] A2. Preprocess the motion data to obtain preliminary data;

[0037] A3. Based on the preliminary data, obtain the shape parameter β and the posture parameter respectively, and use the shape parameter β and the posture parameter as inputs to the human SMPL model;

[0038] A4. The shape parameter β is compared with the base template of the human SMPL model. The mesh of the moving object's static pose is obtained by superimposing and blending the images; the expression for the superposition is as follows:

[0039]

[0040] D∈R 6890×3×10

[0041] β∈R 10

[0042]

[0043] V shape ∈R 6890×3

[0044] Where D is the offset of the principal component, and β is the magnitude of the offset of the principal component. The mesh with the base template, V shape R is the mesh of the silent pose of a moving object, R is the set of real numbers, and mesh is the human body network.

[0045] A5. Overlay the posture parameters with the mesh of the silent posture of the moving object, and blend the skin to obtain a digital 3D twin of the moving object.

[0046] A6. Determine whether the error between the digital 3D twin of the moving object and the moving object is less than a preset value. If yes, complete the construction of the digital 3D twin of the moving object. Otherwise, adjust the shape parameter β and the posture parameter, and return to step A4.

[0047] The beneficial effects of the above-mentioned further scheme are as follows: the human SMPL model is a learnable model, which can better fit the shape of the human body and the deformation under different postures through training, and obtain a high-precision digital three-dimensional twin of the moving object, thus preparing for the construction of a virtual and real training device.

[0048] Furthermore, the method for constructing the motion scene twin model in step S2 includes the following steps:

[0049] B1. Collect landmark data and climate data for real-world sports scenarios;

[0050] B2. Determine the coordinate system of the virtual motion scene, and build a twin model of the motion scene based on the landmark data and climate data;

[0051] B3. Determine whether the error between the motion scene twin model and the real motion scene is less than a preset value. If yes, complete the construction of the motion scene twin model. Otherwise, adjust the landmark data and climate data, and return to step B2.

[0052] The beneficial effects of the above-mentioned further solutions are: real-time acquisition of data on the motion scene of the moving object ensures the accuracy of the motion scene twin model, and prepares for the construction of the virtual and real training device.

[0053] Furthermore, the method for constructing the real-time training model in step S5 includes the following steps:

[0054] C1. Obtain the positioning information of the moving object;

[0055] C2. Back up the real-time motion data and positioning information to obtain comparison data;

[0056] C3. Based on the real-time motion data and positioning information, iteratively optimize the static training model to obtain simulation data;

[0057] C4. Compare the simulation data and the comparison data to obtain the error value between the two;

[0058] C5. Determine whether the error value is less than the set value. If yes, obtain the simulation model and update the static training model to the simulation model, then proceed to step C6. Otherwise, adjust the real-time motion data and return to step C3.

[0059] C6. Determine whether the training of the moving object has ended. If yes, complete the construction of the real-time training model; otherwise, return to step C3.

[0060] The beneficial effects of the above-mentioned further solutions are: to update the training status of the moving object in real time and reflect it on the digital 3D twin of the moving object and the digital twin model of the moving scene, to simulate the training status of the moving object, to form a real-time training model, and to prepare for training experts to analyze the training status of the moving object. Attached Figure Description

[0061] Figure 1 This is a flowchart of the method of the present invention.

[0062] Figure 2 This is a structural diagram of the digital twin system in this invention.

[0063] Figure 3 This is a flowchart of the method for constructing a digital three-dimensional twin of a moving object in this invention.

[0064] Figure 4 This is a flowchart illustrating the method for constructing a motion scene twin model in this invention.

[0065] Figure 5 This is a flowchart of the method for constructing the real-time training model in this invention. Detailed Implementation

[0066] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0067] like Figure 1 As shown, the present invention provides a method for constructing a virtual-real training device based on digital twins, comprising the following steps:

[0068] S1. Build a digital twin system;

[0069] S2. Construct digital 3D twins of moving objects and twin models of moving scenes respectively;

[0070] S3. Merge the digital 3D twin of the moving object and the twin model of the moving scene to obtain a static training model, and import it into the digital twin system;

[0071] In this embodiment, the digital twin of the moving object is fused with the twin model of the moving scene. The coordinate system of the moving scene twin model is established according to the coordinate system of the real space. Based on the moving object's posture, trajectory, initial and final positions, physiological data during the movement, etc., the initial and final positions, movement trajectory, movement posture, three-dimensional model, and real-time data of the body of the moving object's digital twin in the moving scene twin model are determined, so that the moving object's digital twin forms a mapping of the moving object in the real space in the moving scene twin model.

[0072] S4. Acquire real-time motion data of the moving object during its movement, and transmit the real-time motion data to the digital twin system;

[0073] S5. Based on the real-time motion data, iteratively optimize the static training model to obtain the real-time training model, thereby achieving synchronous training between virtual and real systems.

[0074] This embodiment provides a method for constructing a virtual and real training device based on digital twins, which solves the problem of not being able to efficiently identify influencing factors when sports injuries occur due to insufficient understanding of one's own physical fitness, training techniques, or unscientific training methods. At the same time, it digitizes, virtualizes, and makes transparent the sports training process, providing a digital basis for the discovery of training problems, which helps sports athletes improve their training performance more scientifically and efficiently, and effectively avoid sports injuries.

[0075] In this embodiment, data is the core driving force of the virtual-real training device. Unlike traditional models, it places greater emphasis on the mutual mapping and high consistency with the physical space. The data sources must include not only physical and virtual spaces, but also fused data combining the virtual and real worlds. The connection between physical entities and virtual spaces must be bidirectional and compatible. Bidirectionality is specifically manifested in bidirectional connection, bidirectional driving, and bidirectional interaction, while compatibility is manifested in cross-platform, cross-interface, and cross-protocol compatibility. Through effective processing and analysis of massive amounts of real-time and historical data, a digital 3D twin of the moving object is obtained. Based on the motion data of the digital 3D twin of the moving object in the twin model of the motion scene on the virtual and real training device, the training scheme is optimized. Combined with the actual situation of the moving object, the most suitable training plan is obtained, thereby helping the moving object to avoid sports injuries as much as possible, improve the training level, and minimize the risk of injury during the moving object. At the same time, the training process of the moving object becomes digital and visualized. When the moving object is injured during training, the degree of muscle-bone damage can be understood in time, providing scientific guidance for the next step of treatment and shortening the rehabilitation period. Through the application of technology, a large amount of mechanical and physiological data in the process of movement is accumulated, laying a data foundation for studying the body structure and changes in the human body during movement.

[0076] like Figure 2 As shown, the digital twin system in step S1 includes a physical layer, a virtual layer, a service layer, and a data layer;

[0077] The physical layer is used to acquire motion data of moving objects, landmark data of real-world motion scenes, and climate data; obtain real-time motion data based on the motion data, landmark data, and climate data; and transmit the real-time motion data to the data layer and the virtual layer respectively.

[0078] In this embodiment, the physical layer uses flexible wearable sensors to measure data such as the current exercise subject's breathing behavior, sweat composition, body temperature, runner's electrocardiogram (heart rate) and electroencephalogram (Central Nervous System), resting heart rate, safe heart rate threshold during training, CNS status, cardiopulmonary function status, heart rate variability (HRV), muscle function status (strength, speed, endurance, skill), fatigue index, energy metabolism index, and autonomic nervous system balance indicators, and stores the data in the data layer.

[0079] The virtual layer is used to iteratively optimize the static training model based on the real-time motion data and positioning information to obtain a simulation model and simulation data, and to construct a real-time training model; and to transmit the simulation data and simulation model to the data layer and service layer respectively.

[0080] The service layer is used to display the actual movement process of the moving object and the virtual movement process of the digital 3D twin based on the real-time motion data and simulation model, respectively, and to support users in simulating and training the digital 3D twin to obtain simulation data; it is used to acquire the positioning information of the moving object; it is used to monitor the vital signs of the moving object based on the real-time motion data and obtain monitoring results; it is used to determine whether to activate the alarm module based on the monitoring results; it is used to request help from rescue personnel when the vital signs of the moving object are abnormal, and to send the positioning information and monitoring results to the rescue personnel; it is used to obtain service data based on the monitoring results, positioning information and simulation data, and to transmit the service data to the data layer;

[0081] The data layer is used to determine the timeliness of real-time motion data. If the time exceeds a preset value, historical data is obtained. It stores real-time motion data, simulation data, service data, historical data, and fused derived data. The real-time motion data and historical data are transmitted to the service layer.

[0082] In this embodiment, the virtual-real training device based on digital twins is constructed using a digital twin system. The service layer captures video datasets with 3D human posture annotations to determine the motion posture, muscle-skeleton balance, stability, and differences during the movement process, thereby optimizing the digital twin model. When the motion posture of the object is found to be non-standard, the digital twin promptly sends stimulation signals to the object to help it improve its training movements.

[0083] The data layer selects appropriate transmission methods for collected real-time motion data, historical data, fused and derived data, service data, simulation data, and other data to avoid problems such as data loss, interference, tampering, and distortion; it performs corresponding preprocessing on the data to ensure data accuracy, connectivity, and timely communication, enabling data from multiple platforms and multi-user systems to communicate with each other; and it utilizes dynamic heterogeneous data fusion methods to ensure that data fusion can be achieved on the same motion platform.

[0084] The service layer includes a motion data monitoring service module, a positioning module, a motion posture display service module, an alarm module, and a real-time control service module;

[0085] The motion data monitoring service module is used to monitor the vital signs of the moving object based on real-time motion data, obtain monitoring results, and feed the monitoring results back to the real-time control service module and the alarm module respectively.

[0086] The positioning module is used to acquire the positioning information of the moving object and transmit the positioning information to the real-time control service module, the alarm module and the virtual layer respectively;

[0087] The motion posture display module is used to display the actual motion process of the moving object and the virtual motion process of the digital three-dimensional twin of the moving object based on the real-time motion data and simulation model. At the same time, it supports users to simulate and train the digital three-dimensional twin of the moving object based on historical data to obtain simulation data, and transmit the simulation data to the real-time control service module.

[0088] In this embodiment, the motion data on the virtual-real training device is processed and then tested through dynamics to see if it conforms to the motion state in actual physical space. If it does, simulation training is performed to obtain the most suitable training scheme for the subject, thereby formulating a corresponding training plan. The subject then undergoes training according to the training plan to achieve the expected training effect.

[0089] The real-time control service module is used to determine whether to activate the alarm module based on the monitoring results; to obtain the simulation model and real-time motion data from the data layer, and to transmit the real-time motion data to the motion data monitoring service module and the motion posture display module respectively, and to transmit the simulation model to the motion posture display module; to control the posture of the digital 3D twin during simulation training; and to obtain service data based on the monitoring results, positioning information and simulation data.

[0090] In this embodiment, the real-time control service module includes not only a control submodule for the digital twin of the moving object, but also a feedback control submodule for the moving object in the physical world. The control submodule for the digital twin mainly controls the movement of the digital 3D twin, such as running, jumping, walking, and moving along the trajectory of markers. The feedback control submodule guides the training of the moving object based on the movement data of the digital twin. When an unreasonable training plan or intensity is executed, and the digital twin exhibits disordered indicators, it promptly sends a signal to the physical entity to stop the erroneous training method.

[0091] The alarm module is used to request help from rescuers when the vital signs of a moving object are abnormal, and to send the location information and monitoring results to the rescuers.

[0092] In this embodiment, the system's service layer also includes a positioning module and an alarm module. When the moving object is exercising outdoors and is acting alone, if an emergency occurs, the virtual and real training device can promptly send a distress signal through the alarm module and quickly and accurately locate the object through the positioning module, allowing nearby search and rescue workers and medical rescue teams to arrive at the scene as soon as possible to handle the emergency, thereby minimizing the risk factor.

[0093] The data layer includes a data transmission module, a database module, and a data processing module;

[0094] The data transmission module is used to transmit the real-time motion data, simulation data, service data, and historical data and fused derived data generated during the operation of the data processing module to the database module for storage; transmit the real-time motion data, simulation data, service data, historical data, and fused derived data to the data processing module for processing respectively; and transmit both the real-time motion data and historical data to the service layer.

[0095] In this embodiment, the data transmission module selects an appropriate transmission method for the collected real-time driving data, historical data, fused derived data, service data, simulation data, and other data to avoid problems such as data loss, interference, tampering, and distortion; the data processing module performs corresponding preprocessing on the data to ensure the accuracy, connectivity, and timely communication of the data, enabling data from multiple platforms and multiple user systems to communicate with each other; and the dynamic heterogeneous data fusion method ensures that data fusion can be achieved on the same motion platform.

[0096] The database module is used to store real-time motion data, simulation data, service data, historical data, and fused derived data;

[0097] The data processing module is used to determine the timeliness of real-time motion data. If the time is greater than a preset value, historical data is obtained and transmitted to the database module through the data transmission module; otherwise, no processing is performed.

[0098] like Figure 3 As shown, the method for constructing a digital 3D twin of a moving object in step S2 includes the following steps:

[0099] A1. Capture human motion data over a specific time period;

[0100] A2. Preprocess the motion data to obtain preliminary data;

[0101] A3. Based on the preliminary data, obtain the shape parameter β and the posture parameter respectively, and use the shape parameter β and the posture parameter as inputs to the human SMPL model;

[0102] A4. The shape parameter β is compared with the base template of the human SMPL model. The mesh of the moving object's static pose is obtained by superimposing and blending the images; the expression for the superposition is as follows:

[0103]

[0104] D∈R 6890×3×10

[0105] β∈R 10

[0106]

[0107] V shape ∈R 6890×3

[0108] Where D is the offset of the principal component, and β is the magnitude of the offset of the principal component. The mesh with the base template, V shape R is the mesh of the silent pose of a moving object, R is the set of real numbers, and mesh is the human body network.

[0109] A5. Overlay the posture parameters with the mesh of the silent posture of the moving object, and blend the skin to obtain a digital 3D twin of the moving object.

[0110] A6. Determine whether the error between the digital 3D twin of the moving object and the moving object is less than a preset value. If yes, complete the construction of the digital 3D twin of the moving object. Otherwise, adjust the shape parameter β and the posture parameter, and return to step A4.

[0111] In this embodiment, motion data of the moving object is acquired through motion sensing technology. The motion data is preprocessed, and a model of the skeletal key points is obtained using visualization software. Then, a muscle model is built on a simulation platform to obtain a digital 3D twin of the moving object. Virtual motion simulation is performed on this twin to obtain its motion characteristics, which are then parameterized and digitized. The similarity between the moving object and its digital 3D twin is determined based on an error threshold. If the similarity is within the error range, the digital 3D twin is successfully constructed. If it is outside the error range, the parameters are adjusted and optimized to obtain an accurate digital 3D twin of the moving object.

[0112] In this embodiment, sensors (such as optical markers, 3D scanners, flexible wearable devices, etc.) are used to capture data on a certain movement of the human body over a period of time, and the collected data is used to construct a human SMPL model. The human SMPL model includes shape parameters β, a set of 10-dimensional values ​​describing a person's shape, where each dimension can be interpreted as an indicator of human shape, such as height, weight, etc.; and pose parameters, a set of 24×3-dimensional numbers describing the human body's posture at a given moment. The 24 represents 24 predefined human body joints, and the 3 does not refer to the (x, y, z) spatial coordinates as defined in recognition problems, but rather to the rotation angle of that node relative to its parent node. The training process of the human SMPL model includes: Shape Blend Shapes. In this stage, the base template T (or statistical mean template) serves as the basic pose of the entire human body. This base template T is obtained statistically, using N = 6890 vertices to represent the entire mesh. Each vertex has three spatial coordinates (x, y, z), which are different from the joints of the skeleton. Subsequently, parameters are used to describe the desired human pose and the offset of this base pose. These are then superimposed to form the final desired human pose. This process is linear. The base template T set by the human SMPL model is the mean shape obtained by statistically analyzing a large number of real human meshes. By linearly combining the principal shape components (or vertex deviations) and superimposing them on the base template T, a mesh with a silent pose is formed. Here, the principal shape components refer to the main changing components of the mesh obtained statistically from the dataset. Specifically, each principal component is a 6890×3 matrix, where a certain (x, y, z) represents the value relative to the corresponding base template. The offset of the endpoints can be represented by the following formula:

[0113]

[0114] Where D∈R 6890×3×10 For the offset of 10 principal components, β∈R 10 The magnitude of the offset of the 10 principal components. Based on template mesh, V shape ∈R 6890×3 Let R be the mesh after hybridization, R be the set of real numbers, and mesh be the human body network.

[0115] In this embodiment, 23×3 parameters are needed to represent the relative rotation of a non-root node relative to its parent node. To represent the global rotation (also known as orientation) and spatial displacement of the entire human body, such as walking or running, the rotation and displacement of the root node also need to be defined. Similarly, 3 parameters are needed to express the rotation in axis-angle form, and 3 parameters are needed to express the spatial displacement. Because different human body shapes vary greatly, after the hybridization process, it is still necessary to estimate the skeletal points that conform to the formed mesh so that these skeletal points can be rotated to form the final desired posture. After estimating the skeletal point positions, the control points, i.e., skeletal points, are obtained for manipulating the entire human digital model. When rotating the skeletal points, the human body in a static posture can be posed into the desired posture, just like swinging a ball-jointed doll. The endpoints of the human body mesh also change along with the surrounding joints, forming the final human digital model. Therefore, skinning is actually the process of making the human skeleton in a static posture "move" and covering it with "skin," thereby realizing three-dimensional character modeling.

[0116] In this embodiment, muscle physiological signal data collected by a flexible wearable device is used to correct the digital 3D twin of the moving object. Combined with historical data and relevant experimental data from the movement process, the digital 3D twin of the moving object is continuously iterated and optimized, providing a foundation for the next step of the work. Ultimately, a three-dimensional integrated digital model of the bones, muscles, and skin during movement is formed. After the three-dimensional digital model of the human body is completed, a complete digital twin of the moving object is formed.

[0117] In this embodiment, the digital 3D twin of the moving object is the foundation for establishing the digital twin system, and the ultimate goal of all designs is to serve the moving object. The service layer includes a real-time control service module, a motion posture display service module, a positioning module, an alarm module, and a motion data monitoring service module. The real-time control service module performs real-time motion control on the digital twin moving object, such as running, jumping, and directional movement. The motion posture display service module displays the motion posture of the moving object and its digital 3D twin. The motion data monitoring service module collects motion data in real time. The service layer transmits service data to the data layer, which contains real-time motion data, historical data, fused and derived data, service data, and simulation data of the digital 3D twin of the moving object. The physical layer and the virtual layer perform real-time data interaction and mapping. A physical-based method and a data-driven motion synthesis method are used to obtain the digital 3D twin of the moving object. Dynamic modeling is performed based on the motion data to construct the motion model of the moving object. Dynamic numerical analysis of the human body's movement process is conducted. Using the interactive data obtained from the physical layer and the virtual layer, combined with computer rendering technology, continuous optimization is performed in the virtual layer to ultimately obtain a real-time training model of the moving object.

[0118] like Figure 4 As shown, the method for constructing the motion scene twin model in step S2 includes the following steps:

[0119] B1. Collect landmark data and climate data for real-world sports scenarios;

[0120] B2. Determine the coordinate system of the virtual motion scene, and build a twin model of the motion scene based on the landmark data and climate data;

[0121] B3. Determine whether the error between the motion scene twin model and the real motion scene is less than a preset value. If yes, complete the construction of the motion scene twin model. Otherwise, adjust the landmark data and climate data, and return to step B2.

[0122] In this embodiment, the construction of the digital twin model of the sports scene refers to building a virtual sports scene based on data collected by physical sensors, such as temperature and humidity, air quality, air pressure, light intensity, wind speed, ground type, and hardness / softness, as well as the geographical location and architectural landmarks. First, the coordinate system of the virtual space is determined; second, the framework of buildings and other landmarks is established; the virtual sports environment is adjusted based on the data collected by the sensors; and finally, it is rendered to form the sports scene twin model. The sports scene twin model is mainly a static model. Combined with the BeiDou positioning system, it can display the altitude and latitude of the sports scene, air oxygen content, temperature and humidity, whether it is an inland or coastal area, an urban or suburban area, indoor or outdoor, and the ground type (cement, gravel, dirt, or synthetic surface). Based on the relevant data collected by the sensors, virtual reality (VR) technology is used to construct a digital scene on relevant software platforms (Open Sim, Poser, 3DS MAX, Anybody), which is then rendered and the model parameters are continuously optimized to conform to the real sports scene.

[0123] like Figure 5 As shown, the method for constructing the real-time training model in step S5 includes the following steps:

[0124] C1. Obtain the positioning information of the moving object;

[0125] C2. Back up the real-time motion data and positioning information to obtain comparison data;

[0126] C3. Based on the real-time motion data and positioning information, iteratively optimize the static training model to obtain simulation data;

[0127] C4. Compare the simulation data and the comparison data to obtain the error value between the two;

[0128] C5. Determine whether the error value is less than the set value. If yes, obtain the simulation model and update the static training model to the simulation model, then proceed to step C6. Otherwise, adjust the real-time motion data and return to step C3.

[0129] C6. Determine whether the training of the moving object has ended. If yes, complete the construction of the real-time training model; otherwise, return to step C3.

[0130] In this embodiment, the method for constructing the real-time training model refers to generating a three-dimensional motion process database of a digital three-dimensional twin based on data such as motion monitoring data of virtual-real moving objects, motion posture data of virtual-real moving objects, real-time control data of virtual objects, and motion capture data throughout the motion process, using motion editing and synthesis technology.

Claims

1. A method for constructing a virtual-real training device based on digital twins, characterized in that, Includes the following steps: S1. Build a digital twin system; the digital twin system in step S1 includes a physical layer, a virtual layer, a service layer, and a data layer; The physical layer is used to acquire motion data of moving objects, landmark data of real-world motion scenes, and climate data; obtain real-time motion data based on the motion data, landmark data, and climate data; and transmit the real-time motion data to the data layer and the virtual layer respectively. The virtual layer is used to iteratively optimize the static training model based on the real-time motion data and positioning information to obtain the simulation model and simulation data, and to construct the real-time training model. The simulation data and simulation model are then transmitted to the data layer and service layer, respectively. The service layer is used to display the real-world motion process of the moving object and the virtual motion process of the digital 3D twin based on the real-time motion data and simulation model, and to support users in simulating and training the digital 3D twin to obtain simulation data; it is used to acquire the positioning information of the moving object; it is used to monitor the vital signs of the moving object based on the real-time motion data to obtain monitoring results; and it is used to determine whether an alarm module needs to be activated based on the monitoring results. This is used to request help from rescuers when a moving object exhibits abnormal vital signs, and to send the location information and monitoring results to the rescuers; service data is obtained based on the monitoring results, location information, and simulation data, and the service data is transmitted to the data layer; The data layer is used to determine the timeliness of real-time motion data. If the time exceeds a preset value, historical data is obtained. It stores real-time motion data, simulation data, service data, historical data, and fused derived data. The real-time motion data and historical data are then transmitted to the service layer. S2. Construct a digital 3D twin of the moving object and a twin model of the moving scene respectively; the method for constructing the digital 3D twin of the moving object in step S2 includes the following steps: A1. Capture human motion data over a specific time period; A2. Preprocess the motion data to obtain preliminary data; A3. Based on the preliminary data, the shape parameters are obtained respectively. and attitude parameters, and shape parameters The pose parameters are used as inputs to the human SMPL model; A4. The shape parameters Base template of human SMPL model The mesh of the moving object's static pose is obtained by superimposing and blending the images; the expression for the superposition is as follows: in, The shift of the principal component, The magnitude of the principal component offset. Mesh based on the template Mesh for the silent pose of moving objects R Let be the set of real numbers, and mesh be the human body network; A5. Overlay the posture parameters with the mesh of the silent posture of the moving object, and blend the skin to obtain a digital 3D twin of the moving object. A6. Determine whether the error between the digital 3D twin of the moving object and the moving object is less than a preset value. If yes, complete the construction of the digital 3D twin of the moving object; otherwise, adjust the shape parameters. And attitude parameters, and return to step A4; The method for constructing the motion scene twin model in step S2 includes the following steps: B1. Collect landmark data and climate data for real-world sports scenarios; B2. Determine the coordinate system of the virtual motion scene, and build a twin model of the motion scene based on the landmark data and climate data; B3. Determine whether the error between the motion scene twin model and the real motion scene is less than a preset value. If yes, complete the construction of the motion scene twin model; otherwise, adjust the landmark data and climate data, and return to step B2. S3. Merge the digital 3D twin of the moving object and the twin model of the moving scene to obtain a static training model, and import it into the digital twin system; S4. Acquire real-time motion data of the moving object during its movement, and transmit the real-time motion data to the digital twin system; S5. Based on the real-time motion data, iteratively optimize the static training model to obtain a real-time training model, achieving simultaneous virtual and real training; the method for constructing the real-time training model in step S5 includes the following steps: C1. Obtain the positioning information of the moving object; C2. Back up the real-time motion data and positioning information to obtain comparison data; C3. Based on the real-time motion data and positioning information, iteratively optimize the static training model to obtain simulation data; C4. Compare the simulation data and the comparison data to obtain the error value between the two; C5. Determine whether the error value is less than the set value. If yes, obtain the simulation model and update the static training model to the simulation model, then proceed to step C6. Otherwise, adjust the real-time motion data and return to step C3. C6. Determine whether the training of the moving object has ended. If yes, complete the construction of the real-time training model; otherwise, return to step C3.

2. The method for constructing a virtual-real training device based on digital twins according to claim 1, characterized in that, The service layer includes a motion data monitoring service module, a positioning module, a motion posture display service module, an alarm module, and a real-time control service module; The motion data monitoring service module is used to monitor the vital signs of the moving object based on real-time motion data, obtain monitoring results, and feed the monitoring results back to the real-time control service module and the alarm module respectively. The positioning module is used to acquire the positioning information of the moving object and transmit the positioning information to the real-time control service module, the alarm module and the virtual layer respectively; The motion posture display service module is used to display the actual motion process of the moving object and the virtual motion process of the digital three-dimensional twin of the moving object based on the real-time motion data and simulation model. At the same time, it supports users to simulate and train the digital three-dimensional twin of the moving object based on historical data to obtain simulation data, and transmit the simulation data to the real-time control service module. The real-time control service module is used to determine whether the alarm module needs to be activated based on the monitoring results. The simulation model and real-time motion data are acquired from the data layer, and the real-time motion data is transmitted to the motion data monitoring service module and the motion posture display service module, respectively. The simulation model is transmitted to the motion posture display service module. The posture of the digital 3D twin is controlled during the simulation training of the digital 3D twin. Service data is obtained based on the monitoring results, positioning information and simulation data. The alarm module is used to request help from rescuers when the vital signs of a moving object are abnormal, and to send the location information and monitoring results to the rescuers.

3. The method for constructing a virtual-real training device based on digital twins according to claim 1, characterized in that, The data layer includes a data transmission module, a database module, and a data processing module; The data transmission module is used to transmit the real-time motion data, simulation data, service data, and historical data and fusion-derived data generated during the operation of the data processing module to the database module for storage; and to transmit the real-time motion data, simulation data, service data, historical data, and fusion-derived data to the data processing module for processing respectively. Both the real-time motion data and historical data are transmitted to the service layer; The database module is used to store real-time motion data, simulation data, service data, historical data, and fused derived data; The data processing module is used to determine the timeliness of real-time motion data. If the time is greater than a preset value, historical data is obtained and transmitted to the database module through the data transmission module; otherwise, no processing is performed.

Images