A meta-universe digital twin electronic dart system and method based on a non-wearable motion capture technology

Abstract

This invention discloses a metaverse digital twin electronic darts system and method based on wearable motion capture technology, belonging to the field of motion-sensing interaction technology. The system includes the following modules: a wearable motion capture module; a key point data processing and transmission module; a physics simulation engine module; a digital twin scene module; a gameplay rules module; a character growth module; a UGC creation module; and a virtual-real fusion module. This metaverse digital twin electronic darts system and method, based on wearable motion capture technology, requires no wearing of any devices, allowing user movements to be natural while lowering the barrier to user participation. It achieves high-precision motion capture, and significantly improves inference speed, reduces model size, and lowers memory usage through NPU acceleration optimization. Combined with asynchronous processing and dynamic frame interpolation technology, it ensures the smoothness and real-time performance of motion data.

Description

Technical Field

[0001] This invention relates to the field of motion-sensing interaction technology, specifically to a metaverse digital twin electronic dart system and method based on wearable motion capture technology. Background Technology

[0002] With the rise of the metaverse concept, digital sports activities have gradually gained attention. Among them, traditional electronic darts games usually require the use of wearable devices such as VR headsets and motion-sensing controllers. However, motion-sensing devices are inconvenient to wear, the user experience is unnatural, and traditional motion-sensing devices are expensive and have low adoption rates. In addition, traditional motion capture systems have low frame rates and high latency, which affects real-time performance. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a metaverse digital twin electronic dart system and method based on wearable motion capture technology, which solves the problems mentioned in the background.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a metaverse digital twin electronic dart system based on wearable motion capture technology, the system comprising the following modules:

[0005] Wearable motion capture module: Uses a TOF depth camera to capture user movements, identify and track user throwing movements and convert them into three-dimensional motion parameters;

[0006] Key point data processing and transmission module: adopts asynchronous processing mechanism and dynamic data frame interpolation technology to increase the actual detection frame rate of 22fps to a transmission frame rate of 60-90fps;

[0007] Physics simulation engine module: Based on UE5's physics engine, it calculates the dart's trajectory and impact position based on the captured throwing action parameters, including but not limited to angle of attack, initial velocity, and rotational angular velocity;

[0008] Digital Twin Scene Module: Using panoramic holographic oblique photography technology, a digital twin model of a real darts venue is constructed, including physical parameters and environmental characteristics, while providing fantastic creative scenes such as interstellar shooting ranges, underwater caves, and time rifts;

[0009] Gameplay Rules Module: Implements the rule logic for classic game formats such as 501 / 301 straight-down matches and innovative gameplay such as timed challenges and team target range breakthroughs;

[0010] Character growth module: Implements the user character's skill tree, equipment system, and growth path;

[0011] Social interaction module: Enables multi-player collaboration, team competition, spectating, and interactive functions;

[0012] UGC Creation Module: Provides scene creation tools, allowing users to design personalized scenes and custom rules;

[0013] Virtual-Real Integration Module: Supports data exchange between offline physical venues and virtual systems.

[0014] A method applied to the aforementioned metaverse digital twin electronic dart system based on wearable motion capture technology, the method comprising the following steps:

[0015] S1. System Initialization and User Calibration:

[0016] The camera is turned on to collect the user's three-dimensional spatial data in real time. At the same time, the client and the server establish a connection through the network to initialize the UE5 engine scene. The camera calculates the user's height, arm length, standing position and distance to the dartboard by detecting human key points.

[0017] S2. Wearless motion capture and data processing:

[0018] The TOF depth camera continuously captures user actions, generates point cloud data, and inputs it into the MediaPipe framework after filtering and noise reduction preprocessing.

[0019] The MediaPipe framework identifies 21 key points on the hand and 33 key points on the body, and calculates the coordinates of each joint.

[0020] Accelerate model inference using neural network processors;

[0021] The data acquisition and transmission processes are further separated to ensure that the acquisition thread does not block the transmission. Then, virtual frames are generated by predictive interpolation algorithms such as linear interpolation or Kalman filtering to increase the transmission frame rate to 60-90fps.

[0022] The transmission frequency is dynamically adjusted according to the network status, and priority is given to transmitting key action data such as the peak acceleration at the moment of throwing.

[0023] Calculation of throwing parameters based on motion data:

[0024] Direction angle: The angle between the throwing direction and the horizontal plane is calculated by the movement trajectory of key points of the hand, such as the vector from the wrist to the fingertips;

[0025] Initial velocity: Analyze the hand acceleration curve and calculate the linear velocity at the moment of release by combining the arm length;

[0026] Rotational angular velocity: Detects the wrist rotation trajectory and calculates the dart's spin speed using the quaternion rate of change;

[0027] S3. Physical Simulation and Trajectory Calculation:

[0028] User action parameters include direction angle α, initial velocity V0, and rotational angular velocity ω;

[0029] Environmental parameters include gravity g, air density p, and wind speed v;

[0030] Trajectory equation calculation:

[0031]

[0032] The first term is the basic trajectory of the parabola, the second term is the influence of gravity, and the third term is the Magnus effect offset caused by rotation.

[0033] And air resistance and Magnus force:

[0034] Among them, C d =0.42 is the air drag coefficient, S m Magnus coefficient;

[0035] S4, Digital Twin Scene Rendering and Interaction:

[0036] Real-world venue twin: A venue model is constructed using panoramic holographic oblique photography and BIM technology, giving it physical properties such as a floor friction coefficient μ=0.65 and acoustic reflection;

[0037] Creative scene rendering: such as the planetary gravitational field under zero gravity, i.e., the interstellar target range, and the time rift with random ±50% change in flight speed, generating high-precision textures and lighting effects through GAN network;

[0038] Based on user-selected settings, the system renders the dart's trajectory and hit effects in real time, while dynamically playing ambient sounds and hit sound effects according to the scene.

[0039] Furthermore, in step S1, the camera is a TOF depth camera with a resolution of 640×480, a frame rate of 30Hz, a depth accuracy of ±1cm, and a field of view of 75°×65°.

[0040] The client is responsible for image acquisition and rendering, while the server is responsible for data processing and storage.

[0041] Human keypoint detection uses the MediaPipe motion capture recognition framework to identify 21 hand and 33 body keypoints.

[0042] Furthermore, in step S1, the camera detects the current light intensity and automatically adjusts the camera exposure parameters; if the ambient light is insufficient, a supplementary lighting prompt is triggered, and the client can connect to the smart home system via WIFI to adjust the lighting parameters of the smart home lighting fixtures based on the adjustable parameters in the smart home lighting and in conjunction with the camera's optimal operating parameters.

[0043] Furthermore, in step S2, the specific process of accelerating model inference using a neural network processor is as follows:

[0044] The MediaPipe framework was used to train a 21-keypoint detection model for the hand and a 33-keypoint detection model for the body. Keypoint recognition was achieved based on convolutional neural networks or lightweight models such as MobileNet.

[0045] By using quantization techniques supported by the NPU, such as 8-bit fixed-point quantization, floating-point model parameters can be converted into integers, reducing model size by 40% and memory usage by 35%.

[0046] The principle is to reduce computational complexity by reducing the number of bits stored for each parameter, while preserving the accuracy of key features by calibrating the data;

[0047] The quantized model is deployed to the neural network processor on the device, such as the NPU chip of a mobile phone or edge device, and its dedicated hardware units, such as the tensor processing unit (TPU), are used to accelerate matrix operations.

[0048] The NPU accelerates operations such as convolution and pooling through multi-core parallel computing, increasing the overall inference speed from 8fps to 22fps.

[0049] Furthermore, in step S3, the coordinates of the intersection point between the dart and the target are solved according to the trajectory equation to determine whether the target hits the double zone, triple zone, or bullet eye.

[0050] If the dart hits a hard material area, a rebound effect is triggered. The rebound speed is calculated based on the elastic parameter E = 2000 MPa. If the dart hits a soft material target, it inserts and comes to a stop, and the scoreboard is updated.

[0051] Furthermore, the method also includes the following steps:

[0052] S5, Multi-user Interaction and Social Features:

[0053] Room creation: Users can create rooms through the social module, which can support 2-8 people to join, and select the game mode;

[0054] Data synchronization: The server synchronizes the action data of each client via the WebRTC protocol with a latency of ≤200ms, ensuring that the throwing trajectory and score of multiple users are consistent in real time;

[0055] Team Mode: Teams of 3 players are divided into three roles: sniper (high-precision throwing), armor penetration (high-power penetration), and healer (supporting repair of the target). They cooperate in combat through an energy-sharing mechanism.

[0056] Spectator system: Supports switching between third-person perspective and bullseye perspective. Spectators can send emoji comments. The system automatically captures exciting moments such as a grand slam throw and generates replay videos.

[0057] Furthermore, the method also includes the following steps:

[0058] S6. User-Generated Content and System Extensions:

[0059] Users upload photos of real-world venues, and AI algorithms automatically generate 3D twin scenes; users can customize physical parameters such as gravity, wind speed, and obstacle layouts such as moving targets and shield walls.

[0060] Users can design custom game formats such as time-limited + random environmental changes and scoring rules such as triggering a chain explosion of points when a specific area is hit through the rules editor.

[0061] Users can package and upload their own scenarios and rules to the community, and other users can download and experience them directly.

[0062] Creators will receive virtual rewards such as exclusive equipment, skins, and experience boosts based on their likes and download rankings, and high-quality content may be selected for the official recommendation pool.

[0063] Furthermore, the method also includes the following steps:

[0064] S7, Virtual-Real Integration and Commercial Applications:

[0065] If a pressure-sensitive floor is used, it is placed within the camera's capture range, and the user stands on the pressure-sensitive floor to operate. The pressure-sensitive floor works in conjunction with the TOF camera to detect whether the user's standing posture is compliant; when used with physical darts equipped with sensors, the virtual trajectory is calibrated and synchronized with the virtual scene, such as the physical throwing angle.

[0066] Offline store competition results are synchronized to the online leaderboard, and users can participate in darts activities and unlock prizes through LBS technology;

[0067] The system demonstrates the principles of parabolic trajectories through virtual scenarios, such as fixing the initial velocity and adjusting the angle to observe the trajectory changes, for use in physics teaching.

[0068] Design low-intensity training modes for the elderly or patients with upper limb injuries, analyze joint range of motion through motion capture, generate rehabilitation reports, and thus enable the system to be applied to rehabilitation training.

[0069] Furthermore, the method also includes the following steps:

[0070] S8. System Closed Loop and Data Management:

[0071] The user roles are set up with a growth system and a data platform. The growth system gains experience through matches, and the experience can be used to unlock the user role's skills. The data platform stores data such as user action history, win rate, and preferred scenarios, which are used for AI analysis to optimize matching algorithms, such as recommending similar players to play against.

[0072] If the system detects a camera position shift, it triggers an automatic calibration process, prompting the user to stand back in the designated area. The server records data transmission latency, frame rate fluctuations, and other indicators in real time, automatically issuing alarms and switching to a backup node when an anomaly occurs.

[0073] This invention provides a metaverse digital twin electronic dart system and method based on wearable motion capture technology, which has the following beneficial effects:

[0074] 1. This metaverse digital twin electronic dart system and method based on wearable motion capture technology eliminates the need for any wearable devices, allowing users to move naturally and lowering the barrier to entry. The TOF depth camera, combined with the MediaPipe framework, achieves high-precision motion capture. NPU acceleration significantly improves inference speed, reduces model size, and lowers memory usage. Asynchronous processing and dynamic frame interpolation ensure the smoothness and real-time nature of motion data. Furthermore, while the camera detects current light intensity and automatically adjusts its exposure parameters, the client connects to a smart home system via Wi-Fi. Based on adjustable parameters in the smart home lighting system, the system adjusts the lighting parameters of the smart home fixtures in conjunction with the camera's optimal operating parameters, ensuring the camera captures the environment within its optimal range, thereby improving motion capture accuracy. Attached Figure Description

[0075] Figure 1 This is a diagram of the metaverse digital twin electronic dart system architecture based on wearable motion capture technology of the present invention.

[0076] Figure 2 This is a schematic diagram of the motion capture process based on a TOF depth camera and the MediaPipe framework of the present invention;

[0077] Figure 3 This is a block diagram illustrating the asynchronous processing and dynamic frame interpolation technology implementation of the present invention.

[0078] Figure 4 This invention provides a dart physics simulation model based on UE5.

[0079] Figure 5 This is a flowchart of the digital twin scene construction method of the present invention. Detailed Implementation

[0080] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0081] like Figures 1-5As shown, the present invention provides a technical solution: a metaverse digital twin electronic dart system based on wearable motion capture technology, the system comprising the following modules:

[0082] Wearable motion capture module: Uses a TOF depth camera to capture user movements, identify and track user throwing movements and convert them into three-dimensional motion parameters;

[0083] Key point data processing and transmission module: adopts asynchronous processing mechanism and dynamic data frame interpolation technology to increase the actual detection frame rate of 22fps to a transmission frame rate of 60-90fps;

[0084] Physics simulation engine module: Based on UE5's physics engine, it calculates the dart's trajectory and impact position based on the captured throwing action parameters, including but not limited to angle of attack, initial velocity, and rotational angular velocity;

[0085] Digital Twin Scene Module: Using panoramic holographic oblique photography technology, a digital twin model of a real darts venue is constructed, including physical parameters and environmental characteristics, while providing fantastic creative scenes such as interstellar shooting ranges, underwater caves, and time rifts;

[0086] Gameplay Rules Module: Implements the rule logic for classic game formats such as 501 / 301 straight-down matches and innovative gameplay such as timed challenges and team target range breakthroughs;

[0087] Character growth module: Implements the user character's skill tree, equipment system, and growth path;

[0088] Social interaction module: Enables multi-player collaboration, team competition, spectating, and interactive functions;

[0089] UGC Creation Module: Provides scene creation tools, allowing users to design personalized scenes and custom rules;

[0090] Virtual-Real Integration Module: Supports data exchange between offline physical venues and virtual systems.

[0091] A method applied to the aforementioned metaverse digital twin electronic dart system based on wearable motion capture technology, the method comprising the following steps:

[0092] S1. System Initialization and User Calibration:

[0093] The camera is turned on to collect the user's three-dimensional spatial data in real time. At the same time, the client and the server establish a connection through the network to initialize the UE5 engine scene. The camera calculates the user's height, arm length, standing position and distance to the dartboard by detecting human key points.

[0094] The camera is a TOF depth camera with a resolution of 640×480, a frame rate of 30Hz, a depth accuracy of ±1cm, and a field of view of 75°×65°.

[0095] The client is responsible for image acquisition and rendering, while the server is responsible for data processing and storage.

[0096] Human key point detection uses the MediaPipe motion capture recognition framework to identify 21 hand key points and 33 body key points.

[0097] The camera can detect the current light intensity and automatically adjust the camera's exposure parameters. If the ambient light is insufficient, it will trigger a supplementary light prompt. The client can also connect to the smart home system via WIFI and adjust the lighting parameters of the smart home lighting fixtures based on the adjustable parameters in the smart home lighting system and the camera's optimal operating parameters.

[0098] S2. Wearless motion capture and data processing:

[0099] The TOF depth camera continuously captures user actions, generates point cloud data, and inputs it into the MediaPipe framework after filtering and noise reduction preprocessing.

[0100] The MediaPipe framework identifies 21 key points on the hand and 33 key points on the body, and calculates the coordinates of each joint.

[0101] Accelerate model inference using neural network processors;

[0102] The data acquisition and transmission processes are further separated to ensure that the acquisition thread does not block the transmission. Then, virtual frames are generated by predictive interpolation algorithms such as linear interpolation or Kalman filtering to increase the transmission frame rate to 60-90fps.

[0103] The transmission frequency is dynamically adjusted according to the network status, and priority is given to transmitting key action data such as the peak acceleration at the moment of throwing.

[0104] Calculation of throwing parameters based on motion data:

[0105] Direction angle: The angle between the throwing direction and the horizontal plane is calculated by the movement trajectory of key points of the hand, such as the vector from the wrist to the fingertips;

[0106] Initial velocity: Analyze the hand acceleration curve and calculate the linear velocity at the moment of release by combining the arm length;

[0107] Rotational angular velocity: Detects the wrist rotation trajectory and calculates the dart's spin speed using the quaternion rate of change;

[0108] The specific process of accelerating model inference using a neural network processor is as follows:

[0109] The MediaPipe framework was used to train a 21-keypoint detection model for the hand and a 33-keypoint detection model for the body. Keypoint recognition was achieved based on convolutional neural networks or lightweight models such as MobileNet.

[0110] By using quantization techniques supported by the NPU, such as 8-bit fixed-point quantization, floating-point model parameters can be converted into integers, reducing model size by 40% and memory usage by 35%.

[0111] The principle is to reduce computational complexity by reducing the number of bits stored for each parameter, while preserving the accuracy of key features by calibrating the data;

[0112] The quantized model is deployed to the neural network processor on the device, such as the NPU chip of a mobile phone or edge device, and its dedicated hardware units, such as the tensor processing unit (TPU), are used to accelerate matrix operations.

[0113] The NPU accelerates operations such as convolution and pooling through multi-core parallel computing, increasing the overall inference speed from 8fps to 22fps.

[0114] S3. Physical Simulation and Trajectory Calculation:

[0115] User action parameters include direction angle α, initial velocity V0, and rotational angular velocity ω;

[0116] Environmental parameters include gravity g, air density p, and wind speed v;

[0117] Trajectory equation calculation:

[0118]

[0119] The first term is the basic trajectory of the parabola, the second term is the influence of gravity, and the third term is the Magnus effect offset caused by rotation.

[0120] And air resistance and Magnus force:

[0121] Among them, C d =0.42 is the air drag coefficient, S m Magnus coefficient;

[0122] The coordinates of the intersection point between the dart and the target are calculated based on the trajectory equation to determine whether the target hits the double zone, triple zone, or bullet eye.

[0123] If the dart hits a hard material area, a rebound effect is triggered. The rebound speed is calculated based on the elastic parameter E=2000MPa. If the dart hits a soft material target, it is inserted and stops, and the scoreboard is updated.

[0124] S4, Digital Twin Scene Rendering and Interaction:

[0125] Real-world venue twin: A venue model is constructed using panoramic holographic oblique photography and BIM technology, giving it physical properties such as a floor friction coefficient μ=0.65 and acoustic reflection;

[0126] Creative scene rendering: such as the planetary gravitational field under zero gravity, i.e., the interstellar target range, and the time rift with random ±50% change in flight speed, generating high-precision textures and lighting effects through GAN network;

[0127] Based on user-selected settings, the flight trajectory and hit effects of the darts are rendered in real time, while ambient sounds and hit sound effects are played dynamically according to the scene.

[0128] S5, Multi-user Interaction and Social Features:

[0129] Room creation: Users can create rooms through the social module, which can support 2-8 people to join, and select the game mode;

[0130] Data synchronization: The server synchronizes the action data of each client via the WebRTC protocol with a latency of ≤200ms, ensuring that the throwing trajectory and score of multiple users are consistent in real time;

[0131] Team Mode: Teams of 3 players are divided into three roles: sniper (high-precision throwing), armor penetration (high-power penetration), and healer (supporting repair of the target). They cooperate in combat through an energy-sharing mechanism.

[0132] Spectator system: Supports switching between third-person perspective and bullseye perspective. Spectators can send emoji comments. The system automatically captures exciting moments such as a grand slam throw and generates replay videos.

[0133] S6. User-Generated Content and System Extensions:

[0134] Users upload photos of real-world venues, and AI algorithms automatically generate 3D twin scenes; users can customize physical parameters such as gravity, wind speed, and obstacle layouts such as moving targets and shield walls.

[0135] Users can design custom game formats such as time-limited + random environmental changes and scoring rules such as triggering a chain explosion of points when a specific area is hit through the rules editor.

[0136] Users can package and upload their own scenarios and rules to the community, and other users can download and experience them directly.

[0137] Creators will receive virtual rewards such as exclusive equipment, skins, and experience boosts based on their likes and download rankings, and high-quality content may be selected for the official recommendation pool.

[0138] S7, Virtual-Real Integration and Commercial Applications:

[0139] If a pressure-sensitive floor is used, it is placed within the camera's capture range, and the user stands on the pressure-sensitive floor to operate. The pressure-sensitive floor works in conjunction with the TOF camera to detect whether the user's standing posture is compliant; when used with physical darts equipped with sensors, the virtual trajectory is calibrated and synchronized with the virtual scene, such as the physical throwing angle.

[0140] Offline store competition results are synchronized to the online leaderboard, and users can participate in darts activities and unlock prizes through LBS technology;

[0141] The system demonstrates the principles of parabolic trajectories through virtual scenarios, such as fixing the initial velocity and adjusting the angle to observe the trajectory changes, for use in physics teaching.

[0142] Design low-intensity training modes for the elderly or patients with upper limb injuries, analyze joint range of motion through motion capture, generate rehabilitation reports, and thus enable the system to be applied to rehabilitation training.

[0143] S8. System Closed Loop and Data Management:

[0144] The user roles are set up with a growth system and a data platform. The growth system gains experience through matches, and the experience can be used to unlock the user role's skills. The data platform stores data such as user action history, win rate, and preferred scenarios, which are used for AI analysis to optimize matching algorithms, such as recommending similar players to play against.

[0145] If the system detects a camera position shift, it triggers an automatic calibration process, prompting the user to stand back in the designated area. The server records data transmission latency, frame rate fluctuations, and other indicators in real time, automatically issuing alarms and switching to a backup node when an anomaly occurs.

[0146] Based on the above description, this system requires no wearable devices, allowing users to move naturally while lowering the barrier to entry. The TOF depth camera, combined with the MediaPipe framework, achieves high-precision motion capture. NPU acceleration significantly improves inference speed, reduces model size, and lowers memory usage. Asynchronous processing and dynamic frame interpolation ensure the smoothness and real-time nature of motion data. Furthermore, while the camera detects current light intensity and automatically adjusts its exposure parameters, the system connects to a smart home system via Wi-Fi. Based on adjustable parameters in the smart home lighting system, the system adjusts the lighting parameters of the smart home fixtures in conjunction with the camera's optimal operating parameters, ensuring the camera captures the environment within its optimal range, thereby improving motion capture accuracy.

[0147] In summary, the workflow of the Metaverse digital twin electronic dart system based on wearable motion capture technology is briefly described as follows:

[0148] The throwing process of an electronic dart system:

[0149] Users access the system via a TOF depth camera;

[0150] The system automatically calibrates the camera position and user posture;

[0151] When a user makes a throwing motion, the system captures and analyzes it in real time, and calculates throwing parameters based on the motion data.

[0152] The physics engine calculates the dart trajectory based on the captured data, performing non-standard trajectory calculations based on motion parameters and environmental parameters;

[0153] The system renders the dart's flight path and calculates the hit result;

[0154] Update scores and game status, triggering corresponding effects and feedback.

[0155] The embodiments of the present invention are given for illustrative and descriptive purposes only, and are not intended to be exhaustive or to limit the invention to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of the invention, and to enable those skilled in the art to understand the invention and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. A meta-universe digital twin electronic dart system based on a non-wearable motion capture technology, characterized in that: The system comprises the following modules: Non-wearable motion capture module: using TOF depth camera to capture user motion, identify and track user throwing action and convert it into three-dimensional motion parameters; Key point data processing and transmission module: using asynchronous processing mechanism and dynamic data frame filling technology, the actual detection frame rate is increased from 22fps to 60-90fps of the sending frame rate; Physical simulation engine module: based on UE5 physical engine, the flight trajectory and hitting position of the dart are calculated according to the captured throwing action parameters, including but not limited to attack angle, initial speed, angular velocity of rotation; Digital twin scene module: through panoramic holographic tilt photography technology, a digital twin model of the real dart venue is constructed, including physical parameters and environmental characteristics, while providing fantastic creative scenes, such as interstellar target field, underwater cave, time crack; Game rules module: realizing the rules of classic race, i.e. 501 / 301 straight race, and innovative play, i.e. time challenge race and team target field breakout; Character growth module: realizing the skill tree, equipment system and growth path of the user role; Social interaction module: realizing multi-player cooperation, team competition, watching and interaction functions; UGC creation module: providing scene creation tools to allow users to design personalized scenes and customize rules; Virtual-real fusion module: supporting data intercommunication between offline physical venues and virtual systems; The method of the system comprises the following steps: S1, system initialization and user calibration: Start the camera, real-time collect three-dimensional space data of the user, at the same time, the client and the server establish connection through the network, initialize the UE5 engine scene, and the camera calculates the user's height, arm length, standing position and dart target distance through human key point detection; S2, non-wearable motion capture and data processing: TOF depth camera continuously captures user motion, generates point cloud data, and inputs the data into MediaPipe framework after filtering and noise reduction preprocessing; MediaPipe framework identifies 21 hand key points and 33 body key points, and calculates joint coordinates; Through a neural network processor to accelerate model reasoning; Separate data collection and transmission processes to ensure that the collection thread does not block transmission, and then generate virtual frames through a prediction interpolation algorithm, including linear interpolation or Kalman filtering, to increase the sending frame rate to 60-90fps; According to the network state, dynamically adjust the sending frequency, and preferentially transmit key motion data, including acceleration peak at the moment of throwing; Based on the motion data, calculate the throwing parameters: Direction angle: calculate the angle between the throwing direction and the horizontal plane through the hand key point motion trajectory, including the vector from the wrist to the fingertip; Initial speed: analyze the hand acceleration curve and calculate the linear speed at the moment of release combined with the arm length; Rotational angular velocity: detect the wrist rotation trajectory and calculate the dart spin speed through the rate of change of quaternion; S3, physical simulation and trajectory calculation: The user action parameters include a direction angle , an initial speed , an angular speed of rotation ; The environmental parameters include gravity , air density , wind speed ; Trajectory equation calculation: Wherein, the first term is the parabolic basic trajectory, the second term is the gravity influence, and the third term is the Magnus effect offset generated by rotation; And air resistance and Magnus force: wherein is the air resistance coefficient, is the Magnus coefficient; S4, digital twin scene rendering and interaction: Real venue twin: build a venue model through panoramic holographic tilt photography and BIM technology, and give the floor friction coefficient , acoustic reflection, and other physical properties; Creative scene rendering: including the planetary gravitational field under zero gravity, i.e. the interstellar target field, the time crack with the flight speed randomly changing by ±50%, and the high-precision texture and light and shadow effect generated by the GAN network; According to the user selection setting, the flying trajectory and hitting special effect of the dart are rendered in real time, and the environmental sound and hitting sound effect are played dynamically according to the scene.

2. A method characterized by, The application is applied to the meta-universe digital twin electronic dart system based on the non-wearable motion capture technology in claim 1.

3. A method according to claim 2, characterised in that: In step S1, the camera is a TOF depth camera, with a resolution of 640*480, a frame rate of 30Hz, a depth accuracy of ±1cm, and a field of view of 75*65 degrees. The client is responsible for image acquisition and rendering, and the server is responsible for data processing and storage. Human key point detection is achieved by recognizing 21 hand key points and 33 body key points through the MediaPipe motion capture recognition framework.

4. A method according to claim 2, characterized in that: In step S1, the camera detects the current light intensity and automatically adjusts the camera exposure parameters; if the ambient light is insufficient, a light supplement prompt is triggered, and the client can connect the smart home system through WIFI, adjust the lamp parameters of the smart home lighting based on the adjustable parameters in the smart home lighting, and cooperate with the best working parameters of the camera to adjust the lamp parameters of the smart home lighting.

5. The method of claim 2, wherein: In step S2, the specific process of accelerating model inference by the neural network processor is as follows: The hand 21 key point and body 33 key point detection model is trained using the MediaPipe framework, based on a convolutional neural network or a lightweight model, including MobileNet, to realize key point recognition; Through the quantization technology supported by NPU, including 8-bit fixed-point quantization, the floating-point model parameters are converted to integers, the model volume is reduced by 40%, and the memory occupation is reduced by 35%; The principle is to reduce the storage bit number of each parameter and reduce the calculation complexity, while retaining the key feature accuracy through calibration data; The quantized model is deployed to the neural network processor of the device end, including the NPU chip of the mobile phone or edge device, and the special hardware units of the NPU are used to accelerate matrix operations, including the tensor processing unit TPU; NPU accelerates convolution, pooling and other operations through multi-core parallel computing, so that the full-process inference speed is increased from 8fps before optimization to 22fps.

6. A method according to claim 2, characterized in that: In step S3, the intersection coordinates of the dart and the target surface are solved according to the trajectory equation, and it is judged whether it hits the double area, the triple area or the bullseye. If hit hard material area, trigger bounce effect according to the elasticity parameters Calculate bounce speed, hit soft material target surface, dart insertion and static, update score board.

7. A method according to claim 2, characterized by: The method further comprises the following steps: S5, multi-person interaction and social function: Room creation: users create a room through a social module, support 2-8 people to join, and select a match mode; Data synchronization: the server synchronizes the action data of each client through the WebRTC protocol, with a delay of ≤200ms, to ensure that the multi-user throwing trajectory and score are consistent in real time; Team mode: 3 people form a team, with different positions, i.e. high-precision throwing, high-power penetration and treatment, and cooperate through an energy sharing mechanism; Spectator system: supports third-person perspective and bullseye perspective switching, spectators can send expression pop-up windows, the system automatically captures exciting moments, including full penetration throwing and generates a playback video.

8. A method according to claim 7, characterised in that: The method further comprises the following steps: S6, user-generated content and system expansion: Users upload real venue photos, and the system automatically generates a 3D twin scene through AI algorithms; users can customize physical parameters, including gravity, wind speed, and obstacle layout, including moving targets and shield walls; Users design custom game rules through a rule editor, including time limits and random environmental changes, and scoring rules including hitting specific areas to trigger chain explosions; Users can package their custom scenes and rules and upload them to the community, where other users can download and experience them directly; Through likes and download rankings, creators are given virtual rewards, including exclusive equipment, skins, and experience bonuses, and high-quality content can be selected for official recommendations.

9. A method according to claim 8, characterised in that: The method further comprises the following steps: Virtual and real fusion and commercial application: If a pressure-sensitive floor is used, it is placed within the camera's capture range, and users stand on the pressure-sensitive floor to operate. The pressure-sensitive floor works with the TOF camera to detect whether the user's standing posture is compliant; In conjunction with the physical dart, the sensor is equipped with a sensor, which is calibrated with the virtual track, including synchronizing the physical throwing angle to the virtual scene; Offline store competition results are synchronized to the online leaderboard, and users can participate in dart activities through LBS technology and unlock prizes; The system demonstrates the principle of parabola through a virtual scene, including fixed initial speed, adjusting the angle to observe the trajectory change for physics teaching; For the elderly or upper limb injury patients, a low intensity training mode is designed, the joint activity is analyzed through motion capture, and a rehabilitation report is generated, so that the system can be applied to rehabilitation training.

10. A method according to claim 9, wherein: The method further comprises the following steps: System closed loop and data management: User roles have a growth system and a data platform. The growth system gains experience through competition, and through experience, the user's skills can be unlocked. The data platform stores user action history, win rate, and preferred scene data, which are used for AI analysis and optimization of matching algorithms, including recommending similar players for a match; If the system detects that the camera position is offset, it triggers an automatic calibration process, prompting the user to stand in the designated area again. The server records data transmission delay, frame rate fluctuation, and other indicators in real time, and automatically alerts and switches to a backup node when an exception occurs.

Images