Metacosmic digital twin electronic dart system and method based on wearing-free somatosensory motion capture technology

Abstract

The invention discloses an element universe digital twin electronic dart system and method based on a non-wearing motion capture technology, and relates to the technical field of motion interaction, and the system comprises the following modules: a non-wearing motion capture module; a key point data processing and transmission module; a physical simulation engine module; a digital twin scene module; a playing regulation module; a role growth module; a UGC creation module; and a virtual-real fusion module. According to the meta-universe digital twin electronic dart system and method based on the non-wearing somatosensory motion capture technology, no equipment needs to be worn, the user participation threshold can be lowered while the user action is natural, high-precision motion capture is achieved, the reasoning speed is remarkably increased through NPU acceleration optimization, the model size is reduced, and memory occupation is reduced; and the fluency and the real-time performance of the action data are ensured by combining asynchronous processing and a dynamic frame filling technology.

Description

Technical Field

[0001] The present invention relates to the field of somatosensory interaction technology, and specifically to a metaverse digital twin electronic dart system and method based on wearable somatosensory motion capture technology. Background Art

[0002] With the rise of the concept of the metaverse, digital sports activities have gradually attracted attention. Traditional electronic dart games usually require the use of wearable devices such as VR headsets and motion controllers. However, motion sensing devices are inconvenient to wear and the user experience is unnatural. Traditional motion sensing devices are expensive and have a low penetration rate. In addition, traditional motion capture systems have a low frame rate and high latency, which affects real-time performance. Summary of the Invention

[0003] In response to the shortcomings of the existing technology, the present invention provides a metaverse digital twin electronic dart system and method based on wearable body motion capture technology, which solves the problems raised in the above background technology.

[0004] To achieve the above objectives, the present invention is implemented through the following technical solutions: a Metaverse digital twin electronic dart system based on wearable body motion capture technology, the system comprising the following modules:

[0005] Wearable motion capture module: uses a TOF depth camera to capture user movements, identify and track the user's throwing movements and convert them into three-dimensional motion parameters;

[0006] Key point data processing and transmission module: Using asynchronous processing mechanism and dynamic data frame filling technology, the actual detection frame rate of 22fps is increased to a transmission frame rate of 60-90fps;

[0007] Physics simulation engine module: Based on the UE5 physics engine, it calculates the flight trajectory and impact position of the dart according to the captured throwing action parameters. The parameters include but are not limited to the angle of attack, initial velocity, and rotational angular velocity.

[0008] Digital Twin Scene Module: This module uses panoramic holographic tilt photography technology to build a digital twin model of a real darts venue, including physical parameters and environmental characteristics, while also providing fantastical and creative scenes such as interstellar shooting ranges, underwater caves, and time rifts.

[0009] Gameplay rules module: Implements the rule logic for classic game systems such as 501 / 301 straight-out matches and innovative game systems such as time-limited challenges and team shooting range breakouts;

[0010] Character growth module: implements the user character's skill tree, equipment system and growth path;

[0011] Social interaction module: realizes multi-person collaboration, team competition, spectatorship and interaction functions;

[0012] UGC creation module: provides scene creation tools, allowing users to design personalized scenes and customize rules;

[0013] Virtual-reality fusion module: supports data exchange between offline physical venues and virtual systems.

[0014] A method is applied to the aforementioned Metaverse digital twin electronic dart system based on wearable body motion capture technology, the method comprising the following steps:

[0015] S1. System initialization and user calibration:

[0016] The camera is turned on to collect the user's 3D spatial data in real time. At the same time, the client and server establish a connection through the network to initialize the UE5 engine scene. The camera detects key points on the human body and calculates the user's height, arm length, standing position, and distance from the dartboard.

[0017] S2. Non-wearable motion capture and data processing:

[0018] The TOF depth camera continuously captures user movements, generates point cloud data, and then inputs it into the MediaPipe framework after filtering and noise reduction preprocessing;

[0019] The MediaPipe framework identifies 21 key points of the hand and 33 key points of the body and calculates the coordinates of each joint;

[0020] Accelerate model inference through neural network processors;

[0021] Separate the data acquisition and transmission processes to ensure that the acquisition thread does not block transmission. Then, generate virtual frames through predictive interpolation algorithms such as linear interpolation or Kalman filtering to increase the transmission frame rate to 60-90fps.

[0022] Dynamically adjust the transmission frequency according to the network status, giving priority to transmitting key action data such as the peak acceleration at the moment of throwing;

[0023] Calculate throwing parameters based on motion data:

[0024] Direction angle: Calculate the angle between the throwing direction and the horizontal plane through the motion trajectory of key points of the hand, such as the vector from the wrist to the fingertips;

[0025] Initial velocity: Analyze the hand acceleration curve and calculate the linear velocity at the moment of release based on the arm length;

[0026] Rotational angular velocity: Detects the wrist rotation trajectory and calculates the dart's spin speed through the quaternion change rate;

[0027] S3. Physical simulation and trajectory calculation:

[0028] User action parameters include direction angle α, initial velocity V0, and rotation angular velocity ω;

[0029] Environmental parameters include gravity g, air density p, and wind speed v;

[0030] Trajectory equation calculation:

[0031]

[0032] Among them, the first term is the parabolic basic trajectory, the second term is the influence of gravity, and the third term is the Magnus effect offset caused by rotation;

[0033] And air resistance and Magnus force:

[0034] Among them, C d =0.42 is the air resistance coefficient, S m is the Magnus coefficient;

[0035] S4. Digital twin scene rendering and interaction:

[0036] Real venue twin: Using panoramic holographic oblique photography and BIM technology to build a venue model, the floor is given physical properties such as a friction coefficient of μ = 0.65 and acoustic reflection;

[0037] Creative scene rendering: such as the planetary gravitational field in zero gravity, i.e., an interstellar shooting range, and time rifts with random ±50% flight speed variations, using the GAN network to generate high-precision textures and lighting effects;

[0038] Based on the user's selected settings, the dart flight trajectory and hit effects are rendered in real time, and the ambient sound and hit sound effects are played dynamically according to the scene.

[0039] Furthermore, in step S1, the camera is a TOF depth camera with a resolution of 640×480, a frame rate of 30 Hz, a depth accuracy of ±1 cm, and a field of view of 75°×65°;

[0040] The client is responsible for image acquisition and rendering, and the server is responsible for data processing and storage;

[0041] Human body key point detection uses the MediaPipe motion capture recognition framework to identify 21 hand + 33 body key points.

[0042] Furthermore, in step S1, the camera detects the current light intensity and automatically adjusts the camera exposure parameters; if the ambient light is insufficient, a fill light prompt is triggered, and the client can connect to the smart home system via WIFI, and adjust the lighting parameters of the smart home lighting based on the adjustable parameters in the smart home lighting and in conjunction with the camera's optimal working parameters.

[0043] Furthermore, in step S2, the specific process of the neural network processor accelerating model reasoning is as follows:

[0044] Use the MediaPipe framework to train a 21-keypoint hand and 33-keypoint body detection model, and implement keypoint recognition based on convolutional neural networks or lightweight models such as MobileNet;

[0045] By using quantization technologies supported by the NPU, such as 8-bit fixed-point quantization, floating-point model parameters are converted to integers, reducing model size by 40% and memory usage by 35%.

[0046] The principle is to reduce the computational complexity by reducing the number of storage bits for each parameter while retaining the accuracy of key features through calibration data;

[0047] Deploy the quantized model to a device-side neural network processor, such as an NPU chip on a mobile phone or edge device, and use its dedicated hardware units, such as the tensor processing unit (TPU), to accelerate matrix operations.

[0048] The NPU accelerates operations such as convolution and pooling through multi-core parallel computing, increasing the full-process inference speed from 8fps before optimization to 22fps.

[0049] Furthermore, in step S3, the coordinates of the intersection of the dart and the target surface are solved according to the trajectory equation to determine whether the dart hits the double area, triple area or bullseye;

[0050] If the dart hits a hard material area, the rebound effect is triggered and the rebound speed is calculated based on the elastic parameter E = 2000MPa. If the dart hits a soft material target surface, the dart is inserted and stops, and the scoreboard is updated.

[0051] Furthermore, the method further comprises the following steps:

[0052] S5. Multi-person interaction and social functions:

[0053] Room creation: Users create a room through the social module, supporting 2-8 people to join and select the competition format;

[0054] Data synchronization: The server synchronizes the action data of each client through the WebRTC protocol, with a delay of ≤200ms, ensuring that the throwing trajectories and scores of multiple users are consistent in real time;

[0055] Team mode: 3 players form a team, divided into sniper position (high-precision throwing), defense-breaking position (high-force penetration), and healing position (assisting with target repair), and cooperate in combat through an energy-sharing mechanism;

[0056] Spectator system: supports switching between third-person perspective and bull's-eye perspective. Spectators can send emojis and comments. The system automatically captures exciting moments such as slam throws and generates replay videos.

[0057] Furthermore, the method further comprises the following steps:

[0058] S6. User-generated content and system extension:

[0059] Users upload photos of real-world venues and automatically generate 3D twin scenes through AI algorithms; customize physical parameters such as gravity, wind speed, and obstacle layouts such as moving targets and shield walls;

[0060] Users can use the rule editor to design custom competition systems such as time limit + random environment changes, and scoring rules such as hitting a specific area to trigger a chain explosion score;

[0061] Users can upload their own scenarios and rules to the community, and other users can download and experience them directly;

[0062] Creators will be given virtual rewards such as exclusive equipment, skins, and experience bonuses through likes and download rankings, and high-quality content can be selected into the official recommendation pool.

[0063] Furthermore, the method further comprises the following steps:

[0064] S7. Virtual-Real Integration and Commercial Applications:

[0065] If a pressure-sensitive floor is used, it should be placed within the camera's capture range. The user then stands on the pressure-sensitive floor to operate. The pressure-sensitive floor works in conjunction with the TOF camera to detect whether the user's standing posture is in compliance. The physical dart is equipped with a sensor to calibrate the virtual trajectory, such as synchronizing the physical throwing angle to the virtual scene.

[0066] The results of offline store competitions are synchronized to the online leaderboard, and users can participate in dart activities and unlock prizes through LBS technology;

[0067] The system uses virtual scenes to demonstrate the principles of parabola, such as fixing the initial velocity and adjusting the angle to observe the trajectory changes, for physics teaching;

[0068] A low-intensity training mode is designed for the elderly or patients with upper limb injuries. The joint range of motion is analyzed through motion capture and rehabilitation reports are generated, so that the system can be applied to rehabilitation training.

[0069] Furthermore, the method further comprises the following steps:

[0070] S8. System closed loop and data management:

[0071] The user character is equipped with a growth system and a data center. The growth system gains experience through competition, which can unlock the user character's skills. The data center stores data such as user action history, win rate, and preferred scenarios, which are used for AI analysis and optimization of matching algorithms, such as recommending similar players for battle.

[0072] If the system detects that the camera position is offset, it triggers an automatic calibration process and prompts the user to stand again in the designated area. The server records indicators such as data transmission delay and frame rate fluctuation in real time, and automatically issues an alarm and switches to the backup node in the event of an abnormality.

[0073] The present invention provides a metaverse digital twin electronic dart system and method based on wearable motion capture technology, which has the following beneficial effects:

[0074] 1. This Metaverse digital twin electronic dart system and method based on wearable body motion capture technology does not require any wearable equipment, making user movements natural while lowering the threshold for user participation. The TOF depth camera is combined with the MediaPipe framework to achieve high-precision motion capture, and significantly improves the inference speed through NPU acceleration optimization, reduces the model size, and reduces memory usage. The combination of asynchronous processing and dynamic frame interpolation technology ensures the smoothness and real-time nature of motion data. In addition, while the camera can detect the current light intensity and automatically adjust the camera exposure parameters, it cooperates with the client to connect to the smart home system via Wi-Fi. Based on the adjustable parameters in the smart home lighting, the lighting parameters of the smart home lighting are adjusted in conjunction with the camera's optimal working parameters, so that the camera capture environment is as close to its optimal parameters as possible, thereby improving the accuracy of motion capture. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 This is the architecture diagram of the Metaverse digital twin electronic dart system based on wearable motion capture technology of the present invention;

[0076] Figure 2 This is a schematic diagram of the motion capture process based on the TOF depth camera and MediaPipe framework of the present invention;

[0077] Figure 3 This is a block diagram of the asynchronous processing and dynamic frame interpolation technology of the present invention;

[0078] Figure 4 This is the dart physics simulation model based on UE5 of the present invention;

[0079] Figure 5 This is a flow chart of the digital twin scene construction method of the present invention. DETAILED DESCRIPTION

[0080] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0081] like Figure 1-Figure 5As shown, the present invention provides a technical solution: a metaverse digital twin electronic dart system based on wearable body motion capture technology, the system includes the following modules:

[0082] Wearable motion capture module: uses a TOF depth camera to capture user movements, identify and track the user's throwing movements and convert them into three-dimensional motion parameters;

[0083] Key point data processing and transmission module: Using asynchronous processing mechanism and dynamic data frame filling technology, the actual detection frame rate of 22fps is increased to a transmission frame rate of 60-90fps;

[0084] Physics simulation engine module: Based on the UE5 physics engine, it calculates the flight trajectory and impact position of the dart according to the captured throwing action parameters. The parameters include but are not limited to the angle of attack, initial velocity, and rotational angular velocity.

[0085] Digital Twin Scene Module: This module uses panoramic holographic tilt photography technology to build a digital twin model of a real darts venue, including physical parameters and environmental characteristics, while also providing fantastical and creative scenes such as interstellar shooting ranges, underwater caves, and time rifts.

[0086] Gameplay rules module: Implements the rule logic for classic game systems such as 501 / 301 straight-out matches and innovative game systems such as time-limited challenges and team shooting range breakouts;

[0087] Character growth module: implements the user character's skill tree, equipment system and growth path;

[0088] Social interaction module: realizes multi-person collaboration, team competition, spectatorship and interaction functions;

[0089] UGC creation module: provides scene creation tools, allowing users to design personalized scenes and customize rules;

[0090] Virtual-reality fusion module: supports data exchange between offline physical venues and virtual systems.

[0091] A method is applied to the aforementioned Metaverse digital twin electronic dart system based on wearable body motion capture technology, the method comprising the following steps:

[0092] S1. System initialization and user calibration:

[0093] The camera is turned on to collect the user's 3D spatial data in real time. At the same time, the client and server establish a connection through the network to initialize the UE5 engine scene. The camera detects key points on the human body and calculates the user's height, arm length, standing position, and distance from the dartboard.

[0094] The camera is a TOF depth camera with a resolution of 640×480, a frame rate of 30Hz, a depth accuracy of ±1cm, and a field of view of 75°×65°;

[0095] The client is responsible for image acquisition and rendering, and the server is responsible for data processing and storage;

[0096] Human key point detection uses the MediaPipe motion capture recognition framework to identify 21 hand and 33 body key points;

[0097] The camera can detect the current light intensity and automatically adjust the camera exposure parameters. If the ambient light is insufficient, a fill light prompt will be triggered. The client can also connect to the smart home system via Wi-Fi and adjust the parameters of the smart home lighting fixtures based on the adjustable parameters of the smart home lighting and the optimal working parameters of the camera.

[0098] S2. Non-wearable motion capture and data processing:

[0099] The TOF depth camera continuously captures user movements, generates point cloud data, and then inputs it into the MediaPipe framework after filtering and noise reduction preprocessing;

[0100] The MediaPipe framework identifies 21 key points of the hand and 33 key points of the body and calculates the coordinates of each joint;

[0101] Accelerate model inference through neural network processors;

[0102] Separate the data acquisition and transmission processes to ensure that the acquisition thread does not block transmission. Then, generate virtual frames through predictive interpolation algorithms such as linear interpolation or Kalman filtering to increase the transmission frame rate to 60-90fps.

[0103] Dynamically adjust the transmission frequency according to the network status, giving priority to transmitting key action data such as the peak acceleration at the moment of throwing;

[0104] Calculate throwing parameters based on motion data:

[0105] Direction angle: Calculate the angle between the throwing direction and the horizontal plane through the motion trajectory of key points of the hand, such as the vector from the wrist to the fingertips;

[0106] Initial velocity: Analyze the hand acceleration curve and calculate the linear velocity at the moment of release based on the arm length;

[0107] Rotational angular velocity: Detects the wrist rotation trajectory and calculates the dart's spin speed through the quaternion change rate;

[0108] The specific process of neural network processor acceleration model reasoning is as follows:

[0109] Use the MediaPipe framework to train a 21-keypoint hand and 33-keypoint body detection model, and implement keypoint recognition based on convolutional neural networks or lightweight models such as MobileNet;

[0110] By using quantization technologies supported by the NPU, such as 8-bit fixed-point quantization, floating-point model parameters are converted to integers, reducing model size by 40% and memory usage by 35%.

[0111] The principle is to reduce the computational complexity by reducing the number of storage bits for each parameter while retaining the accuracy of key features through calibration data;

[0112] Deploy the quantized model to a device-side neural network processor, such as an NPU chip on a mobile phone or edge device, and use its dedicated hardware units, such as the tensor processing unit (TPU), to accelerate matrix operations.

[0113] The NPU accelerates operations such as convolution and pooling through multi-core parallel computing, increasing the full-process inference speed from 8fps before optimization to 22fps.

[0114] S3. Physical simulation and trajectory calculation:

[0115] User action parameters include direction angle α, initial velocity V0, and rotation angular velocity ω;

[0116] Environmental parameters include gravity g, air density p, and wind speed v;

[0117] Trajectory equation calculation:

[0118]

[0119] Among them, the first term is the parabolic basic trajectory, the second term is the influence of gravity, and the third term is the Magnus effect offset caused by rotation;

[0120] And air resistance and Magnus force:

[0121] Among them, C d =0.42 is the air resistance coefficient, S m is the Magnus coefficient;

[0122] Solve the coordinates of the intersection of the dart and the target surface according to the trajectory equation to determine whether it hits the double area, triple area or bullseye;

[0123] If the dart hits a hard surface, a rebound effect is triggered and the rebound speed is calculated based on the elastic parameter E = 2000 MPa. If the dart hits a soft surface, the dart is inserted and stops, and the scoreboard is updated.

[0124] S4. Digital twin scene rendering and interaction:

[0125] Real venue twin: Using panoramic holographic oblique photography and BIM technology to build a venue model, the floor is given physical properties such as a friction coefficient of μ = 0.65 and acoustic reflection;

[0126] Creative scene rendering: such as the planetary gravitational field in zero gravity, i.e., an interstellar shooting range, and time rifts with random ±50% flight speed variations, using the GAN network to generate high-precision textures and lighting effects;

[0127] Based on user-selected settings, the dart flight trajectory and hit effects are rendered in real time, while ambient sounds and hit sound effects are played dynamically according to the scene.

[0128] S5. Multi-person interaction and social functions:

[0129] Room creation: Users create a room through the social module, supporting 2-8 people to join and select the competition format;

[0130] Data synchronization: The server synchronizes the action data of each client through the WebRTC protocol, with a delay of ≤200ms, ensuring that the throwing trajectories and scores of multiple users are consistent in real time;

[0131] Team mode: 3 players form a team, divided into sniper position (high-precision throwing), defense-breaking position (high-force penetration), and healing position (assisting with target repair), and cooperate in combat through an energy-sharing mechanism;

[0132] Spectator system: supports switching between third-person perspective and bull's-eye perspective. Spectators can send emojis and comments. The system automatically captures exciting moments such as slam throws and generates replay videos.

[0133] S6. User-generated content and system extension:

[0134] Users upload photos of real-world venues and automatically generate 3D twin scenes through AI algorithms; customize physical parameters such as gravity, wind speed, and obstacle layouts such as moving targets and shield walls;

[0135] Users can use the rule editor to design custom competition systems such as time limit + random environment changes, and scoring rules such as hitting a specific area to trigger a chain explosion score;

[0136] Users can upload their own scenarios and rules to the community, and other users can download and experience them directly;

[0137] Creators will be given virtual rewards such as exclusive equipment, skins, and experience bonuses through likes and download rankings, and high-quality content can be selected into the official recommendation pool.

[0138] S7. Virtual-Real Integration and Commercial Applications:

[0139] If a pressure-sensitive floor is used, it should be placed within the camera's capture range. The user then stands on the pressure-sensitive floor to operate. The pressure-sensitive floor works in conjunction with the TOF camera to detect whether the user's standing posture is in compliance. The physical dart is equipped with a sensor to calibrate the virtual trajectory, such as synchronizing the physical throwing angle to the virtual scene.

[0140] The results of offline store competitions are synchronized to the online leaderboard, and users can participate in dart activities and unlock prizes through LBS technology;

[0141] The system uses virtual scenes to demonstrate the principles of parabola, such as fixing the initial velocity and adjusting the angle to observe the trajectory changes, for physics teaching;

[0142] A low-intensity training mode is designed for the elderly or patients with upper limb injuries. The joint range of motion is analyzed through motion capture and rehabilitation reports are generated, so that the system can be applied to rehabilitation training.

[0143] S8. System closed loop and data management:

[0144] The user character is equipped with a growth system and a data center. The growth system gains experience through competition, which can unlock the user character's skills. The data center stores data such as user action history, win rate, and preferred scenarios, which are used for AI analysis and optimization of matching algorithms, such as recommending similar players for battle.

[0145] If the system detects camera position deviation, it triggers an automatic calibration process and prompts the user to stand back in the designated area. The server records indicators such as data transmission delay and frame rate fluctuation in real time, and automatically issues an alarm and switches to the backup node in case of abnormality.

[0146] Based on the above description, this system does not require wearing any equipment, making the user's movements natural while lowering the user participation threshold. The TOF depth camera is combined with the MediaPipe framework to achieve high-precision motion capture, and significantly improves the inference speed through NPU acceleration optimization, reduces the model size, and reduces memory usage. The combination of asynchronous processing and dynamic frame interpolation technology ensures the smoothness and real-time nature of motion data. In addition, while the camera can detect the current light intensity and automatically adjust the camera exposure parameters, it cooperates with the client to connect to the smart home system via WIFI. Based on the adjustable parameters in the smart home lighting, the parameters of the smart home lighting fixtures are adjusted in conjunction with the camera's optimal working parameters, so that the camera capture environment is as close to its optimal parameters as possible, thereby improving the accuracy of motion capture.

[0147] In summary, the workflow of the Metaverse digital twin electronic dart system based on wearable motion capture technology is briefly described as follows:

[0148] Throwing process of electronic dart system:

[0149] The user enters the system through the TOF depth camera;

[0150] The system automatically calibrates the camera position and user posture;

[0151] The user makes a throwing motion, which the system captures and analyzes in real time, calculating the throwing parameters based on the motion data;

[0152] The physics engine calculates the trajectory of the dart based on the captured data. It performs non-standard trajectory calculation based on action parameters and environmental parameters.

[0153] The system renders the dart's flight process and calculates the hit result;

[0154] Update the score and game status, triggering corresponding effects and feedback.

[0155] The embodiments of the present invention are presented for purposes of illustration and description and are not intended to be exhaustive or to limit the invention to the disclosed forms. Many modifications and variations will be apparent to those skilled in the art. The embodiments are chosen and described in order to better illustrate the principles of the invention and its practical application and to enable those skilled in the art to understand the invention and design various embodiments with various modifications as suited for specific applications.

Claims

1. A Metaverse digital twin electronic dart system based on wearable motion capture technology, characterized by: The system includes the following modules: Wearable motion capture module: uses a TOF depth camera to capture user movements, identify and track the user's throwing movements and convert them into three-dimensional motion parameters; Key point data processing and transmission module: Using asynchronous processing mechanism and dynamic data frame filling technology, the actual detection frame rate of 22fps is increased to a transmission frame rate of 60-90fps; Physics simulation engine module: Based on the UE5 physics engine, it calculates the flight trajectory and impact position of the dart according to the captured throwing action parameters. The parameters include but are not limited to the angle of attack, initial velocity, and rotational angular velocity. Digital Twin Scene Module: This module uses panoramic holographic tilt photography technology to build a digital twin model of a real darts venue, including physical parameters and environmental characteristics, while also providing fantastical and creative scenes such as interstellar shooting ranges, underwater caves, and time rifts. Gameplay rules module: Implements the rule logic for classic game systems such as 501 / 301 straight-out matches and innovative game systems such as time-limited challenges and team shooting range breakouts; Character growth module: implements the user character's skill tree, equipment system and growth path; Social interaction module: realizes multi-person collaboration, team competition, spectatorship and interaction functions; UGC creation module: provides scene creation tools, allowing users to design personalized scenes and customize rules; Virtual-reality fusion module: supports data exchange between offline physical venues and virtual systems.

2. A method, applied to the Metaverse digital twin electronic dart system based on wearable motion capture technology as claimed in claim 1, characterized in that: The method comprises the following steps: S1. System initialization and user calibration: The camera is turned on to collect the user's 3D spatial data in real time. At the same time, the client and server establish a connection through the network to initialize the UE5 engine scene. The camera detects key points on the human body and calculates the user's height, arm length, standing position, and distance from the dartboard. S2. Non-wearable motion capture and data processing: The TOF depth camera continuously captures user movements, generates point cloud data, and then inputs it into the MediaPipe framework after filtering and noise reduction preprocessing; The MediaPipe framework identifies 21 key points of the hand and 33 key points of the body and calculates the coordinates of each joint; Accelerate model inference through neural network processors; Separate the data acquisition and transmission processes to ensure that the acquisition thread does not block transmission. Then, generate virtual frames through predictive interpolation algorithms such as linear interpolation or Kalman filtering to increase the transmission frame rate to 60-90fps. Dynamically adjust the transmission frequency according to the network status, giving priority to transmitting key action data such as the peak acceleration at the moment of throwing; Calculate throwing parameters based on motion data: Direction angle: Calculate the angle between the throwing direction and the horizontal plane through the motion trajectory of key points of the hand, such as the vector from the wrist to the fingertips; Initial velocity: Analyze the hand acceleration curve and calculate the linear velocity at the moment of release based on the arm length; Rotational angular velocity: Detects the wrist rotation trajectory and calculates the dart's spin speed through the quaternion change rate; S3. Physical simulation and trajectory calculation: User action parameters include direction angle α, initial velocity V0, and rotation angular velocity ω; Environmental parameters include gravity g, air density p, and wind speed v; Trajectory equation calculation: Among them, the first term is the parabolic basic trajectory, the second term is the influence of gravity, and the third term is the Magnus effect offset caused by rotation; And air resistance and Magnus force: Among them, C d =0.42 is the air resistance coefficient, S m is the Magnus coefficient; S4. Digital twin scene rendering and interaction: Real venue twin: Using panoramic holographic oblique photography and BIM technology to build a venue model, the floor is given physical properties such as a friction coefficient of μ = 0.65 and acoustic reflection; Creative scene rendering: such as the planetary gravitational field in zero gravity, i.e., an interstellar shooting range, and time rifts with random ±50% flight speed variations, using the GAN network to generate high-precision textures and lighting effects; Based on the user's selected settings, the dart flight trajectory and hit effects are rendered in real time, and the ambient sound and hit sound effects are played dynamically according to the scene.

3. A method according to claim 2, characterized in that: In step S1, the camera is a TOF depth camera with a resolution of 640×480, a frame rate of 30 Hz, a depth accuracy of ±1 cm, and a field of view of 75°×65°; The client is responsible for image acquisition and rendering, and the server is responsible for data processing and storage; Human body key point detection uses the MediaPipe motion capture recognition framework to identify 21 hand + 33 body key points.

4. A method according to claim 2, characterized in that: In step S1, the camera detects the current light intensity and automatically adjusts the camera exposure parameters; if the ambient light is insufficient, a fill light prompt is triggered, and the client can connect to the smart home system via WIFI, and adjust the lighting parameters of the smart home lighting based on the adjustable parameters in the smart home lighting and the optimal working parameters of the camera.

5. A method according to claim 2, characterized in that: In step S2, the specific process of the neural network processor accelerating model reasoning is as follows: Use the MediaPipe framework to train a 21-keypoint hand and 33-keypoint body detection model, and implement keypoint recognition based on convolutional neural networks or lightweight models such as MobileNet; By using quantization technologies supported by the NPU, such as 8-bit fixed-point quantization, floating-point model parameters are converted to integers, reducing model size by 40% and memory usage by 35%. The principle is to reduce the computational complexity by reducing the number of storage bits for each parameter while retaining the accuracy of key features through calibration data; Deploy the quantized model to a device-side neural network processor, such as an NPU chip on a mobile phone or edge device, and use its dedicated hardware units, such as the tensor processing unit (TPU), to accelerate matrix operations. The NPU accelerates operations such as convolution and pooling through multi-core parallel computing, increasing the full-process inference speed from 8fps before optimization to 22fps.

6. A method according to claim 2, characterized in that: In step S3, the coordinates of the intersection of the dart and the target surface are solved according to the trajectory equation to determine whether the dart hits the double area, triple area or bullseye; If the dart hits a hard material area, the rebound effect is triggered and the rebound speed is calculated based on the elastic parameter E = 2000MPa. If the dart hits a soft material target surface, the dart is inserted and stops, and the scoreboard is updated.

7. A method according to claim 2, characterized in that: The method further comprises the steps of: S5. Multi-person interaction and social functions: Room creation: Users create a room through the social module, supporting 2-8 people to join and select the competition format; Data synchronization: The server synchronizes the action data of each client through the WebRTC protocol, with a delay of ≤200ms, ensuring that the throwing trajectories and scores of multiple users are consistent in real time; Team mode: 3 players form a team, divided into sniper position (high-precision throwing), defense-breaking position (high-force penetration), and healing position (assisting with target repair), and cooperate in combat through an energy-sharing mechanism; Spectator system: supports switching between third-person perspective and bull's-eye perspective. Spectators can send emojis and comments. The system automatically captures exciting moments such as slam throws and generates replay videos.

8. A method according to claim 7, characterized in that: The method further comprises the steps of: S6. User-generated content and system extension: Users upload photos of real-world venues and automatically generate 3D twin scenes through AI algorithms; customize physical parameters such as gravity, wind speed, and obstacle layouts such as moving targets and shield walls; Users can use the rule editor to design custom competition systems such as time limit + random environment changes, and scoring rules such as hitting a specific area to trigger a chain explosion score; Users can upload their own scenarios and rules to the community, and other users can download and experience them directly; Creators will be given virtual rewards such as exclusive equipment, skins, and experience bonuses through likes and download rankings, and high-quality content can be selected into the official recommendation pool.

9. A method according to claim 8, characterized in that: The method further comprises the steps of: S7. Virtual-Real Integration and Commercial Applications: If a pressure-sensitive floor is used, it is placed within the camera's capture range. The user stands on the pressure-sensitive floor to operate. The pressure-sensitive floor works in conjunction with the TOF camera to detect whether the user's standing posture is in compliance. Equipped with sensors for physical darts, the virtual trajectory calibration, such as the physical throwing angle, is synchronized with the virtual scene; The results of offline store competitions are synchronized to the online leaderboard, and users can participate in dart activities and unlock prizes through LBS technology; The system uses virtual scenes to demonstrate the principles of parabola, such as fixing the initial velocity and adjusting the angle to observe the trajectory changes, for physics teaching; A low-intensity training mode is designed for the elderly or patients with upper limb injuries. The joint range of motion is analyzed through motion capture and rehabilitation reports are generated, so that the system can be applied to rehabilitation training.

10. A method and method according to claim 9, characterized in that: The method further comprises the steps of: S8. System closed loop and data management: The user character is equipped with a growth system and a data center. The growth system gains experience through competition, which can unlock the user character's skills. The data center stores data such as user action history, win rate, and preferred scenarios, which are used for AI analysis and optimization of matching algorithms, such as recommending similar players for battle. If the system detects that the camera position is offset, it triggers an automatic calibration process and prompts the user to stand again in the designated area. The server records indicators such as data transmission delay and frame rate fluctuation in real time, and automatically issues an alarm and switches to the backup node in the event of an abnormality.

Images