Swimming teaching auxiliary system and method based on mixed reality swimming wearable device

Abstract

The embodiment of the invention provides a swimming teaching assistance system and method based on mixed reality swimming wearable equipment, and is applied to the technical field of mixed reality teaching assistance. On the basis of data detected by a gyroscope of the mixed reality head-mounted display equipment and data detected by gyroscopes in the bracelet, the foot ring and the swimming trunks, posture action information of the target user is determined by combining the height, the weight and the sports skill data of the target user; and based on the posture action information of the target user, comparing with a preset action model, and based on a comparison result, generating a corresponding prompt animation and prompt sound in combination with the height, weight and sports skill data of the target user. Based on single gyroscope data and in combination with a corresponding detection algorithm, accurate swimming teaching assistance prompt data can be generated according to various body and skill data of the user, so that swimming teaching assistance can be accurately, concisely and efficiently performed.

Description

Technical Field

[0001] This disclosure relates to the field of mixed reality teaching aids technology, and in particular to a swimming teaching aid system and method based on mixed reality swimming wearable devices. Background Technology

[0002] With the popularization of fitness for all, swimming, as a basic sport and survival skill, is receiving increasing attention from parents.

[0003] However, swimming is a complex sport that requires a high degree of coordination of the human body. When learning to swim, people often repeat the movements of their arms and legs to form muscle memory in order to achieve accurate coordination of the body.

[0004] Existing swimming assistance solutions require various types of sensors to collect complex data, and each user has their own unique characteristics. This makes it impossible for existing teaching assistance solutions to provide concise and targeted instruction based on each user's specific data, resulting in complex, untargeted, and inaccurate swimming teaching assistance solutions.

[0005] Therefore, existing swimming teaching aids are complex, lack specificity, and are not accurate enough. Summary of the Invention

[0006] This disclosure provides a swimming teaching assistance system, method, device, and storage medium based on a mixed reality wearable swimming device.

[0007] According to a first aspect of this disclosure, a swimming teaching assistance system based on a mixed reality swimming wearable device is provided. The system includes a mixed reality head-mounted display device, a pair of wristbands, a pair of ankle bands, swimming trunks, and a server corresponding to the mixed reality head-mounted display device.

[0008] The server establishes communication connections with the wristband, ankle bracelet, and swim trunks through the mixed reality head-mounted display device to obtain angle data detected by the gyroscope of the mixed reality head-mounted display device, as well as angle data detected by the gyroscopes in the wristband, ankle bracelet, and swim trunks. Based on the angle data detected by the gyroscopes of the mixed reality head-mounted display device and the wristband, ankle bracelet, and swim trunks, the server combines the angle data detected by the gyroscopes of the mixed reality head-mounted display device and the gyroscopes in the wristband, ankle bracelet, and swim trunks with the target user's height, weight, and motor skill data to determine the target user's posture and movement information.

[0009] The server is also used to compare the target user's posture and movement information with a preset motion model. Based on the comparison results, combined with the target user's height, weight, and motor skill data, the server generates corresponding prompts and sends them to the mixed reality head-mounted display device. The mixed reality head-mounted display device then generates corresponding prompt animations and sounds based on the prompts.

[0010] In some implementations of the first aspect, the server determines the target user's posture and movement information based on the angle data detected by the gyroscope of the mixed reality head-mounted display device and the angle data detected by the gyroscopes in the wristband, ankle bracelet, and swim trunks, combined with the target user's height, weight, and motor skill data, including:

[0011] The server generates a preliminary human motion model based on the angle data detected by the gyroscope of the mixed reality head-mounted display device, as well as the angle data detected by the gyroscopes in the wristband, ankle band and swim trunks.

[0012] Based on the target user's height, weight, and motor skill data, combined with the initial movements and speed of the human body in the preliminary human motion model, the server determines the target user's arm and leg movements and changes in body orientation.

[0013] In some implementations of the first aspect, the angle data detected by the gyroscope of the aforementioned mixed reality head-mounted display device is G. head =(θ head ,ω head ,α head ), where θ head The head posture angle, ω head Let α be the angular velocity. head Angular acceleration;

[0014] The gyroscope-detected angle data from the wristband, ankle strap, and swim trunks include the angle data G of the left wristband. hand_L =(θ hand_L ,ω hand_L ,α hand_L ), right-hand ring angle data

[0015] G hand_R =(θ hand_R ,ω hand_R ,α hand_R ), left ankle ring angle data G foot_L =(θ foot_L ,ω foot_L ,α foot_L ), the angle data of the right ankle ring G foot_R =(θ foot_R ,ω foot_R ,α foot_R ), swim trunks angle data

[0016] G waist =(θ waist ,ω waist ,α waist ), used to characterize trunk posture, where θ hand_L θ hand_R θ foot_L θ foot_R and θ waist The posture angles ω represent the left hand, right hand, left foot, right foot, and hip, respectively. hand_L ω hand_R ω foot_L ω foot_R and ω waist The angular velocities α are for the left hand, right hand, left foot, right foot, and hip, respectively. hand_L α hand_R α foot_L α foot_R and α waist The angular accelerations are for the left hand, right hand, left foot, right foot, and hip, respectively.

[0017] Based on angle data detected by the gyroscope of the mixed reality head-mounted display device, as well as angle data detected by the gyroscopes in the wristband, ankle strap, and swim trunks, the server generates a preliminary human motion model, including:

[0018] Calculate the relative motion parameters of each limb, including the arm swing angle Δθ. arm =∣θ hand_L -θ hand_R |, Leg swing angle Δθ leg =∣θ foot_L -θ foot_R |, Postural deviation of the buttocks and head Δθ torso =∣θ waist -θ head |;

[0019] Based on a weighted fusion algorithm, the overall motion trend parameters of the human body are calculated. Where w1, w2, w3, and w4 are preset weight coefficients, and satisfy w1 + w2 + w3 + w4 = 1;

[0020] Calculate the motor coordination of each limb

[0021] Where ωi is the angular velocity of each device. Let N be the average angular velocity of each device, and N be the number of devices.

[0022] Based on the motion trend parameter M and the motion coordination degree C, a preliminary model of human movement is determined.

[0023] In some implementations of the first aspect, the target user's height is H, weight is W, and motor skill level is S, where the motor skill level S is divided into levels 1 to N, and N is a positive integer greater than 1; the preliminary action output by the preliminary human motion model is A0 and the motion speed is V, where the preliminary action A0 includes the trunk baseline posture and the initial position of the limbs.

[0024] Based on the target user's height, weight, and motor skill data, combined with the initial movements and speed of the human body in the preliminary human motion model, the server determines the target user's arm and leg movements and changes in body orientation, including:

[0025] Based on the target user's height H, weight W, and motor skill level S, calculate the limb length adjustment coefficient α and strength parameter β:

[0026] α=(H / H0)×(W0 / W) (1 / 3) ,

[0027] β=(W / W0)×(S / S0),

[0028] Among them, H0, W0, and S0 are the benchmark height, benchmark weight, and benchmark skill level, respectively;

[0029] Calculate the dynamic correction factor γ based on the motion velocity V and force parameter β:

[0030] γ=1+k1×ln(V / V0)+k2×(β-1),

[0031] Where V0 is the reference speed, and k1 and k2 are preset proportional coefficients;

[0032] Based on the trunk baseline posture in the initial movement A0, and combined with the limb length adjustment coefficient α and the dynamic correction factor γ, the target user's arm movement A_arm and leg movement A_leg are determined:

[0033] A_arm=A0_arm×α×(1+γ×δ_arm),

[0034] A_leg=A0_leg×α×(1+γ×δ_leg),

[0035] Where A0_arm and A0_leg are the baseline arm and leg movements in the initial movement, respectively, and δ_arm and δ_leg are the limb type weight coefficients;

[0036] The rate of change of human orientation ω is calculated based on the ratio of movement speed V to height H:

[0037] ω=ω0×(V / H)×(S / S0) m ,

[0038] Where ω0 is the baseline steering rate and m is the skill influence index.

[0039] In some implementations of the first aspect, the above-mentioned comparison of the target user's posture and movement information with a preset movement model, and based on the comparison results, combined with the target user's height, weight, and motor skill data, generates corresponding prompt information, including:

[0040] The posture and motion information is compared with a pre-set standard motion model library to calculate the motion matching degree, satisfying the formula Δθ=∣θ. 姿态动作 -θ 标准动作 |, where Δθ is the action matching degree, θ 姿态动作 For posture and motion information, θ 标准动作 This refers to the standard motion information corresponding to the preset standard motion model library;

[0041] Based on the degree of action matching, determine the prompt data and prompt level;

[0042] Based on the prompt data and prompt level, corresponding prompt information is generated, which is then used by the mixed reality head-mounted display device to generate corresponding prompt animations and prompt sounds, so that the user can practice swimming movements according to the prompt animations and prompt sounds.

[0043] In some implementations of the first aspect, the above-mentioned calculation of action matching degree includes:

[0044] The deviations of each posture parameter from the standard motion parameters in the standard motion model library are weighted and summed to satisfy the formula Δθ. (总) =w1Δθ (腿部) +w2Δθ (手臂) +w3Δθ (头部) +w4Δθ (躯干) , where Δθ (总) The parameter is the weighted sum of all attitude parameters, Δθ( 腿部 The deviation of the leg posture parameters from the standard motion parameters in the standard motion model library is represented by w1, and the corresponding weight is Δθ. (手臂) The deviation of the arm's posture parameters from the standard motion parameters in the standard motion model library is represented by the weight w2, Δθ. (头部) The deviation of the head's pose parameters from the standard motion parameters in the standard motion model library is represented by the weight w3, Δθ. (躯干) The deviation between the posture parameters of the torso and the standard motion parameters in the standard motion model library is represented by a weight of w4.

[0045] The weighted summation result is adjusted based on the user's motor skill level to obtain the motion matching degree.

[0046] According to a second aspect of this disclosure, a swimming instruction assistance method based on a mixed reality wearable device is provided, applied to a server corresponding to a mixed reality head-mounted display device, the method comprising:

[0047] Acquire angle data detected by the gyroscope of a mixed reality head-mounted display device, and acquire angle data detected by the gyroscope in wristbands, ankle bracelets, and swim trunks through the mixed reality head-mounted display device;

[0048] Based on the angle data detected by the gyroscope of the mixed reality head-mounted display device, as well as the angle data detected by the gyroscope in the wristband, ankle band and swimming trunks, the target user's posture and movement information is determined by combining the target user's height, weight and motor skill data.

[0049] Based on the target user's posture and movement information, it is compared with a preset movement model. Based on the comparison results, combined with the target user's height, weight, and motor skill data, corresponding prompt information is generated and sent to the mixed reality head-mounted display device. The mixed reality head-mounted display device then generates corresponding prompt animations and sounds based on the prompt information.

[0050] According to a third aspect of this disclosure, an electronic device is provided, comprising: a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the program to implement the method as described in the second aspect of this disclosure.

[0051] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method of the second aspect.

[0052] This disclosure utilizes data detected by a mixed reality head-mounted display, a pair of wristbands, a pair of ankle bracelets, and a gyroscope in the swim trunks, combined with the target user's height, weight, and motor skill data, to provide swimming instruction assistance. A corresponding detection algorithm has been designed to generate accurate swimming instruction assistance prompts based on the user's various physical and skill data, thereby providing accurate swimming instruction assistance.

[0053] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0054] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of this disclosure. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0055] Figure 1 A schematic diagram of the structure of a swimming teaching assistance system based on a mixed reality wearable device according to an embodiment of the present disclosure is shown;

[0056] Figure 2 A flowchart illustrating a swimming instruction aid method based on a mixed reality wearable swimming device according to an embodiment of the present disclosure is shown.

[0057] Figure 3 A block diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0059] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0060] In order to solve the above-mentioned technical problems, this disclosure uses data detected by a mixed reality head-mounted display device, a pair of wristbands, a pair of ankle bracelets, and a gyroscope in the swim trunks, combined with the target user's height, weight, and motor skill data, to provide swimming instruction assistance. A corresponding detection algorithm is designed to generate accurate swimming instruction assistance prompts based on the user's various physical and skill data, thereby providing accurate swimming instruction assistance.

[0061] Figure 1 A schematic diagram of a swimming teaching assistance system based on a mixed reality wearable device, according to an embodiment of this disclosure, is shown. Figure 1 As shown, the swimming instruction assistance system 100 based on mixed reality wearable swimming devices may include:

[0062] Mixed reality head-mounted display device 101, a pair of wristbands 102, a pair of ankle bracelets 103, swimming trunks 104, and a server 105 corresponding to the mixed reality head-mounted display device;

[0063] The server establishes communication connections with the wristband, ankle bracelet, and swim trunks through the mixed reality head-mounted display device to obtain angle data detected by the gyroscope of the mixed reality head-mounted display device, as well as angle data detected by the gyroscopes in the wristband, ankle bracelet, and swim trunks. Based on the angle data detected by the gyroscopes of the mixed reality head-mounted display device and the wristband, ankle bracelet, and swim trunks, the server combines the angle data detected by the gyroscopes of the mixed reality head-mounted display device and the gyroscopes in the wristband, ankle bracelet, and swim trunks with the target user's height, weight, and motor skill data to determine the target user's posture and movement information.

[0064] The server is also used to compare the target user's posture and movement information with a preset motion model. Based on the comparison results, combined with the target user's height, weight, and motor skill data, the server generates corresponding prompts and sends them to the mixed reality head-mounted display device. The mixed reality head-mounted display device then generates corresponding prompt animations and sounds based on the prompts.

[0065] It should be noted that the swimming trunks 104 mentioned above include a gyroscope and a wireless data transmission module to realize the detection of angle data and the transmission of the detection data.

[0066] In some embodiments, the server determines the target user's posture and movement information based on angle data detected by the gyroscope of the mixed reality head-mounted display device and angle data detected by the gyroscopes in the wristband, ankle bracelet, and swim trunks, combined with the target user's height, weight, and motor skill data, including:

[0067] The server generates a preliminary human motion model based on the angle data detected by the gyroscope of the mixed reality head-mounted display device, as well as the angle data detected by the gyroscopes in the wristband, ankle band and swim trunks.

[0068] Based on the target user's height, weight, and motor skill data, combined with the initial movements and speed of the human body in the preliminary human motion model, the server determines the target user's arm and leg movements and changes in body orientation.

[0069] It should be further explained that the aforementioned wristband includes a gyroscope and a wireless data transmission module. This wireless data transmission module is used to send the angle data detected by the gyroscope to the server for the aforementioned calculations. Similarly, the aforementioned anklet includes a gyroscope and a wireless data transmission module, which is used to send the angle data detected by the gyroscope to the server for the aforementioned calculations. The aforementioned swim trunks also include a gyroscope and a wireless data transmission module, which is used to send the angle data detected by the gyroscope to the server for the aforementioned calculations. In other words, the gyroscopes of each device collect relevant data and transmit it to the server via the wireless data transmission module.

[0070] In the above embodiments, based on the angle data detected by the gyroscope of the mixed reality head-mounted display device and the angle data detected by the gyroscope in the wristband, ankle band and swimming trunks, a preliminary human motion model is first generated. Then, based on the preliminary human motion model, the movements of the target user's arms and legs and the changes in the orientation of the human body can be accurately determined, thereby enabling accurate swimming teaching assistance in the future.

[0071] In some embodiments, the angle data detected by the gyroscope of the mixed reality head-mounted display is G. head =(θ head ,ω head ,α head ), where θ head The head posture angle, ω head Let α be the angular velocity. head Angular acceleration;

[0072] The gyroscope-detected angle data from the wristband, ankle strap, and swim trunks include the angle data G of the left wristband. hand_L =(θ hand_L ,ω hand_L ,α hand_L ), right-hand ring angle data

[0073] G hand_R =(θ hand_R ,ω hand_R ,α hand_R ), left ankle ring angle data G foot_L =(θ foot_L ,ω foot_L ,α foot_L ), the angle data of the right ankle ring G foot_R =(θ foot_R ,ω foot_R ,α foot_R ), swim trunks angle data

[0074] G waist =(θ waist ,ω waist ,α waist ), used to characterize trunk posture, where θ hand_L θ hand_R θ foot_L θ foot_R and θ waist The posture angles ω represent the left hand, right hand, left foot, right foot, and hip, respectively. hand_L ω hand_R ω foot_L ω foot_R and ω waist The angular velocities α are for the left hand, right hand, left foot, right foot, and hip, respectively. hand_L α hand_R αfoot_L α foot_R and α waist The angular accelerations are for the left hand, right hand, left foot, right foot, and hip, respectively.

[0075] Based on angle data detected by the gyroscope of the mixed reality head-mounted display device, as well as angle data detected by the gyroscopes in the wristband, ankle strap, and swim trunks, the server generates a preliminary human motion model, including:

[0076] Calculate the relative motion parameters of each limb, including the arm swing angle Δθ. arm =∣θ hand_L -θ hand_R |, Leg swing angle Δθ leg =∣θ foot_L -θ foot_R |, Postural deviation of the buttocks and head Δθ torso =∣θ waist -θ head |;

[0077] Based on a weighted fusion algorithm, the overall motion trend parameters of the human body are calculated. Where w1, w2, w3, and w4 are preset weight coefficients, and satisfy w1 + w2 + w3 + w4 = 1;

[0078] Calculate the motor coordination of each limb

[0079] Where ωi is the angular velocity of each device. Let N be the average angular velocity of each device, and N be the number of devices.

[0080] Based on the motion trend parameter M and the motion coordination degree C, a preliminary model of human movement is determined.

[0081] Specifically, in the process of determining the preliminary model of human movement, in the initial stage, the average angular velocity of the equipment is the initial value n0, and the angular velocity of the equipment is 0. At this time, according to the formula C=1;

[0082] After the motion begins, the angular velocity of the device is x1, and the difference is x1-n0. At this time, according to formula 0... <C<1.

[0083] After a period of time, the angular velocity of the equipment is x2, and the average angular velocity of the equipment is n1. The difference is x2-n1. At this time, according to the formula, C approaches 0.

[0084] The motion trend formula M reflects the amplitude of human body swaying; the larger M is, the greater the amplitude of human body swaying.

[0085] When the value of M is constant and C approaches 0, the movement is considered normal and the swimmer is swimming at a constant speed.

[0086] When the M value increases or decreases and C approaches 1, the action is considered normal, either accelerating or decelerating; otherwise, the action is considered abnormal.

[0087] In the above embodiments, based on the angle data detected by the gyroscope of the mixed reality head-mounted display device, as well as the angle data detected by the gyroscopes of the wristband, ankle band and swimming trunks, the relative motion parameters of each limb, the overall motion trend parameters of the human body, and the motion coordination of each limb are calculated to accurately determine the preliminary model of human movement, so as to accurately assist in swimming teaching in the future.

[0088] In some embodiments, the target user's height is H, weight is W, and motor skill level is S, where the motor skill level S is divided into levels 1 to N, and N is a positive integer greater than 1; the preliminary action output by the preliminary human motion model is A0 and the movement speed is V, where the preliminary action A0 includes the trunk baseline posture and the initial position of the limbs; specifically, the movement speed V is directly proportional to the height H and inversely proportional to the weight W, so V = V0 * H / W.

[0089] Based on the target user's height, weight, and motor skill data, combined with the initial movements and speed of the human body in the preliminary human motion model, the server determines the target user's arm and leg movements and changes in body orientation, including:

[0090] Based on the target user's height H, weight W, and motor skill level S, calculate the limb length adjustment coefficient α and strength parameter β:

[0091] α=(H / H0)×(W0 / W) (1 / 3) ,

[0092] β=(W / W0)×(S / S0),

[0093] Among them, H0, W0, and S0 are the benchmark height, benchmark weight, and benchmark skill level, respectively;

[0094] Calculate the dynamic correction factor γ based on the motion velocity V and force parameter β:

[0095] γ=1+k1×ln(V / V0)+k2×(β-1),

[0096] Where V0 is the reference speed, and k1 and k2 are preset proportional coefficients;

[0097] Based on the trunk baseline posture in the initial movement A0, and combined with the limb length adjustment coefficient α and the dynamic correction factor γ, the target user's arm movement A_arm and leg movement A_leg are determined:

[0098] A_arm=A0_arm×α×(1+γ×δ_arm),

[0099] A_leg=A0_leg×α×(1+γ×δ_leg),

[0100] Where A0_arm and A0_leg are the baseline arm and leg movements in the initial movement, respectively, and δ_arm and δ_leg are the limb type weight coefficients;

[0101] The rate of change of human orientation ω is calculated based on the ratio of movement speed V to height H:

[0102] ω=ω0×(V / H)×(S / S0) m ,

[0103] Where ω0 is the baseline steering rate and m is the skill influence index.

[0104] In the above embodiment, based on the target user's height H, weight W, and motor skill level S, a limb length adjustment coefficient α and a strength parameter β are calculated. Based on the movement speed V and the strength parameter β, a dynamic correction factor γ is calculated. Based on the trunk baseline posture in the initial movement A0, combined with the limb length adjustment coefficient α and the dynamic correction factor γ, the target user's arm movement A_arm and leg movement A_leg are determined. The rate of change of human orientation ω is calculated based on the ratio of movement speed V to height H to accurately determine various parameters, thereby enabling accurate swimming teaching assistance in the future.

[0105] In some embodiments, based on the target user's posture and movement information, a comparison is made with a preset movement model. Based on the comparison results, and combined with the target user's height, weight, and motor skill data, corresponding prompt information is generated, including:

[0106] The posture and motion information is compared with a pre-set standard motion model library to calculate the motion matching degree, satisfying the formula Δθ=∣θ. 姿态动作 -θ 标准动作 |, where Δθ is the action matching degree, θ 姿态动作 For posture and motion information, θ 标准动作 This refers to the standard motion information corresponding to the preset standard motion model library;

[0107] Based on the degree of motion matching, determine the prompt data and prompt level; where the prompt level should be S set before the exercise, and the prompt data is obtained by multiplying the coefficient by the baseline motion data.

[0108] Based on the prompt data and prompt level, corresponding prompt information is generated for the mixed reality head-mounted display device to generate corresponding prompt animations and prompt sounds, so that the user can practice swimming movements according to the prompt animations and prompt sounds. The prompt animations and prompt sounds are motion correction animations and voice instructions to help the user practice swimming movements.

[0109] Specifically, first, the screen displays an animation of the action to be performed. After the user completes the action, the data is uploaded to the server. If the data does not meet the requirements after comparison, the action needs to be performed again. After completion, the next action is performed.

[0110] In some embodiments, calculating the action matching degree includes:

[0111] The deviations of each posture parameter from the standard motion parameters in the standard motion model library are weighted and summed to satisfy the formula Δθ. (总) =w1Δθ( 腿部 )+w2Δθ (手臂) +w3Δθ (头部) +w4Δθ (躯干) , where Δθ (总) The parameter is the weighted sum of all attitude parameters, Δθ( 腿部 The deviation of the leg posture parameters from the standard motion parameters in the standard motion model library is represented by w1, and the corresponding weight is Δθ. (手臂) The deviation of the arm's posture parameters from the standard motion parameters in the standard motion model library is represented by the weight w2, Δθ. (头部) The deviation of the head's pose parameters from the standard motion parameters in the standard motion model library is represented by the weight w3, Δθ. (躯干) The deviation between the posture parameters of the torso and the standard motion parameters in the standard motion model library is represented by a weight of w4.

[0112] The weighted sum above is adjusted based on the user's motor skill level to obtain the motion matching degree. The user's motor skill level coefficient is S; the higher the motor skill level, the closer S is to 1, and the lower the motor skill level, the closer S is to 0. The adjustment method described above is Δθ. (总) *S, the closer S is to 0, the greater the Δθ (总) The smaller the S value, the higher the action match. Conversely, the closer S is to 1, the better the action match.

[0113] Δθ (总) The larger the S value, the worse the action matching. Therefore, Δθ is needed first. (总) The smaller the better, secondly, when Δθ (总) Once determined, users need to adjust their skill actions to achieve the highest possible accuracy in matching the actions.

[0114] This disclosure utilizes data detected by a mixed reality head-mounted display, a pair of wristbands, a pair of ankle bracelets, and a gyroscope in the swim trunks, combined with the target user's height, weight, and motor skill data, to provide swimming instruction assistance. A corresponding detection algorithm has been designed to generate accurate swimming instruction assistance prompts based on the user's various physical and skill data, thereby providing accurate swimming instruction assistance.

[0115] The above is an introduction to the system embodiments. The following method embodiments will further illustrate the present disclosure.

[0116] Figure 2 A flowchart illustrating a swimming instruction aid method based on a mixed reality swimming wearable device according to an embodiment of the present disclosure is shown. The method is applied to a server corresponding to a mixed reality head-mounted display device.

[0117] like Figure 2 As shown, the swimming instruction aid method 200 based on mixed reality wearable swimming devices may include:

[0118] S201, acquire angle data detected by the gyroscope of the mixed reality head-mounted display device, and acquire angle data detected by the gyroscopes in the wristband, ankleband and swim trunks through the mixed reality head-mounted display device;

[0119] S202, Based on the angle data detected by the gyroscope of the mixed reality head-mounted display device and the angle data detected by the gyroscope in the wristband, ankle band and swimming trunks, combined with the target user's height, weight and motor skill data, determine the target user's posture and movement information;

[0120] S203: Based on the target user's posture and movement information, compare it with a preset action model. Based on the comparison results, combined with the target user's height, weight, and motor skill data, generate corresponding prompt information and send it to the mixed reality head-mounted display device. The mixed reality head-mounted display device then generates corresponding prompt animations and sounds based on the prompt information.

[0121] In some embodiments, the server determines the target user's posture and movement information based on angle data detected by the gyroscope of the mixed reality head-mounted display device and angle data detected by the gyroscopes in the wristband, ankle bracelet, and swim trunks, combined with the target user's height, weight, and motor skill data, including:

[0122] The server generates a preliminary human motion model based on the angle data detected by the gyroscope of the mixed reality head-mounted display device, as well as the angle data detected by the gyroscopes in the wristband, ankle band and swim trunks.

[0123] Based on the target user's height, weight, and motor skill data, combined with the initial movements and speed of the human body in the preliminary human motion model, the server determines the target user's arm and leg movements and changes in body orientation.

[0124] In some embodiments, the angle data detected by the gyroscope of the mixed reality head-mounted display device described above is G. head =(θ head ,ω head ,α head ), where θhead The head posture angle, ω head Let α be the angular velocity. head Angular acceleration;

[0125] The gyroscope-detected angle data from the wristband, ankle strap, and swim trunks include the angle data G of the left wristband. hand_L =(θ hand_L ,ω hand_L ,α hand_L ), right-hand ring angle data

[0126] G hand_R =(θ hand_R ,ω hand_R ,α hand_R ), left ankle ring angle data G foot_L =(θ foot_L ,ω foot_L ,α foot_L ), the angle data of the right ankle ring G foot_R =(θ foot_R ,ω foot_R ,α foot_R ), swim trunks angle data

[0127] G waist =(θ waist ,ω waist ,α waist ), used to characterize trunk posture, where θ hand_L θ hand_R θ foot_L θ foot_R and θ waist The posture angles ω represent the left hand, right hand, left foot, right foot, and hip, respectively. hand_L ω hand_R ω foot_L ω foot_R and ω waist The angular velocities α are for the left hand, right hand, left foot, right foot, and hip, respectively. hand_L α hand_R α foot_L α foot_R and α waist The angular accelerations are for the left hand, right hand, left foot, right foot, and hip, respectively.

[0128] Based on angle data detected by the gyroscope of the mixed reality head-mounted display device, as well as angle data detected by the gyroscopes in the wristband, ankle strap, and swim trunks, the server generates a preliminary human motion model, including:

[0129] Calculate the relative motion parameters of each limb, including the arm swing angle Δθ. arm =∣θ hand_L -θ hand_R|, Leg swing angle Δθ leg =∣θ foot_L -θ foot_R |, Postural deviation of the buttocks and head Δθ torso =∣θ waist -θ head |;

[0130] Based on a weighted fusion algorithm, the overall motion trend parameters of the human body are calculated. Where w1, w2, w3, and w4 are preset weight coefficients, and satisfy w1 + w2 + w3 + w4 = 1;

[0131] Calculate the motor coordination of each limb

[0132] Where ωi is the angular velocity of each device. Let N be the average angular velocity of each device, and N be the number of devices.

[0133] Based on the motion trend parameter M and the motion coordination degree C, a preliminary model of human movement is determined.

[0134] In some embodiments, the target user's height is H, weight is W, and motor skill level is S, where the motor skill level S is divided into 1 to N levels, and N is a positive integer greater than 1; the preliminary action output by the preliminary human motion model is A0 and the motion speed is V, where the preliminary action A0 includes the trunk baseline posture and the initial position of the limbs.

[0135] Based on the target user's height, weight, and motor skill data, combined with the initial movements and speed of the human body in the preliminary human motion model, the server determines the target user's arm and leg movements and changes in body orientation, including:

[0136] Based on the target user's height H, weight W, and motor skill level S, calculate the limb length adjustment coefficient α and strength parameter β:

[0137] α=(H / H0)×(W0 / W) (1 / 3) ,

[0138] β=(W / W0)×(S / S0),

[0139] Among them, H0, W0, and S0 are the benchmark height, benchmark weight, and benchmark skill level, respectively;

[0140] Calculate the dynamic correction factor γ based on the motion velocity V and force parameter β:

[0141] γ=1+k1×ln(V / V0)+k2×(β-1),

[0142] Where V0 is the reference speed, and k1 and k2 are preset proportional coefficients;

[0143] Based on the trunk baseline posture in the initial movement A0, and combined with the limb length adjustment coefficient α and the dynamic correction factor γ, the target user's arm movement A_arm and leg movement A_leg are determined:

[0144] A_arm=A0_arm×α×(1+γ×δ_arm),

[0145] A_leg=A0_leg×α×(1+γ×δ_leg),

[0146] Where A0_arm and A0_leg are the baseline arm and leg movements in the initial movement, respectively, and δ_arm and δ_leg are the limb type weight coefficients;

[0147] The rate of change of human orientation ω is calculated based on the ratio of movement speed V to height H:

[0148] ω=ω0×(V / H)×(S / S0) m ,

[0149] Where ω0 is the baseline steering rate and m is the skill influence index.

[0150] In some embodiments, the above-mentioned comparison of the target user's posture and movement information with a preset movement model, and the generation of corresponding prompt information based on the comparison results, combined with the target user's height, weight, and motor skill data, including:

[0151] The posture and motion information is compared with a preset standard motion model library to calculate the motion matching degree;

[0152] Based on the degree of action matching, determine the prompt data and prompt level;

[0153] The corresponding prompt message is generated based on the prompt data and the prompt level.

[0154] In some embodiments, the calculation of action matching degree includes:

[0155] The deviations of each posture parameter from the standard motion parameters in the standard motion model library are calculated by weighted summation;

[0156] The weighted summation result is adjusted based on the user's motor skill level to obtain the motion matching degree.

[0157] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0158] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0159] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0160] Figure 3 A block diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0161] Device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 302 or a computer program loaded from storage unit 308 into random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0162] Multiple components in device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of monitors, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0163] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as method 100. For example, in some embodiments, method 100 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of method 100 described above may be performed.

[0164] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0165] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0166] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0167] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0168] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0169] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0170] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0171] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A swimming teaching assistance system based on a mixed reality swimming wear device, characterized by, The system comprises a mixed reality head-mounted display device, a pair of hand rings, a pair of foot rings, swim trunks and a server corresponding to the mixed reality head-mounted display device; The server establishes a communication connection with the hand rings, the foot rings and the swim trunks through the mixed reality head-mounted display device, acquires angle data detected by a gyroscope of the mixed reality head-mounted display device and angle data detected by a gyroscope of the hand rings, the foot rings and the swim trunks through the mixed reality head-mounted display device, and determines posture action information of a target user based on the angle data detected by the gyroscope of the mixed reality head-mounted display device and the angle data detected by the gyroscope of the hand rings, the foot rings and the swim trunks in combination with height, weight and sports skill data of the target user. The server is further configured to compare the posture action information of the target user with a preset action model, generate corresponding prompt information based on a comparison result in combination with the height, weight and sports skill data of the target user, and send the prompt information to the mixed reality head-mounted display device, so that the mixed reality head-mounted display device generates corresponding prompt animation and prompt sound based on the prompt information.

2. The system of claim 1, wherein, The server determines the posture action information of the target user based on the angle data detected by the gyroscope of the mixed reality head-mounted display device and the angle data detected by the gyroscope of the hand rings, the foot rings and the swim trunks in combination with the height, weight and sports skill data of the target user, and comprises: The server generates a preliminary human action model based on the angle data detected by the gyroscope of the mixed reality head-mounted display device and the angle data detected by the gyroscope of the hand rings, the foot rings and the swim trunks. The server determines the action of the arms and legs and the change of the orientation of the human body of the target user based on the preliminary action and the speed of the movement of the human body in the preliminary human action model in combination with the height, weight and sports skill data of the target user.

3. The system of claim 2, wherein, The angle data detected by the gyroscope of the mixed reality head-mounted display device is G head =(θ head ,ω head ,α head ), wherein θ head is a head posture angle, ω head is a head angular velocity, and α head is a head angular acceleration The angle data detected by the gyroscope of the bracelet, the anklet and the swimwear respectively includes left bracelet angle data G hand_L =(θ hand_L ,ω hand_L ,α hand_L ), right bracelet angle data G hand_R =(θ hand_R ,ω hand_R ,α hand_R ), left ankle ring angle data G foot_L =(θ foot_L ,ω foot_L ,α foot_L ), the angle data of the right ankle ring G foot_R =(θ foot_R ,ω foot_R ,α foot_R ), swim trunks angle data G waist =(θ waist ,ω waist ,α waist ), used to characterize trunk posture, where θ hand_L θ hand_R θ foot_L θ foot_R and θ waist The posture angles ω represent the left hand, right hand, left foot, right foot, and hip, respectively. hand_L ω hand_R ω foot_L ω foot_R and ω waist The angular velocities α are for the left hand, right hand, left foot, right foot, and hip, respectively. hand_L α hand_R α foot_L α foot_R and α waist The angular accelerations are for the left hand, right hand, left foot, right foot, and hip, respectively. The server generates a preliminary human action model based on the angle data detected by the gyroscope of the mixed reality head-mounted display device and the angle data detected by the gyroscope of the hand rings, the foot rings and the swim trunks, and comprises: The relative movement parameters of each limb are calculated, wherein the arm swing angle Δθ arm = | θ hand_L - θ hand_R |, leg swing angle Δθ leg = | θ foot_L - θ foot_R |, posture deviation Δθ of the hip and the head torso = | θ waist - θ head | ; Calculate the human whole body motion trend parameter based on a weighted fusion algorithm wherein w1, w2, w3, w4 are preset weight coefficients, and satisfy w1+w2+w3+w4=1; Computing the motor coordination of each limb where ωi is the angular velocity of each device, is the average angular velocity of each device, and N is the number of devices. The preliminary human action model is determined according to the movement trend parameter M and the movement coordination degree C.

4. The system of claim 2, wherein, The height of the target user is H, the weight is W and the sports skill level is S, wherein the sports skill level S is divided into 1 to N levels, and N is a positive integer greater than 1; the preliminary action A0 and the movement speed V output by the preliminary human action model, wherein the preliminary action A0 comprises a trunk reference posture and an initial position of a limb; The server determines the action of the arms and legs and the change of the orientation of the human body of the target user based on the preliminary action and the speed of the movement of the human body in the preliminary human action model in combination with the height, weight and sports skill data of the target user, and comprises: The limb length adjustment coefficient α and the strength parameter β are calculated according to the height H, the weight W and the sports skill level S of the target user: a = (H / H0) x (W0 / W) (1 / 3) , β = (W / W0) × (S / S0), wherein H0, W0 and S0 are the reference height, the reference weight and the reference skill level, respectively. A dynamic correction factor γ is calculated based on the movement speed V and the strength parameter β: γ = 1 + k1 x ln(V / V0) + k2 x (β-1), wherein V0 is a reference speed, and k1 and k2 are preset proportionality coefficients; The arm action A_arm and the leg action A_leg of the target user are determined according to the trunk reference posture in the preliminary action A0, in combination with the limb length adjustment coefficient α and the dynamic correction factor γ: A_arm = A0_arm x α x (1 + γ x δ_arm), A_leg = A0_leg x α x (1 + γ x δ_leg), wherein A0_arm and A0_leg are the arm and leg reference actions in the preliminary action, and δ_arm and δ_leg are limb type weight coefficients. The human body orientation change rate ω is calculated according to the ratio of the movement speed V to the height H: ω = ω0x (V / H) x (S / S0) m , wherein ω0 is a reference turning rate, and m is a skill influence index.

5. The system of claim 1, wherein, The posture action information of the target user is compared with a preset action model, and corresponding prompt information is generated based on the comparison result, in combination with the height, weight and movement skill data of the target user, including: The posture action information is compared with a preset standard action model library, an action matching degree is calculated, and a formula Δθ = |θ 姿态动作 -θ 标准动作 | is satisfied, wherein Δθ is the action matching degree, θ 姿态动作 is the posture action information, and θ 标准动作 is corresponding standard action information in the preset standard action model library; prompt data and a prompt level are determined according to the action matching degree. The prompt data and the prompt level are used to generate corresponding prompt information, so that the mixed reality head-mounted display device generates corresponding prompt animations and prompt sounds based on the prompt information, so that the user practices swimming actions according to the prompt animations and the prompt sounds.

6. The system of claim 5, wherein, The action matching degree is calculated, including: The deviations of the posture parameters and the standard action parameters in the standard action model library are weighted and summed to satisfy the formula Δθ (总) = w1Δθ 腿部 + w2Δθ (手臂) + w3Δθ (头部) + w4Δθ (躯干) , wherein Δθ (总) is the weighted sum of the posture parameters, Δθ 腿部 is the deviation of the posture parameter of the leg from the standard action parameter in the standard action model library, and the corresponding weight is w1, Δθ (手臂) is the deviation of the posture parameter of the arm from the standard action parameter in the standard action model library, and the corresponding weight is w2, Δθ (头部) is the deviation of the posture parameter of the head from the standard action parameter in the standard action model library, and the corresponding weight is w3, and Δθ (躯干) is the deviation of the posture parameter of the torso from the standard action parameter in the standard action model library, and the corresponding weight is w4. The action matching degree is obtained by adjusting the above weighted sum result according to the user movement skill level.

7. A swimming teaching assistance method based on a mixed reality swimming wearing device, applied to a server corresponding to a mixed reality head-mounted display device, and the method comprises the steps of: The method includes: Obtaining angle data detected by a gyroscope of a mixed reality head-mounted display device and angle data detected by a gyroscope in a bracelet, a leg ring and a swimming trunks through the mixed reality head-mounted display device; Determine the posture action information of the target user based on the angle data detected by the gyroscope of the mixed reality head-mounted display device and the angle data detected by the gyroscope in the bracelet, the leg ring and the swimming trunks, in combination with the height, weight and movement skill data of the target user; The posture action information of the target user is compared with a preset action model, and corresponding prompt information is generated based on the comparison result, in combination with the height, weight and movement skill data of the target user, including:

8. The method of claim 7, wherein, The server determines the posture action information of the target user based on the angle data detected by the gyroscope of the mixed reality head-mounted display device and the angle data detected by the gyroscope in the bracelet, the leg ring and the swimming trunks, in combination with the height, weight and movement skill data of the target user, including: The server generates a preliminary human action model based on the angle data detected by the gyroscope of the mixed reality head-mounted display device and the angle data detected by the gyroscope in the bracelet, the leg ring and the swimming trunks; The server determines the arm, leg and body orientation changes of the target user based on the height, weight and movement skill data of the target user, in combination with the preliminary action of the human body in the preliminary human action model and the speed of the movement.

9. An electronic device, comprising: It includes: At least one processor; and a memory connected in communication with the at least one processor; characterized in that the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of claim 7 or 8.

10. A non-transitory computer readable storage medium storing computer instructions, the computer instructions being configured to cause a computer to perform the method of claim 7 or 8. ​

Images