Virtual reality-based dance teaching equipment and teaching method thereof

Abstract

The invention provides virtual reality-based dance teaching equipment and a teaching method thereof, and the equipment comprises a VR module which is used for providing multi-view immersive visual experience; the motion capture module is used for capturing the joint angle of the dancer in real time and the limb track feedback module and is used for simulating the ground counter-acting force in dancing through vibration or pressure, enhancing motion perception and providing a touch prompt for motion correction; the virtual scene module is used for creating a dance virtual scene; and the action analysis module analyzes the joint action deviation based on multiple view angles, generates an instant score according to the joint angle deviation and the rhythm synchronization rate, and records historical data to generate a progress curve. According to the method, the problems of resource shortage, feedback lag and other pain points in traditional dance teaching are solved, efficient, personalized and interesting dance learning experience is achieved through multi-modal interaction and a dynamic adjustment mechanism, and the application of the method is expected to promote dance education to develop in the intelligent and universal direction.

Description

Technical Field

[0001] The present invention relates to the field of teaching equipment, and in particular to dance teaching equipment and a teaching method based on virtual reality. Background Art

[0002] Dance teaching, as an important part of art education, has traditionally relied on teachers' experience and students' repeated practice. However, the traditional teaching model has the following limitations: limited teaching resources: the number of high-quality dance teachers is limited, and the teaching scene is constrained by objective conditions such as venue and time, making it difficult to meet large-scale and personalized needs; movement feedback is delayed: during the students' practice, teachers need to correct their movements through visual observation, which is highly subjective and has untimely feedback problems, and it is difficult to quantify the specific parameters of movement deviation (such as joint angles and rhythm synchronization rates).

[0003] In recent years, virtual reality (VR) technology has been gradually applied to dance instruction, creating immersive virtual environments to provide students with a more intuitive learning experience. For example, existing technologies use motion capture devices to record student movements and compare them with standard movement models. However, these technologies are still limited to basic movement recognition and lack the ability to deeply analyze movement timing characteristics (such as rhythm matching and continuity optimization) or dynamically adjust difficulty.

[0004] Therefore, it is necessary to provide a new dance teaching equipment based on virtual reality and its teaching method to solve the above technical problems. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention provides a dance teaching device and a teaching method based on virtual reality.

[0006] The virtual reality-based dance teaching device provided by the present invention includes a VR head display for providing a multi-perspective immersive visual experience;

[0007] Motion capture module, used to capture dancers' joint angles and limb trajectories in real time;

[0008] A feedback module is used to simulate ground reaction forces during dance through vibration or pressure, enhancing movement perception and providing tactile cues for movement correction;

[0009] Virtual scene module, used to create virtual dance scenes;

[0010] The motion analysis module analyzes joint motion deviations from multiple perspectives, generates instant scores based on joint angle deviations and rhythm synchronization rates, and records historical data to generate progress curves.

[0011] The AI ​​coaching module is used to analyze students' movement data through machine learning models and generate personalized correction suggestions.

[0012] Furthermore, the virtual scene module also includes adjusting the dance floor lighting, background music rhythm and audience interaction effects through real-time rendering technology to match the trainee's movement performance and emotional state.

[0013] Furthermore, the multi-perspective analysis of joint motion deviation includes: obtaining the focal length, distortion coefficient, rotation matrix and translation vector of each perspective through multi-perspective data fusion, dynamically adjusting the perspective weight according to the occlusion situation, using the triangulation method to calculate the 3D coordinates of the key points, correcting the deviation value according to the perspective angle, eliminating the perspective error, perceiving the motion deviation through tactile perception, and adjusting the posture in time.

[0014] Furthermore, the motion capture module includes inertial sensors, depth cameras and tactile feedback gloves. The inertial sensors are installed on the limbs and torso to collect acceleration and angular velocity data. The depth cameras are used for 2D-3D motion mapping. The tactile feedback gloves are used to provide muscle force prompts through vibration motors.

[0015] Furthermore, the recording of historical data to generate a progress curve includes calculating a moving average score of the latest n training sessions, and using linear regression analysis to fit the relationship between time and score.

[0016] Another aspect of the present invention provides a teaching method for dance teaching equipment based on virtual reality, the method comprising the following steps:

[0017] Step 1: Initialize the virtual scene: Start the virtual reality environment generation module and load the preset virtual dance scene;

[0018] Step 2: Motion capture and evaluation: The motion capture module collects the trainee's motion data in real time and calculates the joint angle deviation and rhythm synchronization rate;

[0019] Step 3: Generate personalized feedback: Use the AI ​​coaching module to analyze movement data, generate corrective suggestions, and output them through the work analysis module;

[0020] Step 4: Dynamic difficulty adjustment: Adjust the training difficulty coefficient based on the progress curve of the historical data recording and analysis module;

[0021] Step 5: Record training results: Store current training data in the history database and update the progress curve.

[0022] Compared with related technologies, the dance teaching equipment and teaching method based on virtual reality provided by the present invention have the following beneficial effects:

[0023] 1. Through the deep integration of virtual reality technology and artificial intelligence, this invention not only solves the pain points of traditional dance teaching such as lack of resources and delayed feedback, but also realizes an efficient, personalized and interesting dance learning experience through multimodal interaction and dynamic adjustment mechanism. Its application is expected to promote the development of dance education towards intelligence and popularization.

[0024] 2. Based on multi-perspective analysis, the present invention dynamically assigns weights through visibility coefficients to reduce the impact of occlusion on deviation calculation. It combines triangulation and depth information to eliminate perspective errors in 2D coordinates, integrates deviation values ​​from multiple perspectives, and provides more accurate movement evaluation. Combining multi-perspective analysis with a real-time feedback mechanism can significantly improve the accuracy and user experience of virtual reality dance teaching.

[0025] 3. The present invention analyzes historical scoring trends based on the student's progress curve through a linear regression algorithm, and dynamically adjusts the training difficulty coefficient, such as increasing the complexity of rotation movements and shortening the beat tolerance time, to ensure that students are always in a "challenge-growth" balance. Through virtual reality technology, rare dance styles can be recorded and widely disseminated, promoting the inheritance and innovation of dance culture. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A structural block diagram of the virtual reality-based dance teaching device provided by the present invention;

[0027] Figure 2 A flowchart of the teaching method of the dance teaching device based on virtual reality provided by the present invention;

[0028] Figure 3 This is a flowchart of the multi-perspective analysis of joint motion deviations provided by the present invention. DETAILED DESCRIPTION

[0029] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0030] Please refer to Figure 1 、 Figure 2 as well as Figure 3 ,in, Figure 1 A structural block diagram of the virtual reality-based dance teaching device provided by the present invention; Figure 2 A flowchart of the teaching method of the dance teaching device based on virtual reality provided by the present invention; Figure 3 This is a flowchart of the multi-perspective analysis of joint motion deviations provided by the present invention.

[0031] In the specific implementation process, Figure 1 As shown, the dance teaching equipment based on virtual reality includes a VR head display for providing a multi-perspective immersive visual experience;

[0032] Motion capture module, used to capture dancers' joint angles and limb trajectories in real time;

[0033] A feedback module is used to simulate ground reaction forces during dance through vibration or pressure, enhancing movement perception and providing tactile cues for movement correction;

[0034] The virtual scene module is used to create a virtual dance scene. Through real-time rendering technology, it adjusts the dance floor lighting, background music rhythm, and audience interaction effects to match the trainee's movement performance and emotional state.

[0035] The motion analysis module analyzes joint motion deviations from multiple perspectives, generates instant scores based on joint angle deviations and rhythm synchronization rates, and records historical data to generate a progress curve. This progress curve is generated by calculating the moving average score of the last n training sessions and using linear regression to analyze the relationship between fitting time and score.

[0036] The AI ​​coaching module is used to analyze students' movement data through machine learning models and generate personalized correction suggestions.

[0037] It should be noted that the matching degree between user actions and music rhythm is evaluated:

[0038]

[0039] Among them, R sync is the rhythm synchronization rate (range 0 to 1, the larger the value, the better the synchronization), T is the total number of music beats (such as 120 beats), t user is the time point when the user completes the t-th action (e.g., the action completion time for the 10th beat is 2.1 seconds), t music is the time point of the t-th beat of the music (e.g., the theoretical time of the 10th beat is 2.0 seconds), and τ is the error tolerance time threshold (e.g., 0.5 seconds);

[0040] The joint angle deviation and rhythm synchronization rate are combined to generate the final score:

[0041] S=w1·(1-Δ joint / 100)+w2·R sync

[0042] Among them, S is the comprehensive score (ranging from 0 to 1, with larger values ​​indicating better performance), and w1 and w2 are weight coefficients (e.g., w1 = 0.6, w2 = 0.4).

[0043] The joint angle difference between the user action and the standard action is quantified:

[0044]

[0045] Among them, Δ jointis the total deviation of the joint angle (range 0 to 100%), N is the number of key joints (such as 5 main joints), is the actual angle of the user's i-th joint (e.g., hip flexion 45°), is the i-th joint angle of the standard action (such as hip flexion 60°), is the maximum range of motion of the i-th joint (refer to joint range of motion data, such as the maximum flexion of the hip joint is 120°).

[0046] Example:

[0047] When the user practices the "Latin dance cha-cha step", the actual angle of the hip joint Standard Angle Maximum range of motion The deviation of a single joint is: If the average deviation of the five joints is 12%, then Δ joint =12%;

[0048] If the joint angle deviation Δ joint =12%, rhythm synchronization rate R sync =0.75, the comprehensive score is:

[0049] 0.6·(1-0.12)+0.4·0.75=0.528+0.3=0.828

[0050] That is a score of 82.8 points.

[0051] Generate a learning curve using historical data and calculate the moving average score for the last n training runs:

[0052]

[0053] Use linear regression to analyze the relationship between fitting time and score:

[0054] S(t)=a·t+b

[0055] Among them, the slope a is the rate of progress (a positive slope indicates improvement, and a negative slope indicates regression).

[0056] Here is an example:

[0057] The user's ratings in the past five training sessions were 82.8, 85.3, 88.1, 90.5, and 92.2;

[0058] The moving average rating is

[0059] After linear regression fitting, if the slope a = 1.5, it means that the score increases by 1.5 points after each training session.

[0060] It should be noted that the reference Figure 3 As shown, the joint motion deviation analysis based on multi-viewpoints includes: obtaining the focal length, distortion coefficient, rotation matrix and translation vector of each viewpoint through multi-viewpoint data fusion, dynamically adjusting the viewpoint weight according to the occlusion situation, using the triangulation method to calculate the 3D coordinates of the key points, correcting the deviation value according to the viewpoint angle, eliminating the perspective error, and perceiving the motion deviation through tactile perception to adjust the posture in time;

[0061] Use triangulation to calculate the 3D coordinates of key points:

[0062]

[0063] Example:

[0064] The 2D coordinates of the key points of the user's hand in three perspectives are:

[0065] Front view: (x1, y1) = (300, 200)

[0066] Side view: (x2, y2) = (400, 150)

[0067] Back view: (x3, y3) = (250, 250)

[0068] Get 3D coordinates through triangulation calculation

[0069] Correct the deviation value according to the viewing angle to eliminate perspective error:

[0070]

[0071] Among them, Δ raw is the original deviation value (calculated based on 2D coordinates), d avg is the average viewing distance (such as the average depth of all viewing angles), d v The depth of the current viewing angle (e.g., the front view is 1.5 meters away from the user);

[0072] Here is an example:

[0073] If the original deviation of the front view is Δ raw =2.0cm, average visual distance d avg =2.0cm, then the deviation after compensation is:

[0074] It should be noted that the motion capture module includes inertial sensors, depth cameras and tactile feedback gloves. The inertial sensors are installed on the limbs and torso to collect acceleration and angular velocity data. The depth camera is used for 2D-3D motion mapping, and the tactile feedback gloves are used to provide muscle force prompts through vibration motors.

[0075] Example 2

[0076] In a specific implementation process, refer to Figure 2 As shown, a teaching method of dance teaching equipment based on virtual reality includes the following steps:

[0077] Step 1: Initialize the virtual scene: Start the virtual reality environment generation module and load the preset virtual dance scene;

[0078] Step 2: Motion capture and evaluation: The motion capture module collects the trainee's motion data in real time and calculates the joint angle deviation and rhythm synchronization rate;

[0079] Step 3: Generate personalized feedback: Use the AI ​​coaching module to analyze movement data, generate corrective suggestions, and output them through the work analysis module;

[0080] Step 4: Dynamically adjust the difficulty: Adjust the training difficulty coefficient based on the progress curve of the historical data recording and analysis module:

[0081] D next =D current (1+α·ΔS)

[0082] Among them, ΔS is the score change rate, and α is the difficulty adjustment coefficient;

[0083] Step 5: Record training results: Store current training data in the history database and update the progress curve.

[0084] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, or of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiment.

[0085] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, from all points of view, the embodiments should be regarded as illustrative and non-restrictive. The scope of the present invention is defined by the appended claims rather than the foregoing description. It is intended that all changes that come within the meaning and range of equivalents of the claims be included in the present invention, and any reference signs in the claims should not be construed as limiting the claims to which they relate.

[0086] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A dance teaching device based on virtual reality, characterized in that: Includes VR module to provide multi-perspective immersive visual experience; Motion capture module, used to capture dancers' joint angles and limb trajectories in real time; A feedback module is used to simulate ground reaction forces during dance through vibration or pressure, enhancing movement perception and providing tactile cues for movement correction; Virtual scene module, used to create virtual dance scenes; The motion analysis module analyzes joint motion deviations from multiple perspectives, generates instant scores based on joint angle deviations and rhythm synchronization rates, and records historical data to generate progress curves.

2. The dance teaching device based on virtual reality according to claim 1, characterized in that: It also includes an AI coaching module that analyzes student movement data through machine learning models and generates personalized correction suggestions.

3. The dance teaching device based on virtual reality according to claim 2, characterized in that: The virtual scene module also includes adjusting the dance floor lighting, background music rhythm and audience interaction effects through real-time rendering technology to match the trainee's movement performance and emotional state.

4. The dance teaching device based on virtual reality according to claim 3, characterized in that: The multi-perspective analysis of joint motion deviation includes: obtaining the focal length, distortion coefficient, rotation matrix and translation vector of each perspective through multi-perspective data fusion, dynamically adjusting the perspective weight according to the occlusion situation, using the triangulation method to calculate the 3D coordinates of the key points, correcting the deviation value according to the perspective angle, eliminating the perspective error, perceiving the motion deviation through tactile perception, and adjusting the posture in time.

5. The dance teaching device based on virtual reality according to claim 4, characterized in that: The motion capture module includes inertial sensors, depth cameras and tactile feedback gloves. The inertial sensors are installed on the limbs and torso to collect acceleration and angular velocity data. The depth camera is used for 2D-3D motion mapping. The tactile feedback gloves are used to provide muscle force prompts through vibration motors.

6. The dance teaching device based on virtual reality according to claim 5, characterized in that: The VR module includes a VR head display.

7. The dance teaching device based on virtual reality according to claim 6, characterized in that: The recording of historical data to generate a progress curve includes calculating a moving average score of the latest n training sessions and using linear regression analysis to fit the relationship between time and score.

8. The dance teaching device based on virtual reality according to claim 7, characterized in that: 。 9. A teaching method for a dance teaching device based on virtual reality, applicable to the dance teaching device based on virtual reality according to any one of claims 1 to 8, characterized in that: The method comprises the following steps: Step 1: Initialize the virtual scene: Start the virtual reality environment generation module and load the preset virtual dance scene; Step 2: Motion capture and evaluation: The motion capture module collects the trainee's motion data in real time and calculates the joint angle deviation and rhythm synchronization rate; Step 3: Generate personalized feedback: Use the AI ​​coaching module to analyze movement data, generate corrective suggestions, and output them through the work analysis module; Step 4: Dynamic difficulty adjustment: Adjust the training difficulty coefficient based on the progress curve of the historical data recording and analysis module; Step 5: Record training results: Store current training data in the history database and update the progress curve.

Images