Attractive teaching interaction system based on virtual reality

Abstract

The invention provides a beauty teaching interaction system based on virtual reality, and the system comprises a VR / AR scene engine module which is used for scene modeling and prop modeling, and is used for completing the production of scene assets; the motion capture module is used for establishing hardware equipment and software tools, carrying out motion capture and positioning of students, designing a teaching management background, and carrying out remote interaction and teaching between teachers and the students; the background building module is used for presetting a plurality of scene libraries and carrying out guided learning of the students; the interaction and output module is used for configuring a virtual creation tool and obtaining and exporting a final work; and the exhibition hall display module is used for building a virtual art exhibition hall and carrying out virtual display and work scoring. According to the invention, various beauty teaching situations are constructed through VR / AR and other technologies, so that students can experience artistic works in a virtual scene, carry out diversified artistic creation and accept guidance, and teachers can carry out real-time remote interaction and tutoring, break physical distance limitation and realize immersive artistic learning.

Description

Technical Field

[0001] The present invention relates to the technical field of aesthetic education teaching, and in particular to an aesthetic education teaching interactive system based on virtual reality. Background Art

[0002] Aesthetic education requires students to have a deep understanding and practice of works of art. Traditional teaching is limited by physical space and teaching equipment, making it difficult to provide students with an immersive and interactive learning environment. For example: 1) Lack of real art scene experience: Ordinary aesthetic education cannot allow students to immerse themselves in environments such as theaters, galleries, and concert halls, which affects their artistic perception experience. 2) Cross-regional limitations: There is a lack of high-quality remote interaction solutions between teachers and students. Even using video links, it is impossible to accurately guide actual artistic behavior. 3) Inability to achieve diversified artistic creation: Traditional art tools are limited in hardware and methods, resulting in a single form of creation. Summary of the Invention

[0003] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide an aesthetic education teaching interactive system based on virtual reality, which constructs various aesthetic education teaching scenarios through VR / AR and other technologies, so that students can experience works of art, carry out diversified artistic creation and receive guidance in virtual scenes. At the same time, teachers can also conduct real-time remote interaction and guidance, breaking the limitations of physical distance and realizing immersive art learning.

[0004] To achieve the above-mentioned purpose, the present invention provides the following solution: an interactive system for aesthetic education based on virtual reality, comprising:

[0005] The VR / AR scene engine module is used to perform scene modeling and prop modeling to obtain a scene model. Based on the scene model, it performs texturing, material processing, motion capture animation generation, game engine import, interaction and scene configuration to complete the production of VR / AR scene assets;

[0006] The motion capture module is used to build hardware equipment and software tools to capture and locate students' motions, design a teaching management background, conduct remote interaction and teaching between teachers and students, and record the first interaction data and student performance results in real time to obtain the first student learning report and feedback data;

[0007] A backend construction module is used to preset multiple scenario libraries, set scenario functions, course tasks and personalized tasks of the scenario libraries, and based on the scenario libraries, collect second interaction data in real time, display students' learning progress, and configure teacher guidance tools to guide students' learning;

[0008] The interaction and output module is used to configure virtual creation tools, record students' virtual creation behaviors, and obtain the final works, and then classify, store and export the final works and design interactive triggers;

[0009] The exhibition hall display module is used to build a virtual art exhibition hall, conduct a virtual display of the final works, and score and analyze the final works to provide feedback on the works;

[0010] Among them, the VR / AR scene engine module, the motion capture module, the background construction module, the interaction and output module and the exhibition hall display module are interconnected.

[0011] Optionally, the VR / AR scene engine module includes:

[0012] A scene creation unit is used to import an artistic scene into a 3D modeling software for model generation to obtain a scene model, and then use segmented modeling technology to divide the scene model into multiple independent modules to complete the scene modeling, and then use low-polygon modeling technology to model props in the scene model;

[0013] A scene processing unit is used to use 3D texture production software to perform texture drawing and material processing on the scene model, and to set the reflection, transparency and texture of the lights and walls in the scene model. Then, through the motion capture equipment, the natural movement data of the character is obtained, and the 3D character animation software is used to perform bone mapping and smoothing processing on the natural movement data of the character to create a virtual guide character or interactive object;

[0014] An engine import and configuration unit is used to select a game development engine, import the scene model into the selected game development engine, perform scene and prop management, player interaction logic design, and scene physical property configuration to obtain a VR / AR engine, and then perform scene light source debugging and rendering performance optimization to complete scene construction;

[0015] An AR mode expansion unit, which utilizes an AR software development kit and a mobile camera to perform environmental scanning and plane positioning of the art scene to adjust the proportional consistency between virtual and real objects. Furthermore, in AR mode, it adds audio-visual interaction with artworks and virtual painting capabilities. The audio-visual interaction includes voice commentary of artworks and the configuration of behavioral logic for three-dimensional dynamic objects. The virtual painting function allows for drawing using a virtual brush.

[0016] An integrated testing unit is used to verify the display effects of virtual assets on different hardware and adjust the scene ratio.

[0017] Optionally, the motion capture module includes:

[0018] The capture framework unit is used to select positioning equipment and software tools, use the positioning equipment to capture head and hand motions, and use the software tools to convert the captured motion coordinates into skeletal animation to obtain a standard skeletal structure. Then, a motion smoothing algorithm is used to eliminate capture jitter and data delay. The positioning equipment includes a VR controller and helmet, motion capture gloves, a full-body positioning device, and a mobile camera. The software functions include an SDK and a kinematics library.

[0019] A data mapping unit is used to transmit the captured movements to the VR / AR engine in real time for physical mapping and animation generation, and to optimize rendering in combination with a depth sensor. Furthermore, a skeletal mapping engine is used to automatically identify and provide feedback on incorrect postures.

[0020] A remote interactive unit is used to develop a teaching management background for real-time monitoring and capturing of movements, and to configure the teaching management background with multi-view panoramic viewing functions, movement correction functions, real-time mapping functions, and real-time learning functions;

[0021] The data recording and feedback unit is used to record the student's gesture path, skeletal data and operation completion time in real time to obtain first interaction data, combine the first interaction data with the student's performance results, generate a first student learning report, and train an action-based temporal modeling network. The gesture prediction function is added to the training process to obtain a feedback model. The feedback model is used to analyze the student's abnormal posture to obtain feedback data; wherein, the first student learning report includes dance movement scores, virtual painting analysis and simulated instrument performance matching index.

[0022] Optionally, the perspective panoramic viewing function is used to switch the perspective of the virtual camera to observe the student performance from multiple angles;

[0023] The action correction function is used to convert the teacher's input voice into text or animation to guide students to correct their movements, realizing the combination of voice input and virtual scenes;

[0024] The real-time mapping function is used to track the student's real-world position through the AR development tool and SLAM technology to dynamically overlay virtual art elements into the camera's field of view;

[0025] The real-time learning function is used to mark and modify students' virtual paintings in a two-way manner in an AR environment, and to display the target movement trajectory through AR animation, so that students can follow the virtual dynamic arrows to complete the movements, thereby realizing students' painting training and dance training.

[0026] Optionally, the backend building module includes:

[0027] A virtual teaching room unit is used to preset a variety of scene libraries and set scene functions, course tasks and personalized tasks of the scene libraries;

[0028] A progress display unit is used to assign student login permissions, collect the student's immersion time, task progress, and interaction point trigger frequency after entering the virtual scene, obtain second interaction data, store real-time behavior in a NoSQL database and task completion status in a MySQL database based on the second interaction data, and generate a second student learning report showing learning progress based on the second interaction data; the second student learning report includes task execution time, task completeness, and action completion accuracy;

[0029] The guidance tool unit is used to configure a thumbnail map including the multi-perspective panoramic viewing function, configure a guidance tool including the action correction function, the real-time mapping function and the real-time learning function, and add collaborative tasks in the virtual scene to realize a multi-student collaborative mode and a competition mode.

[0030] Optionally, the scene library includes a dance class scene, an art gallery scene, and a concert hall scene, wherein the dance class scene includes a virtual floor and a dynamic dance guide model, the art gallery scene includes a variety of artworks, and the concert hall scene includes a stage, a virtual audience, and a virtual instrument simulation;

[0031] The scene functions include prop addition, lighting adjustment, material adjustment and teaching script editor.

[0032] Optionally, the interaction and output module includes:

[0033] An art creation tool unit is used to configure virtual tools, use Unity Physics or Unreal Engine Physics to provide physical feedback of the virtual tools, and bind the handle to the virtual tools through a positioning device to achieve the integration of motion capture and virtual space;

[0034] A creation recording unit is used to record the creation trajectory, use a compression algorithm to downsample redundant trajectory points in the creation trajectory, store the downsampled creation trajectory in a NoSQL database, and then classify, store and export the completed final works;

[0035] An interactive triggering unit is used to create a virtual interactive object based on the final work, and to design a node triggering mechanism based on the virtual interactive object to realize the function of automatically popping up relevant instructions after touching the virtual interactive object;

[0036] The AR mode expansion unit is used to use AR software development tools to overlay a virtual canvas onto a desktop or wall in a real scene, and to record the overlay process of the virtual canvas in real time in the form of video, and to automatically save the painting effect after the virtual canvas is overlaid in the form of pictures.

[0037] Optionally, the creation trajectory includes three-dimensional coordinates, creation data points and a point sequence of the creation data points, the point sequence includes a timestamp, a manual creation path and tool settings, and the final work includes a painting, a sculpture and a music work, the painting is in a picture format, the sculpture is in a 3D model format, and the music work is in a MIDI data file format including a performance trajectory.

[0038] Optionally, the exhibition hall display module includes:

[0039] A results archiving and conversion unit is used to integrate the creative trajectory and the final work to obtain comprehensive creative data, classify and store the comprehensive creative data according to preset work archiving rules, and call API and AI image enhancement technology to export the comprehensive creative data;

[0040] A virtual exhibition hall design unit is used to build a virtual art exhibition hall, design the work display logic, and based on the virtual art exhibition hall, design a virtual tour and real-time comment mode to realize the explanation and comment of the works, and generate the work comment record;

[0041] The review and scoring unit is used to perform teacher scoring, student scoring and AI scoring on the final work, and then conduct a comprehensive analysis of the scoring results to generate work recommendations.

[0042] Optionally, the work archiving rules are used to generate an identification ID and embed metadata based on the final work; the identification ID includes the student ID, task name, submission time and file type, and the metadata includes student information, scoring dimensions and creation tool parameters.

[0043] The present invention provides an interactive system for aesthetic education and teaching based on virtual reality, which discloses the following technical effects:

[0044] 1. Highly Replicated Immersive Scenes: 1) 3D modeling and real-time rendering recreate various scenes, including famous art galleries, theaters, concert halls, and stages, enhancing the student learning experience. 2) In AR mode, digital art elements are overlaid on real-world spaces through mobile cameras, allowing students to interact with art in real-world environments.

[0045] 2. Flexible Art Creation: 1) By providing a variety of virtual art tools (such as virtual paintbrushes, carving knives, and musical instruments), students can create freely in 3D space and produce exhibitable works of art. 2) By integrating virtual and real life, for example, students can use a controller to create a "virtual oil painting." The system automatically records the image generation process and automatically exports a schematic diagram or 3D model of the work after the creation is completed.

[0046] 3. Breaking through geographical limitations and enhancing real-time interaction: 1) VR headsets or sensors capture students' hearing, vision, and movements, enabling two-way interaction with digital characters or objects in virtual scenes. 2) Teachers can use the software to view and guide students' actions or performances in the virtual environment, such as dance moves or virtual instrument playing postures, achieving the effect of "remote face-to-face instruction."

[0047] 4. Interactive teaching management backend: 1) Teachers can create virtual teaching rooms, add preset scenes, specify learning tasks and interactive scripts, such as requiring students to select two works for comparison and evaluation in a virtual art gallery. 2) Manage student logins, view students' learning progress and interaction indicators in the virtual environment (such as immersion time, work completion, etc.), and initiate voice / text / video guidance at any time.

[0048] 5. Virtual results display and evaluation: Teachers and students, or teachers and students from different schools, can conduct real-time comments and presentations in the virtual exhibition space, expanding teaching horizons and learning value.

[0049] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 A schematic diagram of the system architecture provided by an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of the virtual work creation process provided by an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of the remote interactive teaching process provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0055] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0056] like Figure 1 As shown, the present invention provides an aesthetic education and teaching interactive system based on virtual reality, including a VR / AR scene engine module, a motion capture module, a background construction module, an interaction and output module and an exhibition hall display module that are interconnected.

[0057] 1. VR / AR scene engine module

[0058] like Figure 2 As shown, it is used to perform scene modeling and prop modeling to obtain a scene model. Based on the scene model, it performs mapping processing, material processing, motion capture animation generation, game engine import, interaction and scene configuration to complete the production of VR / AR scene assets. The VR / AR scene engine module includes:

[0059] 1.1 Scene creation unit

[0060] It is used to import artistic scenes into 3D modeling software, such as Blender, Maya, etc., to generate models and obtain scene models. It then uses segmented modeling technology to divide the scene model into multiple independent modules to complete the scene modeling, and then uses low-polygon modeling technology to model props in the scene model.

[0061] Venue modeling: For architectural scenes such as famous art galleries, theaters, concert halls, and stages, import the venue building's CAD files or reference drawings to generate models. Use segmented modeling technology to divide the venue into independent modules, such as stage areas, easels, seats, etc., to facilitate subsequent editing.

[0062] Prop modeling: For detailed props such as virtual brushes, carving knives, musical instruments, etc., low-polygon modeling technology (Low-Poly) is used to ensure high rendering efficiency while retaining artistic details.

[0063] 1.2 Scene Processing Unit

[0064] It is used to use 3D mapping software (Substance Painter or Adobe Photoshop) to map and process the materials of the scene model, and set the reflection, transparency and texture materials of the lights and walls in the scene model to enhance the realism of the venue. Then, through motion capture equipment such as Vicon and OptiTrack, the character's natural motion data is obtained, and 3D character animation software such as MotionBuilder is used to perform bone mapping and smoothing on the character's natural motion data to ensure smooth character movements, so as to virtually guide characters or interactive objects.

[0065] 1.3 Engine import and configuration unit

[0066] It is used to select a game development engine, import the scene model into the selected game development engine, perform scene and prop management, player interaction logic design and scene physical property configuration, obtain the VR / AR engine, and then perform scene light source debugging and rendering performance optimization to complete the scene construction.

[0067] Optional game development engines include Unity, which is easy to use and suitable for lightweight 3D scenes and rapid prototyping. Unreal Engine (UE) supports high-precision rendering and is suitable for high-fidelity scenes in art venues.

[0068] Scene and prop management: Utilize the engine's scene management function to load venues and props in layers according to structure, such as the front exhibition area, stage area, backstage, etc.

[0069] Player interaction logic design: Players can move freely within the venue, managed by keyboard, gamepad, or VR controller, and path planning is controlled by a navigation mesh (NavMesh). Famous painting / prop interaction implements a "select" trigger, automatically popping up text descriptions or voice interpretations when players approach a famous painting or object.

[0070] Scene physics configuration: Use the engine's physics module to set the collision properties and motion feedback of objects within the scene. For example, realistically configure the mass, friction, and other physical effects of virtual painting tools to demonstrate dynamic effects (such as stage lighting changing with interaction and curtains physically swinging).

[0071] 1.4AR mode expansion unit

[0072] This tool uses the AR SDK and mobile camera to perform environmental scanning and surface positioning of art scenes, adjusting the proportional consistency between virtual and real-world objects. Environmental scanning: Scans the real space to detect flat surfaces, such as tabletops, floors, and walls. Surface positioning: Dynamically places art assets, such as famous paintings and sculptures, within the real-world environment.

[0073] In AR mode, audio-visual interaction function and virtual painting function of artworks are added; the audio-visual interaction function includes voice explanation of artworks and behavioral logic configuration of three-dimensional dynamic objects, and the virtual painting function is drawn through a virtual brush. For example, students holding AR devices can use virtual brushes to draw on the background captured by the camera, and superimpose it on the visual space in real time.

[0074] Voice explanation, such as clicking on a statue to trigger the corresponding voice explanation; behavioral logic configuration of three-dimensional dynamic objects, such as "rotation management of shadow angle" or "layered display of the creation details of famous paintings."

[0075] 1.5 Integration Test Unit

[0076] Used to verify the display effect of virtual assets on different hardware (VR helmets, AR iPads), adjust the scene ratio, and prevent clipping.

[0077] 2. Motion capture module

[0078] like Figure 2 、 Figure 3 As shown, the hardware equipment and software tools are used to capture and locate students' motions, design a teaching management background, conduct remote interaction and teaching between teachers and students, and record first interaction data and student performance results in real time to obtain first student learning reports and feedback data; the motion capture module includes:

[0079] 2.1 Capture frame unit

[0080] It is used to select positioning equipment and software tools, use positioning equipment to capture head and hand motions, and use software tools to convert the captured motion coordinates into skeletal animation to obtain a standard skeletal structure, and then use motion smoothing algorithms to eliminate capture jitter and data delays.

[0081] 1) The positioning device (or sensor) includes:

[0082] VR controllers and headsets, such as Oculus Quest and HTC Vive, are used to monitor head posture and position information. The VR headset's IMU (inertial measurement unit) obtains head rotation and position coordinates in real time to synchronize the field of view and the virtual character's head position.

[0083] head-mounted display

[0084] Immersive Experience: Provides users with a high-resolution, low-latency visual experience, allowing them to immerse themselves in a virtual art education classroom. For example, high-end head-mounted displays (HMDs) such as the HTC Vive and PICO can present detailed images and realistic scenes, making users feel as if they are in a real art gallery or concert hall. Head tracking: Captures the user's head movement and orientation in real time, ensuring that the user's perspective in the virtual environment is synchronized with their real-life movements, enhancing immersion and interactivity.

[0085] Motion capture gloves, such as the Manus Prime gloves, are used to capture students' hand movements and knuckle dynamics. Joint data captured by the gloves or cameras is mapped to a virtual hand model, with multi-point detection configured, such as finger bending degree and grasping angle, to support the operation of virtual brushes, carving knives, and other instruments.

[0086] Controllers and Sensors: Interactive Operations: Equipped with controllers such as handles and data gloves, users can interact with objects in the virtual environment through gestures and movements. For example, in a virtual painting class, users can use handles to simulate brushes to create paintings; in dance classes, motion sensors can capture the user's body movements and enable interaction with a virtual dance partner.

[0087] Full-body positioning devices, such as OptiTrack or SteamVR base stations, are used to accurately capture full-body skeletal postures. Based on the HumanoidAvatar system, a standard skeletal structure is constructed to convert the captured coordinates into skeletal animations. Motion smoothing algorithms, such as Kalman filtering or bidirectional filtering, are used to eliminate capture jitter and data delays.

[0088] The mobile camera (for AR mode) uses the camera to identify key points and track gestures or body movements.

[0089] 2) The software functions include:

[0090] SDKs, such as the SteamVR SDK and Oculus SDK, provide device interfaces for capturing and transmitting real-time data;

[0091] Kinematic libraries such as DeepMotion or IKinema generate smooth skeletal or mesh animations.

[0092] 2.2 Data Mapping Unit

[0093] It is used to transmit the captured movements to the VR / AR engine in real time for physical mapping and animation generation, and to optimize rendering in combination with the depth sensor, and then use the skeleton mapping engine to automatically identify and feedback incorrect postures.

[0094] Skeletal Animation Mapping: Use Unity / UE4 animation controllers to adjust the avatar's skeletal animation based on the 3D position coordinates output by the VR headset and handheld device. Action data is bound to virtual props, such as brushes and musical instruments, to achieve "manual brush operation" or "instrument triggering."

[0095] Rendering Logic: GPU-optimized algorithms render real-time captured data at a frame rate of 60-90 FPS to ensure smooth virtual feedback. A depth sensor is used to enable dual-segment model rendering. For example, dynamic line effects can be created based on brush strokes.

[0096] Also optional: 1) Real-time rendering engine: Using professional virtual reality engines such as Unity 3D and Unreal Engine, which are responsible for the creation, rendering and real-time updating of virtual scenes. These engines provide rich graphics rendering capabilities and efficient physical simulation tools, which can present users with a realistic virtual art education classroom environment. 2) Physics engine: Simulates the physical behavior of objects in the virtual world, such as collisions, gravity, friction, etc., to make user interactions more realistic and natural. For example, in a virtual sculpture class, when users use tools to sculpt virtual clay, the physics engine can simulate real force feedback and deformation effects. 3) Audio engine: Responsible for processing sound effects in the virtual environment, including three-dimensional sound effects, environmental sound effects, voice interaction, etc. Through the audio engine, users can hear realistic music, sound effects and the voices of others in the virtual art education classroom, enhancing immersion and interactivity.

[0097] Virtual skeleton and real-world feedback: The skeleton mapping engine automatically identifies and provides feedback on incorrect postures. For example, in a dance, the system marks areas with abnormal movement ranges and automatically generates virtual prompts.

[0098] 2.3 Remote Interaction Unit

[0099] It is used to develop a teaching management background for real-time monitoring and capturing actions, in which a multi-perspective panoramic viewing function, an action correction function, a real-time mapping function and a real-time learning function are configured.

[0100] The panoramic viewing function is used to switch the virtual camera perspective to observe student performance from multiple angles.

[0101] The movement correction function is used to convert the voice input by the teacher into text or animation to guide students to correct their movements, realizing the combination of voice input and virtual scenes; for example, in dance teaching, the system will automatically correct the direction of students' movements according to the teacher's voice instructions, and the teacher will guide students to adjust body parts or tool interaction angles through prompt boxes or dynamic arrows (AR aperture).

[0102] The real-time mapping feature uses AR development tools and SLAM technology to track students' real-world locations through point clouds, dynamically overlaying virtual art elements into the camera's field of view. For example, a 3D simulated musical score can be displayed on a real stage, allowing students to control virtual instruments through gestures. SLAM technology is used to supplement plane detection to ensure that virtual objects are aligned with their physical locations.

[0103] The real-time learning function is used to mark and modify students' virtual paintings in a two-way manner in an AR environment, and to display the target movement trajectory through AR animation, so that students can follow the virtual dynamic arrows to complete the movements, thereby realizing students' painting and dance training.

[0104] 2.4 Data recording and feedback unit

[0105] It is used to record the student's gesture path, skeleton data and operation completion time in real time to obtain the first interaction data, combine the first interaction data and the student's performance results to generate the first student learning report, and train the action-based timing modeling network. The gesture prediction function is added during the training process to obtain a feedback model. The feedback model is used to analyze the student's abnormal posture to obtain feedback data.

[0106] The first student learning report includes a dance movement score, a virtual drawing analysis, and a simulated instrument performance matching index. The dance movement score is calculated based on the matching calculation between the trigger area and the skeletal angle, and the virtual drawing analysis is analyzed based on the handwriting accuracy score.

[0107] 3. Backend construction module

[0108] like Figure 2 、 Figure 3 As shown, it is used to preset multiple scenario libraries, set the scenario functions, course tasks and personalized tasks of the scenario libraries, and based on the scenario libraries, collect the second interaction data in real time, display the students' learning progress, and configure teacher guidance tools to guide students' learning; the background construction module includes:

[0109] 3.1 Virtual Teaching Room Unit

[0110] It is used to preset multiple scenario libraries, set the scenario functions, course tasks and personalized tasks of the scenario libraries, and then assign teaching tasks.

[0111] 1) The scene library includes:

[0112] Dance class scene, including a virtual floor and dynamic dance guidance model;

[0113] Art gallery scenes display multiple artworks and can generate interactive tasks in designated areas, such as information display;

[0114] The concert hall scene includes a stage, virtual audience, and supports virtual instrument simulation.

[0115] 2) The scene functions include: prop addition, lighting adjustment, material adjustment and teaching script editor.

[0116] 3.2 Progress Display Unit

[0117] It is used to assign student login permissions, collect the immersion time, task progress and interaction point triggering frequency of students after entering the virtual scene, obtain second interaction data, store real-time behavior in a NoSQL database based on the second interaction data, store task completion status in a MySQL database, and generate a second student learning report showing learning progress based on the second interaction data; the second student learning report includes task execution time, task completeness and action completion accuracy; for example, "Student A is currently in the virtual art gallery and has completed 2 / 4 of the task points."

[0118] 3.3 Guidance Tool Unit

[0119] It is used to configure a thumbnail map including the multi-view panoramic viewing function, configure guidance tools including the motion correction function, the real-time mapping function, and the real-time learning function, and add collaborative tasks in the virtual scene to realize multi-student collaborative mode and competition mode. Multi-student collaborative mode, such as "two students must work together to draw a picture" or "multi-person class to perform a song"; the competition mode, such as supporting multiple people to compete in tasks at the same time, such as showing who completed the task faster or more accurately in real time through a scoreboard.

[0120] 4. Interaction and output module

[0121] like Figure 2 、 Figure 3 As shown, it is used to configure virtual creation tools, record students' virtual creation behaviors, obtain final works, and then classify, store and export the final works and design interactive triggers; the interaction and output module includes:

[0122] 4.1 Artistic Creation Tools Unit

[0123] It is used to configure virtual tools, use UnityPhysics or Unreal Engine Physics to provide physical feedback of the virtual tools, and bind the handle to the virtual tools through a positioning device to achieve the integration of captured actions and virtual space.

[0124] 1) Virtual Tools

[0125] Virtual Brush: Supports multiple brush types, such as oil brushes, watercolor brushes, and airbrushes (configurable pressure, angle, and thickness). The color palette allows students to freely select and mix colors.

[0126] Graver Tool: A tool for creating virtual sculptures that allows students to smooth, dig, and carve virtual clay blocks. It supports dynamic modification and detail optimization of polygonal meshes.

[0127] Musical Instrument Simulator: For virtual instruments such as piano, erhu, and violin, it supports real-time playback of notes based on students' gestures.

[0128] 2) Give creative tools real-world physical properties, for example: a virtual brush stroke simulates changes with pressure and speed, and the rotation / angle of a carving tool affects the depth of the carving.

[0129] 3) Integration of motion capture and virtual space: Using students’ motion capture skeletal data (e.g., hand or full body) to synchronize virtual tool manipulation. For example, motion mapping for a paintbrush tool = wrist position + direction control. Dynamic instrument playing gestures = finger position sensors + pitch calculation algorithms.

[0130] 4.2 Creation Record Unit

[0131] Used to record the creative trajectory, use a compression algorithm to downsample the redundant track points in the creative trajectory, and store the downsampled creative trajectory in a NoSQL database, and then classify, store and export the completed final works;

[0132] The creation trajectory includes three-dimensional coordinates (recording every change in the student's movement in painting or sculpting), creation data points and a point sequence of the creation data points, wherein the point sequence includes a timestamp, manual creation path and tool settings (brush strength, sculpting pressure).

[0133] The final product includes:

[0134] Painting works are in image format, and 3D paintings are stored as high-resolution images through texture projection algorithms;

[0135] Sculpted works are converted into 3D models in `.FBX` or `.OBJ` formats, which can be exported and used in other design software;

[0136] The MIDI data file format of the musical composition, including the performance track, can be exported to an audio format.

[0137] 4.3 Interaction Trigger Unit

[0138] It is used to create a virtual interactive object according to the final work, and design a node trigger mechanism based on the virtual interactive object to realize the function of automatically popping up relevant instructions after touching the virtual interactive object.

[0139] 4.4AR Mode Extension Unit

[0140] It is used to use AR software development tools to overlay a virtual canvas onto a desktop or wall in a real scene, and to record the overlay process of the virtual canvas in real time in the form of video, and to automatically save the painting effect after the virtual canvas is overlaid in the form of pictures.

[0141] 5. Exhibition hall display module

[0142] like Figure 2 、 Figure 3 As shown, it is used to build a virtual art exhibition hall, conduct virtual display of the final works, score and analyze the final works, and complete work feedback; the exhibition hall display module includes:

[0143] 5.1 Achievement Archiving and Transformation Unit

[0144] It is used to integrate the creative trajectory and the final work to obtain comprehensive creative data, classify and store the comprehensive creative data according to preset work archiving rules, and call API and AI image enhancement technology to export the comprehensive creative data;

[0145] The work archiving rules are used to generate an identification ID and embed metadata based on the final work; the identification ID includes the student ID, task name, submission time and file type, and the metadata includes student information, scoring dimensions and creation tool parameters.

[0146] 5.2 Virtual Exhibition Hall Design Unit

[0147] It is used to build a virtual art exhibition hall, design the work display logic, and based on the virtual art exhibition hall, design a virtual tour and real-time comment mode to realize the explanation and comment of the works, and generate the work comment record.

[0148] 1) Exhibition hall scene construction:

[0149] Use Unity / Unreal Engine to create a multi-person VR art exhibition hall. Based on the physical space design, it includes the following areas:

[0150] Personal Works Wall: Displays paintings and sculptures uploaded by students;

[0151] Group Cooperation Exhibition Area: Focus on displaying team collaboration projects;

[0152] Dynamic display area: plays videos of performance and creation process.

[0153] 2) Works display logic:

[0154] The students' achievements are dynamically loaded into the exhibition hall scene as virtual objects (wall frames, statues, etc.), and connected with real-time metadata: work title, author name, and task background display.

[0155] Set interactive triggers: Click on an artwork to display a detailed introduction or the creation process. Interact with the content (zoom in, rotate the artwork, etc.) using a virtual laser pointer or gestures.

[0156] 5.3 Comments and Ratings

[0157] It is used to perform teacher scoring, student scoring and AI scoring on the final work, and then conduct a comprehensive analysis of the scoring results to generate work recommendations.

[0158] Teacher Comments: Intuitive scoring is provided through a 5-star rating system. Detailed comments are provided, supplemented by text, voice, or emoji feedback. For example, "The movements are very delicate, and the engraving edges are optimized and improved."

[0159] Peer Review: Create an anonymous grading option to facilitate fair peer review. Provide recommended tagging features such as "Creativity Monitoring" and "Vivid Colors" to intuitively reflect creative strengths.

[0160] AI Scoring: Uses computer vision technology to analyze standardized techniques in works, such as symmetry and color harmony, and generates an AI score.

[0161] The aesthetic education interactive system of the present invention can also be applied to campus aesthetic education second classroom, course supplement and innovative content experience, public cultural services, social art popularization, art training institution special courses, etc. For example:

[0162] 1) Immersive art appreciation and experience

[0163] Virtual Art Exhibitions: Create realistic virtual art galleries to showcase a variety of paintings, sculptures, photographs, and other artworks. Users can browse freely in the virtual exhibition halls, appreciate the details of the works up close, and gain a deeper understanding of the background and artistic value of the works through audio and text explanations.

[0164] Immersive Concerts and Theatre Performances: Build virtual concert halls and theaters, allowing users to experience high-quality music and theatre performances as if they were in real-life venues. Users can choose from different seating perspectives to experience the atmosphere and sound effects of the venue, and can also interact with other audience members.

[0165] Recreating historical and cultural scenes: Using virtual reality technology to recreate historical aesthetic scenes and cultural events, such as ancient art studios and Renaissance court performances, allows users to more intuitively experience the artistic styles and cultural atmosphere of different historical periods, broadening their aesthetic education horizons.

[0166] 2) Interactive Art Creation and Practice

[0167] Virtual Painting and Sculpture Creation: A rich selection of virtual painting and sculpting tools are provided, allowing users to create in virtual space using controllers or data gloves. The system supports simulation of a variety of painting media and sculpting materials, allowing users to freely unleash their creativity and create unique works of art. Users can also save their works in a virtual gallery for sharing and exchange with other users.

[0168] Music Creation and Performance: In the virtual music studio, users can create and perform music using virtual instruments. The system provides highly realistic models of various instruments and a library of sound effects. Users can manipulate instruments using gestures or controllers, and hear the sound effects in real time. Furthermore, users can form virtual bands with other music enthusiasts to perform and exchange ideas.

[0169] Dance Choreography and Performance: Create a virtual dance rehearsal room where users can choreograph and practice dance moves. The system captures the user's body movements and translates them into the movements of a virtual dancer, allowing users to view and adjust the dance in real time. Users can also invite other users to join in a dance performance, record and share the performance video.

[0170] 3) Virtual teaching and training

[0171] Virtual Classrooms and Courses: Teachers can create their own classrooms in virtual reality for online instruction. Using a variety of teaching tools, including virtual whiteboards, PowerPoint presentations, and video playback, combined with voice instructions and real-time interaction, this provides students with an immersive teaching experience. Students can ask questions, discuss, submit assignments, and fully communicate with teachers and other students in the virtual classroom.

[0172] Skills training and simulation: Virtual reality technology is used to simulate and train aesthetic skills that require practical application, such as painting techniques, instrument playing skills, and dance movement standards. Students can practice repeatedly in a virtual environment, and the system provides real-time feedback and guidance to help them master the correct skills and methods.

[0173] Sharing Teaching Resources and Case Studies: On the virtual aesthetic education platform, teachers can share their own teaching resources and excellent cases, building a rich teaching resource library. Other teachers can draw inspiration and learn from these resources to improve their teaching quality. Students can also access these resources, broadening their learning channels and obtaining more learning materials and references.

[0174] Therefore, the present invention provides an aesthetic education teaching interactive system based on virtual reality, and constructs various aesthetic education teaching scenarios through VR / AR and other technologies, so that students can experience works of art, perform diversified artistic creation and receive guidance in virtual scenes. At the same time, teachers can also conduct real-time remote interaction and guidance, breaking the limitations of physical distance and realizing immersive art learning.

[0175] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0176] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. An interactive system for aesthetic education based on virtual reality, characterized in that: include: The VR / AR scene engine module is used to perform scene modeling and prop modeling to obtain a scene model. Based on the scene model, it performs mapping, material processing, motion capture animation generation, game engine import, interaction and scene configuration to complete the production of VR / AR scene assets; The motion capture module is used to build hardware equipment and software tools to capture and locate students' motions, design a teaching management background, conduct remote interaction and teaching between teachers and students, and record the first interaction data and student performance results in real time to obtain the first student learning report and feedback data; A backend construction module is used to preset multiple scenario libraries, set scenario functions, course tasks and personalized tasks of the scenario libraries, and based on the scenario libraries, collect second interaction data in real time, display students' learning progress, and configure teacher guidance tools to guide students' learning; The interaction and output module is used to configure virtual creation tools, record students' virtual creation behaviors, and obtain the final works, and then classify, store and export the final works and design interactive triggers; The exhibition hall display module is used to build a virtual art exhibition hall, conduct a virtual display of the final works, and score and analyze the final works to provide feedback on the works; Among them, the VR / AR scene engine module, the motion capture module, the background construction module, the interaction and output module and the exhibition hall display module are interconnected.

2. The interactive system for aesthetic education based on virtual reality according to claim 1, characterized in that: The VR / AR scene engine module includes: A scene creation unit is used to import an artistic scene into a 3D modeling software for model generation to obtain a scene model, and then use segmented modeling technology to divide the scene model into multiple independent modules to complete the scene modeling, and then use low-polygon modeling technology to model props in the scene model; A scene processing unit is used to use 3D texture production software to perform texture drawing and material processing on the scene model, and to set the reflection, transparency and texture of the lights and walls in the scene model. Then, through the motion capture equipment, the natural movement data of the character is obtained, and the 3D character animation software is used to perform bone mapping and smoothing processing on the natural movement data of the character to create a virtual guide character or interactive object; An engine import and configuration unit is used to select a game development engine, import the scene model into the selected game development engine, perform scene and prop management, player interaction logic design, and scene physical property configuration to obtain a VR / AR engine, and then perform scene light source debugging and rendering performance optimization to complete scene construction; An AR mode expansion unit, which utilizes an AR software development kit and a mobile camera to perform environmental scanning and plane positioning of the art scene to adjust the proportional consistency between virtual and real objects. Furthermore, in AR mode, it adds audio-visual interaction with artworks and virtual painting capabilities. The audio-visual interaction includes voice commentary of artworks and the configuration of behavioral logic for three-dimensional dynamic objects. The virtual painting function allows for drawing using a virtual brush. An integrated testing unit is used to verify the display effects of virtual assets on different hardware and adjust the scene ratio.

3. The interactive system for aesthetic education based on virtual reality according to claim 2, characterized in that: The motion capture module includes: The capture framework unit is used to select positioning equipment and software tools, use the positioning equipment to capture head and hand motions, and use the software tools to convert the captured motion coordinates into skeletal animation to obtain a standard skeletal structure. Then, a motion smoothing algorithm is used to eliminate capture jitter and data delay. The positioning equipment includes a VR controller and helmet, motion capture gloves, a full-body positioning device, and a mobile camera. The software functions include an SDK and a kinematics library. A data mapping unit is used to transmit the captured movements to the VR / AR engine in real time for physical mapping and animation generation, and to optimize rendering in combination with a depth sensor. Furthermore, a skeletal mapping engine is used to automatically identify and provide feedback on incorrect postures. A remote interactive unit is used to develop a teaching management background for real-time monitoring and capturing of movements, and to configure the teaching management background with multi-view panoramic viewing functions, movement correction functions, real-time mapping functions, and real-time learning functions; The data recording and feedback unit is used to record the student's gesture path, skeletal data and operation completion time in real time to obtain first interaction data, combine the first interaction data with the student's performance results, generate a first student learning report, and train an action-based temporal modeling network. The gesture prediction function is added to the training process to obtain a feedback model. The feedback model is used to analyze the student's abnormal posture to obtain feedback data; wherein, the first student learning report includes dance movement scores, virtual painting analysis and simulated instrument performance matching index.

4. The interactive system for aesthetic education based on virtual reality according to claim 3, characterized in that: The panoramic viewing function is used to switch the virtual camera perspective to observe student performance from multiple angles; The action correction function is used to convert the teacher's input voice into text or animation to guide students to correct their movements, realizing the combination of voice input and virtual scenes; The real-time mapping function is used to track the student's real-world position through the AR development tool and SLAM technology to dynamically overlay virtual art elements into the camera's field of view; The real-time learning function is used to mark and modify students' virtual paintings in a two-way manner in an AR environment, and to display the target movement trajectory through AR animation, so that students can follow the virtual dynamic arrows to complete the movements, thereby realizing students' painting training and dance training.

5. The interactive system for aesthetic education based on virtual reality according to claim 4, characterized in that: The backend building module includes: A virtual teaching room unit is used to preset a variety of scene libraries and set scene functions, course tasks and personalized tasks of the scene libraries; A progress display unit is used to assign student login permissions, collect the student's immersion time, task progress, and interaction point trigger frequency after entering the virtual scene, obtain second interaction data, store real-time behavior in a NoSQL database and task completion status in a MySQL database based on the second interaction data, and generate a second student learning report showing learning progress based on the second interaction data; the second student learning report includes task execution time, task completeness, and action completion accuracy; The guidance tool unit is used to configure a thumbnail map including the multi-perspective panoramic viewing function, configure a guidance tool including the action correction function, the real-time mapping function and the real-time learning function, and add collaborative tasks in the virtual scene to realize a multi-student collaborative mode and a competition mode.

6. The virtual reality-based aesthetic education interactive system according to claim 5, characterized in that: The scene library includes a dance class scene, an art gallery scene, and a concert hall scene. The dance class scene includes a virtual floor and a dynamic dance guide model, the art gallery scene includes a variety of artworks, and the concert hall scene includes a stage, a virtual audience, and a virtual instrument simulation. The scene functions include prop addition, lighting adjustment, material adjustment and teaching script editor.

7. The interactive system for aesthetic education based on virtual reality according to claim 6, characterized in that: The interaction and output module includes: An art creation tool unit, which is used to configure virtual tools, use Unity Physics or Unreal Engine Physics to provide physical feedback of the virtual tools, and bind the handle to the virtual tools through a positioning device to achieve the integration of motion capture and virtual space; A creation recording unit is used to record the creation trajectory, use a compression algorithm to downsample redundant trajectory points in the creation trajectory, store the downsampled creation trajectory in a NoSQL database, and then classify, store and export the completed final works; An interactive triggering unit is used to create a virtual interactive object based on the final work, and to design a node triggering mechanism based on the virtual interactive object to realize the function of automatically popping up relevant instructions after touching the virtual interactive object; The AR mode expansion unit is used to use AR software development tools to overlay a virtual canvas onto a desktop or wall in a real scene, and to record the overlay process of the virtual canvas in real time in the form of video, and to automatically save the painting effect after the virtual canvas is overlaid in the form of pictures.

8. The interactive system for aesthetic education based on virtual reality according to claim 7, characterized in that: The creation trajectory includes three-dimensional coordinates, creation data points and a point sequence of the creation data points. The point sequence includes a timestamp, a manual creation path and tool settings. The final work includes a painting, a sculpture and a music work. The painting is in a picture format, the sculpture is in a 3D model format, and the music work is in a MIDI data file format including a performance trajectory.

9. The interactive system for aesthetic education based on virtual reality according to claim 8, characterized in that: The exhibition hall display module includes: A results archiving and conversion unit is used to integrate the creative trajectory and the final work to obtain comprehensive creative data, classify and store the comprehensive creative data according to preset work archiving rules, and call API and AI image enhancement technology to export the comprehensive creative data; A virtual exhibition hall design unit is used to build a virtual art exhibition hall, design the work display logic, and based on the virtual art exhibition hall, design a virtual tour and real-time comment mode to realize the explanation and comment of the works, and generate the work comment record; The review and scoring unit is used to perform teacher scoring, student scoring and AI scoring on the final work, and then conduct a comprehensive analysis of the scoring results to generate work recommendations.

10. The interactive system for aesthetic education based on virtual reality according to claim 9, characterized in that: The work archiving rules are used to generate an identification ID and embed metadata based on the final work; the identification ID includes the student ID, task name, submission time and file type, and the metadata includes student information, scoring dimensions and creation tool parameters.

Images