Virtual reality technology-based cultural and creative experience system

Abstract

The invention relates to the technical field of virtual reality and cultural creativity, and discloses a virtual reality technology-based cultural creativity experience system, which comprises a resource arrangement module, an activity design module, a VR situation design module, a VR interactive design module and a creativity generation auxiliary module. According to the cultural and creative experience system based on the virtual reality technology, physical characteristics, manufacturing process parameters and cultural background knowledge of handicrafts are systematically integrated through a physical layer knowledge unit, a technical layer knowledge unit and a cultural layer knowledge unit in a resource arrangement module, specifically, the physical layer adopts a convolutional neural network to extract characteristics such as material reflectivity, and the physical layer adopts a convolutional neural network to extract characteristics such as material reflectivity; the technical layer analyzes technological parameters by using a long-short-term memory network, the cultural layer constructs an association network by means of a word vector model, the multi-level knowledge base structure enables a user to obtain comprehensive handicraft information in the experience process, the knowledge inheritance efficiency is greatly improved, in this way, the depth and breadth of learning are enhanced, and the learning efficiency is improved. And the propagation and inheritance of culture are also promoted.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual reality and cultural and creative technologies, and particularly to a cultural and creative experience system based on virtual reality technology. Background Art

[0002] Virtual reality technology (VR) provides an interactive experience for users through immersive three-dimensional environment simulation; the cultural and creative industry (cultural and creative) is committed to the inheritance and innovation of traditional handicrafts. The combination of the two can break through the limitations of traditional teaching and design, enabling users to learn the handicraft production process, explore the cultural background, and carry out creative design in a virtual environment.

[0003] However, the existing VR cultural and creative systems still have the following problems: traditional systems fail to systematically integrate the physical characteristics, production process parameters, and cultural background knowledge of handicrafts, resulting in low knowledge inheritance efficiency; the interaction method is single, only supporting basic gesture operations, lacking tactile feedback and multi-language voice interaction, and the user experience is limited; the innovative design assistance function is weak, and it is unable to generate design schemes with both aesthetic and engineering feasibility through intelligent algorithms. These problems seriously restrict the efficiency and quality of handicraft inheritance and innovation. Therefore, a cultural and creative experience system based on virtual reality technology is proposed. Summary of the Invention

[0004] (I) Technical Problems to be Solved

[0005] Aiming at the deficiencies of the existing technology, the present invention provides a cultural and creative experience system based on virtual reality technology, which has the advantages of significantly improving the efficiency of handicraft inheritance, enhancing the user's immersion, and generating design schemes with both aesthetic value and engineering feasibility through a structured knowledge base and multi-modal interaction technology, and solves the following problems existing in the existing VR cultural and creative systems: traditional systems fail to systematically integrate the physical characteristics, production process parameters, and cultural background knowledge of handicrafts, resulting in low knowledge inheritance efficiency; the interaction method is single, only supporting basic gesture operations, lacking tactile feedback and multi-language voice interaction, and the user experience is limited; the innovative design assistance function is weak, and it is unable to generate design schemes with both aesthetic and engineering feasibility through intelligent algorithms. These problems seriously restrict the efficiency and quality of handicraft inheritance and innovation.

[0006] (II) Technical Solutions

[0007] To achieve the above-mentioned purpose of significantly improving the efficiency of handicraft inheritance, enhancing the user's immersion, and generating design schemes with both aesthetic value and engineering feasibility through a structured knowledge base and multi-modal interaction technology, the present invention provides the following technical solutions: A cultural and creative experience system based on virtual reality technology includes a resource sorting module, an activity design module, a VR scenario design module, a VR interaction design module, and a creative generation assistance module, and further includes:

[0008] The resource sorting module constructs a handicraft cultural and creative knowledge base through a knowledge graph, specifically including:

[0009] S1.1: Physical layer knowledge unit. Extract the material reflectivity, color space coordinates (CIELAB values), 3D point cloud coordinates, and pattern Fourier transform features of handicrafts through a convolutional neural network (CNN), and store them in the form of a triangular mesh model;

[0010] S1.2: Skill layer knowledge unit. Analyze the expert interview text and physical parameters collected by sensors through a long short-term memory network (LSTM) to generate structured data containing step numbers, tool models, and process parameters (temperature, pressure, time);

[0011] S1.3: Cultural layer knowledge unit. Parse the literature, folk records, and oral history texts related to handicrafts through a word vector model (Word2Vec) to construct an association network containing cultural symbols, historical events, and geographical distributions;

[0012] The activity design module designs experience activities based on the Kolb experiential learning model, specifically including:

[0013] S2.1: Experience stage. Simulate the traditional handicraft production environment through a virtual reality scene to guide users to complete tasks such as virtual material selection, tool operation, and finished product inspection;

[0014] S2.2: Reflection stage. Use a voice interaction system to ask users about their understanding of the operation steps, and record the user operation time, number of errors, and voice response text;

[0015] S2.3: Abstraction stage. Combine user behavior data with the content of the knowledge base to generate a text report containing process principles and cultural backgrounds;

[0016] S2.4: Verification stage. Simulate the structural strength designed by users through finite element analysis, and output a stress distribution map and a feasibility score;

[0017] The VR scenario design module uses ray tracing technology to generate a dynamic virtual scene, including:

[0018] Scene generation unit: Build scenes such as historical blocks, workshops, marketplaces, and sacrificial sites based on 3D modeling data, and simulate the movement trajectories of raindrops, snowflakes, flames, and water flows through a particle system;

[0019] Environment fusion unit: Real-time scan the real environment where the user is located through the SLAM algorithm, and superimpose virtual elements onto the real space, with a superimposition accuracy ≤ 5 cm;

[0020] The VR interaction design module specifically includes:

[0021] Gesture recognition unit: uses the MobileNet-LSTM network to recognize users' gesture actions in real time, with a recognition accuracy of ≥95% and a latency of ≤0.2 seconds;

[0022] Haptic feedback unit: simulates the touch of tool operation through an electromagnetic vibrator and a six-axis force sensor, with a vibration frequency range of 10 Hz - 1000 Hz and an amplitude range of 0.1 mm - 5 mm;

[0023] Voice interaction unit: parses users' instructions based on the DeepSpeech model, supports Chinese, English, and Japanese input, with a recognition accuracy of ≥92%;

[0024] The creative generation assistance module optimizes the user design through a reinforcement learning algorithm:

[0025] Inputs the sketch or 3D model of the user design, and generates improved solutions through a generative adversarial network (GAN);

[0026] Recommends design parameters based on the user's historical operation data, including color matching ratio and structural symmetry coefficient.

[0027] Preferably, the technical skill layer knowledge unit of the resource sorting module adopts a dual-channel analysis method, specifically including:

[0028] The first channel: parses the expert interview text through the BERT model to extract keywords of process parameters;

[0029] The second channel: collects torque data of tool operation through a piezoelectric sensor to build a process parameter database;

[0030] Fuses the data of the two channels to generate a composite knowledge entry including text description, parameter range, and operation video.

[0031] Preferably, the scene generation unit of the VR scenario design module adopts a hierarchical rendering technique, specifically including:

[0032] The base layer: renders static scenes including buildings, backgrounds, streets, and decorative elements;

[0033] The dynamic layer: superimposes a particle system to simulate raindrops (density 1000 / m 3 , speed 0.5 m / s), snowflakes (density 500 / m 3 , speed 0.3 m / s), and flame effects;

[0034] The interaction layer: renders virtual objects generated by users' operations in real time (such as the effect of virtual clay sculpting), with a resolution of 4K;

[0035] The hierarchical rendering technique is implemented through GPU parallel computing, with a frame rate of ≥60 fps and a latency of ≤20 ms.

[0036] Preferably, the gesture recognition unit adopts a multi-modal fusion algorithm, specifically including:

[0037] The input data includes RGB images (resolution 1080p), depth images (resolution 720p), and inertial sensor data (accelerometer, gyroscope);

[0038] Use the LSTM network to extract spatio-temporal features from the sequence data;

[0039] Enhance the recognition weight of key actions through the attention mechanism;

[0040] Output action instructions, including "pinch tool" and "rotate object".

[0041] Preferably, the haptic feedback unit includes an adaptive damping adjustment module, specifically including:

[0042] Dynamically adjust the frequency and amplitude of the electromagnetic vibrator according to the physical parameters of the virtual operation (material Young's modulus, tool type);

[0043] Real-time detect the user's operation force through a six-axis force sensor, and the feedback damping force is proportional to the stiffness of the virtual material.

[0044] Preferably, the GAN network of the creative generation assistance module adopts a dual discriminator architecture, specifically including:

[0045] The first discriminator: evaluate the aesthetic value of the design, including the symmetry coefficient (0 - 1) and color coordination (color difference ΔE ≤ 3);

[0046] The second discriminator: evaluate the feasibility of the design, including structural strength (stress ≤ material yield strength) and material applicability (thermal expansion coefficient matching);

[0047] The generator optimizes the output through adversarial training, and finally outputs a design scheme that meets the user's preferences and is feasible.

[0048] Preferably, the system further includes a cross-platform collaboration module, supporting:

[0049] Multiple users access the same virtual scene through different terminals, including PCs, HMDS, and mobile devices;

[0050] Real-time synchronize the user operation data, and the synchronization delay ≤ 50ms;

[0051] Record the user's creative contributions through blockchain technology, and generate a digital copyright certificate containing a timestamp and a hash value.

[0052] Preferably, the physical layer knowledge unit of the resource sorting module adopts a multi-view three-dimensional reconstruction technology, specifically including:

[0053] Collect 360° images of handicrafts through an 8-camera array (resolution 12 million pixels).

[0054] Use the Structure from Motion (SfM) algorithm to generate a high-precision three-dimensional model (point cloud density ≥ 1 million points / m²).

[0055] Restore the surface texture of the material through a spectral reflection modeling algorithm (wavelength range 400 - 700 nm).

[0056] (III) Beneficial effects

[0057] Compared with the prior art, the present invention provides a cultural and creative experience system based on virtual reality technology, having the following beneficial effects:

[0058] 1. In this cultural and creative experience system based on virtual reality technology, the physical characteristics, production process parameters, and cultural background knowledge of handicrafts are systematically integrated through the knowledge units of the physical layer, technique layer, and cultural layer in the resource collation module. Specifically, in the physical layer, a convolutional neural network is used to extract characteristics such as material reflectivity; in the technique layer, a long short-term memory network is used to analyze process parameters; and in the cultural layer, a word vector model is used to construct an association network. This multi-level knowledge base structure enables users to obtain comprehensive handicraft information during the experience process, greatly improving the knowledge inheritance efficiency. In this way, not only the depth and breadth of learning are enhanced, but also the dissemination and inheritance of culture are promoted.

[0059] 2. This cultural and creative experience system based on virtual reality technology introduces a gesture recognition unit, a tactile feedback unit, and a voice interaction unit, enriching the user's interaction methods. The gesture recognition unit uses a MobileNet-LSTM network to achieve accurate gesture action capture, while the tactile feedback unit adjusts the frequency and amplitude of the electromagnetic vibrator in real time according to virtual operations to simulate the real tool operation feeling. In addition, the voice interaction unit supports multiple language inputs and can accurately parse user instructions. These units cooperate with each other to provide users with an immersive interactive environment, significantly enhancing the user experience. For example, during the simulation of pottery throwing, users can not only control the turntable speed through gestures, but also feel the change in resistance when their fingers touch the clay, and at the same time ask about the next steps in voice. Description of the drawings

[0060] Figure 1 It is a schematic structural diagram of the cultural and creative experience system of the present invention. Detailed implementation manners

[0061] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the embodiments and drawings of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0062] Please refer to Figure 1 , a cultural and creative experience system based on virtual reality technology, including a resource sorting module, an activity design module, a VR scenario design module, a VR interaction design module, and a creative generation assistance module, further including:

[0063] The resource sorting module constructs a handicraft cultural and creative knowledge base through a knowledge graph, specifically including:

[0064] S1.1: Physical layer knowledge unit, extracting the material reflectivity, color space coordinates (CIELAB values), three-dimensional point cloud coordinates, and pattern Fourier transform features of handicrafts through a convolutional neural network (CNN), and storing them in the form of a triangular mesh model;

[0065] S1.2: Skill layer knowledge unit, analyzing the expert interview text and physical parameters collected by sensors through a long short-term memory network (LSTM) to generate structured data including step numbers, tool models, and process parameters (temperature, pressure, time);

[0066] S1.3: Cultural layer knowledge unit, parsing handicraft-related literature, folk records, and oral history texts through a word vector model (Word2Vec) to construct an association network including cultural symbols, historical events, and geographical distributions;

[0067] The activity design module designs experience activities based on the Kolb experiential learning model, specifically including:

[0068] S2.1: Experience stage: Simulating the traditional handicraft production environment through a virtual reality scene to guide users to complete tasks such as virtual material selection, tool operation, and finished product inspection;

[0069] S2.2: Reflection stage: Asking users about their understanding of the operation steps through a voice interaction system, and recording the user operation time, error times, and voice response text;

[0070] S2.3: Abstraction stage: Combining user behavior data with the content of the knowledge base to generate a text report including process principles and cultural backgrounds;

[0071] S2.4: Verification stage: Simulating the structural strength designed by users through finite element analysis and outputting a stress distribution diagram and a feasibility score;

[0072] The VR scenario design module uses ray tracing technology to generate a dynamic virtual scene, including:

[0073] Scene generation unit: Construct scenes such as historical blocks, workshops, marketplaces, and sacrificial sites based on 3D modeling data, and simulate the movement trajectories of raindrops, snowflakes, flames, and water flows through a particle system;

[0074] Environment fusion unit: Real-time scan the real environment where the user is located through the SLAM algorithm, and superimpose virtual elements on the real space, with a superimposition accuracy ≤ 5 cm;

[0075] The VR interaction design module specifically includes:

[0076] Gesture recognition unit: Use the MobileNet-LSTM network to real-time recognize the user's gesture actions, with a recognition accuracy ≥ 95%, and a delay ≤ 0.2 seconds;

[0077] Haptic feedback unit: Simulate the touch of tool operation through an electromagnetic vibrator and a six-axis force sensor, with a vibration frequency range of 10 Hz - 1000 Hz and an amplitude range of 0.1 mm - 5 mm;

[0078] Voice interaction unit: Parse the user's instructions based on the DeepSpeech model, support Chinese, English, and Japanese input, with a recognition accuracy ≥ 92%;

[0079] The creative generation assistance module optimizes the user's design through a reinforcement learning algorithm:

[0080] Input the sketch or 3D model designed by the user, and generate an improved solution through a generative adversarial network (GAN);

[0081] Recommend design parameters based on the user's historical operation data, including color matching ratio and structural symmetry coefficient.

[0082] Example 1:

[0083] This example details the model construction, data processing flow, and hardware co-design of the physical layer knowledge unit, focuses on solving the problem of high-precision digitization of the material, geometry, and pattern features of traditional handicrafts, and realizes data interaction with other modules through a standardized interface.

[0084] (1) Model construction:

[0085] The core of the physical layer knowledge unit is an improved ResNet-50 convolutional neural network, which contains 16 residual blocks. Each residual block adopts a stacked structure of two 3×3 convolutional layers with a kernel stride of 1 and a padding of 1 to maintain the size of the feature map. The activation function uses Leaky ReLU (negative slope 0.3) to reduce the vanishing gradient problem. The network input is an RGB-D image (the resolution of the RGB channel is 4096×2304 pixels, and the resolution of the depth channel is 1024×768 pixels). The depth data is obtained by a structured light scanner (Artec Eva) with an accuracy of 0.1 mm. The outputs include:

[0086] Material reflectivity: Parametrically represented by the differential reflectance model (Bidirectional Reflectance Distribution Function, BRDF), including the diffuse reflection coefficient (0.1 - 0.9), the specular reflection coefficient (0.01 - 0.5), and the Fresnel term (using the Schlick approximation, the reflectivity varies with the incident angle).

[0087] CIELAB color coordinates: Through the conversion of the Lab color space, the range of L is 0 - 100, and the ranges of a and b* are -128 to 127. The quantization accuracy is 0.1 unit.

[0088] 3D point cloud: Through the dense reconstruction of multi-view images (using the COLMAP software), the point cloud density is 1000 points per cubic centimeter. After downsampling, a triangular mesh model is generated (the number of patches ≤ 500,000), and the vertex coordinates are reserved to 6 decimal places.

[0089] Pattern spectral features: The frequency domain amplitude spectrum (resolution 256×256) of the pattern area is extracted through Fourier transform. The frequency band is limited to 0 - 50 cycles / mm, and a Butterworth high-pass filter (cutoff frequency 0.5 cycles / mm) is used to remove low-frequency noise.

[0090] (2) Data processing flow:

[0091] Preprocessing stage:

[0092] Image correction: Use the calibration matrix of OpenCV (the internal parameter matrix K = [[2304,0,1024],[0,2304,1536],[0,0,1]]) for lens distortion correction, with the radial distortion coefficients k1 = -0.0005 and k2 = 0.0001.

[0093] Depth alignment: Align the RGB and depth images through the ICP algorithm (the number of iterations ≤ 100, the convergence threshold 0.01 mm), with an error < 0.5 pixels.

[0094] Feature extraction stage:

[0095] Material analysis: After the BRDF parameters are output by the network forward propagation, the reflectance consistency is verified through Monte Carlo integration (sampling number 1024) to ensure the conservation of specular reflection and diffuse reflection energy (error < 5%).

[0096] 3D reconstruction: After the point cloud is estimated by normal vectors (using the PCA algorithm, window radius 5mm), a closed mesh is generated through the Poisson surface reconstruction algorithm (depth 10, voxel size 0.2mm), and the hole filling rate > 99%.

[0097] Storage and interface:

[0098] Database design: The SQLite database includes a material table (storing BRDF parameters), a color table (CIELAB coordinates), a geometry table (mesh vertex / facet index), and a pattern table (spectral feature vectors).

[0099] Data interface: An API is provided through the gRPC protocol (using Protobuf serialization), supporting other modules to call material parameters (such as the VR rendering module calling BRDF parameters to generate realistic lighting).

[0100] (III) Hardware co - design:

[0101] Data acquisition end: An industrial - grade camera pan - tilt (PHOTONICSCIENCE PCT - 180) is equipped with a Phase One IQ4150MP camera. The rotation accuracy of the pan - tilt is ±0.01°, and the multi - axis movement is controlled through the EtherCAT bus (sampling rate 100Hz). The structured light scanner (Artec Eva) is triggered synchronously with the camera (trigger signal delay < 1ms).

[0102] Computing unit: An NVIDIA Tesla V100 GPU (16GB video memory) deploys a TensorRT - optimized model, and the inference latency ≤ 200ms / image; the CPU (Intel Xeon E5 - 2686 v4, 18 cores) processes point cloud registration and mesh reconstruction, and the memory occupancy ≤ 12GB.

[0103] (IV) Connection with other modules:

[0104] Provided to the activity design module: 3D model mesh data (GLB format) for VR scene construction, and material parameters (BRDF) for physical rendering;

[0105] Feedback to the cultural layer knowledge unit: Association of cultural symbols through texture features (spectral data) (such as the "ice crack pattern" and the cultural background of Song Dynasty celadon).

[0106] Example 2:

[0107] This embodiment elaborates in detail the dual-channel LSTM network architecture, sensor hardware design, and data fusion algorithm for knowledge units at the skill level, aiming to extract structured parameters during the handicraft production process and provide real-time process constraints for the VR interaction module through a standardized interface.

[0108] (1) Algorithm design:

[0109] At the skill level, a dual-channel LSTM network is adopted. The main channel processes text data, and the auxiliary channel processes sensor data. The input dimension of the main channel is a word vector (256 dimensions, trained by Word2Vec with a window size of 5 and 100 iterations). Context features are extracted through a bidirectional LSTM (with 512 hidden units). The auxiliary channel inputs sensor data (128 dimensions, including temperature, pressure, acceleration, etc.), and captures time series features through a unidirectional LSTM (with 256 hidden units). The outputs of the two channels are weighted and fused through an attention mechanism (Bahdanau attention), and finally, structured process parameters are output (such as "throwing speed: 12 - 15 rpm, duration: 45 ± 5 s").

[0110] (2) Sensor system design:

[0111] Hardware configuration:

[0112] Temperature sensor: Pt100 (accuracy ±0.1 °C, range 0 - 1200 °C, sampling rate 10 Hz), and data is transmitted through an NI cDAQ-9188 acquisition card (16-bit ADC).

[0113] Force / torque sensor: ATI Nano17 (six-axis force sensor, range ±50 N, resolution 0.01 N, sampling rate 1 kHz), and high-frequency noise is eliminated through digital filtering (Butterworth low-pass filter with a cut-off frequency of 100 Hz).

[0114] IMU module: MPU-9250 (angular velocity accuracy ±0.06 ° / s, acceleration accuracy ±0.004 g), sampling rate 100 Hz. The data of the gyroscope and accelerometer are fused through a complementary filter (complementary coefficient 0.98) to calculate the device attitude (Euler angle accuracy ±0.5 °).

[0115] (3) Data fusion algorithm:

[0116] Kalman filter: The state vector includes temperature, pressure, and three-dimensional position. The covariance matrix P is initialized as a diagonal matrix (diagonal elements are 0.1), the process noise Q = diag([0.01, 0.01, 0.001]), and the measurement noise R = diag([0.001, 0.001, 0.0001]).

[0117] Time alignment: Align sensor data with different sampling rates to a 100Hz reference clock through linear interpolation.

[0118] (IV) Process parameter generation process:

[0119] Text processing: After the expert interview text is segmented by Jieba, key operation steps (such as "when drawing the billet, it needs to rotate counterclockwise") are extracted through the BERT model (pre-trained Chinese BERT-Base), and the process step labels (such as "drawing billet forming") are labeled using the conditional random field (CRF).

[0120] Parameter mapping: Through the weighted concatenation (weight 0.6:0.4) of the hidden state (dimension 512) output by the LSTM and the sensor data (dimension 128), it is input into the fully connected layer (ReLU activation, Dropout rate 0.5) to generate standardized parameters.

[0121] Verification and storage: The parameters are verified through the rule engine (such as "the firing temperature needs to be >1200°C") and stored in MongoDB (JSON format), supporting real-time calls by the activity design module.

[0122] (V) Interaction with other modules:

[0123] Provide to the VR interaction module: Real-time process parameters (such as "drawing billet rotation speed") are used to constrain user operations (such as triggering a warning when the rotation speed > 15rpm);

[0124] Feedback to the cultural layer knowledge unit: Associate historical process documents through process step labels (such as "wood firing technique" and the technical background of the Ming Dynasty dragon kiln).

[0125] Example III

[0126] This example focuses on the design of the TransE model and the construction of the knowledge graph for the cultural layer knowledge unit. Through multi-source text data, it mines the association network of cultural symbols, historical events, and geographical distributions, and provides cultural narrative support for the VR scenario module.

[0127] (I) Knowledge graph construction:

[0128] The cultural layer uses an improved TransE model with an entity embedding dimension of 200 and a relationship embedding dimension of 100. The model constructs an association network through the following steps:

[0129] Entity recognition:

[0130] Named entity recognition (NER): Use the BERT-Base-Chinese model (pre-trained on the Chinese corpus), and identify entity types (such as "cultural symbols", "historical events") through the CRF decoder.

[0131] Disambiguation processing: Calculate similarity through Word2Vec word vectors (threshold 0.8), and merge synonyms (such as "blue and white porcelain" and "blue and white ground porcelain").

[0132] Relationship mining:

[0133] Explicit relationships: Extract the subject-predicate-object structure through dependency syntax analysis (Stanford Parser) (such as "Jingdezhen" → "established" → "imperial kiln").

[0134] Implicit relationships: Mine indirect associations through co-occurrence statistics (pointwise mutual information PMI > 3.5) (such as "Longquan Kiln" and "Southern Song Dynasty" are associated because they co-occur in the literature).

[0135] Embedding training:

[0136] Loss function: Use Smooth L1 Loss, and the optimization goal is to maximize the difference between the positive example score and the negative example score (margin = 1).

[0137] Training parameters: Adam optimizer (learning rate 0.001, weight decay 0.0001), batch size 128, negative sampling ratio 5:1.

[0138] (2) Graph storage and query:

[0139] Neo4j database: Node types include "cultural symbols" (attributes: name, era, distribution area), "historical events" (attributes: time, location, associated figures), "geographical distribution" (attributes: longitude and latitude, administrative division), and relationship types include "has" (cultural symbol → historical event), "is located in" (cultural symbol → geographical distribution).

[0140] Query optimization: Accelerate SPARQL queries through indexes (such as "cultural symbol.name"). For example, querying "all Song Dynasty events related to 'blue and white porcelain'" requires a response of < 200ms.

[0141] (3) Linkage with other modules:

[0142] Provide to the VR scenario module: Association information of cultural symbols (such as the background story of "ice crack pattern" and "Southern Song Dynasty imperial kiln") for generating narrative content of virtual scenarios.

[0143] Feedback to the activity design module: Trigger cultural background explanations (such as "the intertwined pattern symbolizes endless growth and originates from Buddhist art in the Tang Dynasty") through user operation data (such as selecting "intertwined pattern").

[0144] Example 4:

[0145] This embodiment realizes the full - process closed - loop of the Kolb experiential learning model through virtual reality technology, covering physical engine simulation, multimodal data acquisition, and finite element analysis, and improves the learning effect of users through a feedback mechanism.

[0146] (I) VR Scene Construction:

[0147] Physical Engine Configuration: NVIDIA PhysX 5.0 is used to implement the simulation of non - linear elastic bodies for clay shaping. The material parameters include:

[0148] Young's modulus: 1.5 MPa (simulating the plastic deformation of clay)

[0149] Poisson's ratio: 0.45 (approximate volume conservation)

[0150] Viscous damping coefficient: 0.8 N·s / m (simulating finger extrusion resistance)

[0151] Rendering Optimization: The Nanite virtualized geometry technology of Unreal Engine 5 dynamically splits high - precision models (number of polygons > 1 million) into micropolygons to ensure that the video memory occupancy ≤ 12 GB during 60 FPS rendering.

[0152] (II) Data Feedback Mechanism:

[0153] Data Acquisition in the Experience Stage:

[0154] Operation Trajectory: The Leap Motion sensor (6 degrees of freedom, sampling rate 120 Hz) is used to record the pose (XYZ coordinates, Euler angles) of the user's hand and store it in CSV format.

[0155] Virtual Material State: The volume of clay (error < 0.5%) and surface roughness (calculated through normal distribution, threshold 0.1 - 1.0) are recorded for each frame.

[0156] Speech Processing in the Reflection Stage:

[0157] Speech Transcription: The Google Cloud Speech - to - Text API (recognition accuracy 98%) is used to convert the user's answer into text, and then sentiment analysis (positive / neutral / negative) and keyword extraction (such as "throwing pottery", "shrinkage rate") are performed through the BERT model (pre - trained Chinese RoBERTa).

[0158] Finite Element Analysis in the Verification Stage:

[0159] Thermo-mechanical coupling simulation: In ANSYS Mechanical APDL, the temperature field (firing curve at 1200 °C) is set, and the stress caused by thermal expansion is calculated (safety threshold ≤ 35 MPa). The stress nephogram (resolution 1024×768) and feasibility score (0 - 10 points, based on the stress distribution uniformity) are fed back to the user.

[0160] (III) System closed-loop design:

[0161] Data flow: User operation data → Activity design module → Cultural layer knowledge unit → Generate feedback report → VR interface display.

[0162] Error correction mechanism: When the user operation parameters exceed the safe range (such as rotational speed > 15 rpm), the system triggers haptic feedback (applying a reverse force of 0.5 N by the HaptX glove) and gives a voice prompt "The rotational speed is too high. Please reduce the speed."

[0163] Example 5:

[0164] This example details the co-design of ray tracing rendering and haptic feedback hardware, achieving immersive interaction through the integration of a high-precision physics engine and sensors, ensuring the realism and safety of virtual operations.

[0165] (I) Ray tracing configuration:

[0166] Material system:

[0167] BRDF model: Anisotropic specular highlights (Anisotropic GGX distribution, anisotropy parameter 0.8) simulate the directional reflection of wood grain.

[0168] Subsurface scattering: The Henyey-Greenstein phase function (g = 0.7) is used to simulate the internal light scattering of jade, with a transparency coefficient of 0.3.

[0169] Performance optimization:

[0170] BVH acceleration structure: The tree height ≤ 20 layers, the number of nodes ≤ 2^20, and dynamic subdivision (updating 10% of the nodes per frame) is used to balance accuracy and performance.

[0171] DLSS super-resolution: Tensor Cores acceleration is enabled to upscale the native resolution of 1080p to 4K, maintaining 60 FPS.

[0172] (II) Haptic feedback design:

[0173] HaptX glove: 256 tactile point array, each point simulates the contact force through a piezoelectric actuator (frequency response 20 - 200 Hz). The haptic signal is generated by the PID control algorithm:

[0174] Control equations:

[0175]

[0176] where K p = 0.5, K i = 0.1, K d = 0.05, and the error e(t) is the difference between the virtual force and the actual tactile force.

[0177] Damping model: The resistance of the virtual tool (such as a carving knife) is calculated by the following formula:

[0178] F 阻尼 = μ·v + λ·v 2

[0179] where μ = 0.8 (linear damping coefficient), γ = 0.3 (nonlinear damping coefficient), and v is the tool speed (m / s).

[0180] (III) Multi-user collaboration mechanism:

[0181] Network synchronization: The UDP protocol (bandwidth 100 Mbps) is used to transmit user pose data, and a prediction-correction algorithm is adopted to compensate for the delay:

[0182] Prediction model: Linear extrapolation (step size 0.1 s), and the error compensation is calculated based on the historical pose change rate.

[0183] Synchronization interval: Synchronization is performed once per frame (1 / 60 s), the position synchronization error < 5 mm, and the attitude synchronization error < 2°.

[0184] Example VI:

[0185] In this example, multi-objective optimization is achieved through a genetic algorithm, an innovative design solution is generated by combining user preferences and physical constraints, and the digital production of handicrafts is realized through parametric modeling and 3D printing technology.

[0186] (I) Algorithm implementation:

[0187] Coding method: The design solution uses parametric modeling (such as the height-diameter ratio and wall thickness distribution of ceramic vessels) and is encoded as a real number vector (dimension 128).

[0188] Genetic operations:

[0189] Crossover: Uniform crossover (probability 0.8), and the offspring genes randomly inherit the loci of parent 1 or 2.

[0190] Mutation: Gaussian mutation (σ = 0.1), and the mutation loci are randomly selected (probability 0.1).

[0191] Multi-objective optimization:

[0192] Aesthetic indicators: Rendering scheme through the StyleGAN2 generator (256×256 resolution), and using VGG-16 to calculate the style similarity with classic works (LPIPS distance ≤ 0.3).

[0193] Engineering indicators: Calculating the structural strength (safety factor ≥ 3) and coefficient of thermal expansion (ΔL / L0 ≤ 0.001) of the finite element model.

[0194] Pareto front: Retaining non-dominated solutions through the NSGA-II algorithm, with a population size of 100 for each generation and 100 generations of iteration.

[0195] (2) User interaction design:

[0196] Parameter constraint interface: The user sets through the slider:

[0197] Volume limit: 0.1L ≤ V ≤ 5L (step size 0.1L).

[0198] Wall thickness constraint: 3mm ≤ t ≤ 10mm (step size 0.1mm).

[0199] Preference adjustment: Real-time adjustment of the aesthetic and engineering weights through the weight slider (initial value 0.6:0.4, step size 0.05).

[0200] Scheme export: The optimal solution is output in STL format, supporting 3D printing (layer thickness 0.1mm, filling rate 20%), and manufacturing through API calls to the Stratasys Fortus 450mc printer.

[0201] (3) Linkage with other modules:

[0202] Call physical layer data: Obtain the material shrinkage rate (such as the clay shrinkage rate of 15%) for engineering indicator calculation.

[0203] Cultural layer input: When the user selects the "blue and white porcelain" style, the algorithm preferentially generates design schemes with traditional patterns.

[0204] To sum up, this cultural and creative experience system based on virtual reality technology systematically integrates the physical characteristics, production process parameters, and cultural background knowledge of handicrafts through the knowledge units of the physical layer, technique layer, and cultural layer in the resource collation module. Specifically, the physical layer uses a convolutional neural network to extract characteristics such as material reflectivity, the technique layer uses a long short-term memory network to analyze process parameters, and the cultural layer constructs an association network with the help of a word vector model. This multi-level knowledge base structure enables users to obtain comprehensive handicraft information during the experience process, greatly improving the efficiency of knowledge inheritance. In this way, not only the depth and breadth of learning are enhanced, but also the dissemination and inheritance of culture are promoted.

[0205] Moreover, this cultural and creative experience system based on virtual reality technology incorporates a gesture recognition unit, a tactile feedback unit, and a voice interaction unit, enriching the user's interaction methods. The gesture recognition unit uses the MobileNet-LSTM network to achieve accurate capture of gesture actions, while the tactile feedback unit adjusts the frequency and amplitude of the electromagnetic vibrator in real time according to virtual operations to simulate the real tool operation feeling. In addition, the voice interaction unit supports multiple language inputs and can accurately parse user instructions. These units cooperate with each other to provide the user with an immersive interactive environment, significantly enhancing the user experience. For example, during the simulation of pottery throwing, the user can not only control the turntable speed through gestures but also feel the change in resistance when the fingers touch the clay, and at the same time ask about the next steps using voice, solving the following problems existing in the existing VR cultural and creative systems: The traditional system fails to systematically integrate the physical characteristics, production process parameters, and cultural background knowledge of handicrafts, resulting in low knowledge inheritance efficiency; the interaction method is single, only supporting basic gesture operations, lacking tactile feedback and multi-language voice interaction, and the user experience is limited; the innovative design assistance function is weak, and it is unable to generate design solutions that combine aesthetics and feasibility through intelligent algorithms. These problems seriously restrict the efficiency and quality of handicraft inheritance and innovation.

[0206] The relevant modules involved in this system are all hardware system modules or functional modules that combine computer software programs or protocols in the prior art with hardware. The computer software programs or protocols themselves involved in this functional module are all well-known technologies to those skilled in the art and are not the improvements of this system; the improvement of this system is the interaction relationship or connection relationship between each module, that is, the overall structure of the system is improved to solve the corresponding technical problems to be solved by this system.

[0207] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A cultural and creative experience system based on virtual reality technology, comprising a resource sorting module, an activity design module, a VR scenario design module, a VR interaction design module, and a creative generation assistance module, characterized in that, It further includes: The resource arrangement module constructs a handicraft cultural and creative knowledge base through a knowledge graph, specifically including: S1.1: Physical layer knowledge unit, which extracts the material reflectivity, color space coordinates (CIELAB values), 3D point cloud coordinates, and pattern Fourier transform features of handicrafts through a convolutional neural network (CNN), and stores them in the form of a triangular mesh model; S1.2: Skill layer knowledge unit, which analyzes the expert interview text and physical parameters collected by sensors through a long short-term memory network (LSTM) to generate structured data including step numbers, tool models, and process parameters (temperature, pressure, time); S1.3: Cultural layer knowledge unit, which analyzes the literature, folk records, and oral history texts related to handicrafts through a word vector model (Word2Vec) to construct an association network including cultural symbols, historical events, and geographical distributions; The activity design module designs experience activities based on the Kolb experiential learning model, specifically including: S2.1: Experience stage: Simulate the traditional handicraft production environment through a virtual reality scene, and guide users to complete tasks such as virtual material selection, tool operation, and finished product inspection; S2.2: Reflection stage: Ask users about their understanding of the operation steps through a voice interaction system, and record the user operation time, error times, and voice response text; S2.3: Abstraction stage: Combine the user behavior data with the content of the knowledge base to generate a text report including process principles and cultural backgrounds; S2.4: Inspection stage: Simulate the structural strength designed by the user through finite element analysis, and output the stress distribution map and feasibility score; The VR scenario design module uses ray tracing technology to generate a dynamic virtual scene, including: Scene generation unit: Construct scenes such as historical blocks, workshops, marketplaces, and sacrificial places based on 3D modeling data, and simulate the movement trajectories of raindrops, snowflakes, flames, and water flows through a particle system; Environment fusion unit: Real-time scan the real environment where the user is located through the SLAM algorithm, and superimpose virtual elements on the real space, with a superimposition accuracy ≤ 5 cm; The VR interaction design module specifically includes: Gesture recognition unit: Use the MobileNet-LSTM network to real-time recognize the user's gesture actions, with a recognition accuracy ≥ 95% and a delay ≤ 0.2 seconds; Haptic feedback unit: Simulate the touch of tool operation through an electromagnetic vibrator and a six-axis force sensor, with a vibration frequency range of 10 Hz - 1000 Hz and an amplitude range of 0.1 mm - 5 mm; Voice interaction unit: Parse the user's instructions based on the DeepSpeech model, support Chinese, English, and Japanese input, with a recognition accuracy ≥ 92%; The creative generation assistance module optimizes the user's design through a reinforcement learning algorithm: Input the sketch or 3D model designed by the user, and generate an improved solution through a generative adversarial network (GAN); Recommend design parameters according to the user's historical operation data, including color matching ratio and structural symmetry coefficient.

2. The cultural and creative experience system based on virtual reality technology according to claim 1, wherein The skill layer knowledge unit of the resource arrangement module adopts a dual-channel analysis method, specifically including: The first channel: Parse the expert interview text through the BERT model to extract the keywords of process parameters; Second channel: Collect torque data of tool operations through piezoelectric sensors and construct a process parameter database; Fuse the data of the two channels to generate a composite knowledge entry containing text descriptions, parameter ranges, and operation videos.

3. A cultural and creative experience system based on virtual reality technology according to claim 1, characterized in that, The scene generation unit of the VR scenario design module adopts a hierarchical rendering technique, specifically including: Base layer: Render static scenes including buildings, backgrounds, streets, and decorative elements; Dynamic layer: Overlay particle system to simulate raindrops (density 1000 / m 3 , speed 0.5 m / s), snowflakes (density 500 / m 3 , speed 0.3 m / s) and flame effects; Interaction layer: Real-time render virtual objects generated by user operations (such as virtual clay sculpting effects) with a resolution of 4K; The hierarchical rendering technique is implemented through GPU parallel computing, with a frame rate ≥ 60fps and a latency ≤ 20ms.

4. A cultural and creative experience system based on virtual reality technology according to claim 1, characterized in that, The gesture recognition unit adopts a multi-modal fusion algorithm, specifically including: Input data includes RGB images (resolution 1080p), depth images (resolution 720p), and inertial sensor data (accelerometer, gyroscope); Use an LSTM network to extract spatio-temporal features from sequence data; Enhance the recognition weight of key actions through an attention mechanism; Output action instructions, including "pick up tool" and "rotate object".

5. A cultural and creative experience system based on virtual reality technology according to claim 1, characterized in that, The tactile feedback unit includes an adaptive damping adjustment module, specifically including: Dynamically adjust the frequency and amplitude of the electromagnetic vibrator according to the physical parameters of the virtual operation (material Young's modulus, tool type); Real-time detect the user's operation force through a six-axis force sensor, and the feedback damping force is proportional to the stiffness of the virtual material.

6. The cultural and creative experience system based on virtual reality technology according to claim 1, wherein, The GAN network of the creative generation assistance module adopts a dual discriminator architecture, specifically including: First discriminator: Evaluate the aesthetic value of the design, including symmetry coefficient (0 - 1) and color coordination (color difference ΔE ≤ 3); Second discriminator: Evaluate the feasibility of the design, including structural strength (stress ≤ material yield strength) and material applicability (thermal expansion coefficient matching); The generator optimizes the output through adversarial training and finally outputs a design scheme that meets user preferences and is feasible.

7. A cultural and creative experience system based on virtual reality technology according to claim 1, characterized in that, The system also includes a cross-platform collaboration module, which supports: Multiple users access the same virtual scene through different terminals, including PCs, HMDS, and mobile devices; Real-time synchronize user operation data with a synchronization latency ≤ 50ms; Record the user's creative contributions through blockchain technology and generate a digital copyright certificate containing a timestamp and a hash value.

8. The cultural and creative experience system based on virtual reality technology according to claim 1, characterized in that The physical layer knowledge unit of the resource sorting module adopts a multi-view three-dimensional reconstruction technology, specifically including: Collect 360° images of handicrafts through an 8-camera array (resolution 12 million pixels); Use the Structure from Motion (SfM) algorithm to generate a high-precision three-dimensional model (point cloud density ≥ 1 million points / ㎡); Restore the material surface texture through a spectral reflection modeling algorithm (wavelength range 400 - 700nm).

Images