Non-abandoned artwork virtual display method and system based on VR technology

Abstract

The invention relates to the technical field of cultural heritage digitization, in particular to a VR technology-based non-abandoned artwork virtual display method and system, and the method comprises the following steps: collecting video data of various non-abandoned actions based on non-abandoned type information, recognizing the change mode of the character actions through analyzing the geometric structure and changes in the video sequence data, and displaying the change mode of the character actions. Recording image data of the clothes and the handicrafts, collecting audio data, and generating a basic data collection record; according to the method, the comprehensiveness and the accuracy of non-missing display data are ensured through video, image and audio acquisition, and the fidelity of a visual effect is improved and the accessibility and the ornamental value of display are improved by combining the construction of a three-dimensional model and the mapping of performance actions; through combination of simulation of a propagation path of sound in a scene and setting of object interaction feedback parameters, authenticity of non-disciplinary display is improved, user experience is enriched, and powerful support is provided for cultural education and inheritance through combination of planning of a cultural theme guide path.

Description

Technical Field

[0001] The present invention relates to the field of cultural heritage digitization technology, and in particular to a method and system for virtual display of intangible cultural heritage artworks based on VR technology. Background Art

[0002] The field of cultural heritage digitization technology focuses on using a variety of digital technologies to record, preserve, restore and share the world's cultural heritage, covering applications from 3D scanning and modeling, digital image acquisition, to virtual reality and augmented reality. It aims to create digital copies of cultural assets that can be widely accessed, protect valuable cultural assets that are easily damaged or lost due to natural disasters or human factors, help cultural heritage protection experts study and restore historical sites, and enable the public to learn and appreciate a variety of cultural treasures without geographical restrictions, for use in education and tourism industries.

[0003] Among them, the virtual display method of intangible cultural heritage artworks aims to use a variety of digital technologies to display and disseminate intangible cultural heritage. Through virtual exhibitions, including traditional dances, music performances, and handicraft skills, and by increasing interactive and educational content, the public's interest in and awareness of protection of intangible cultural heritage can be enhanced, so that the inheritance and education of intangible cultural heritage can transcend geographical and temporal limitations, allowing global audiences to easily access and understand intangible cultural heritage around the world. It aims to provide new ways for the continued dissemination and protection of intangible cultural heritage through modern technological means, help enhance the public's understanding and respect for the value of a variety of precious cultural heritages, and provide a dynamic platform for cultural inheritance.

[0004] Traditional virtual display methods of intangible cultural heritage artworks lack support for capturing dynamic performing arts and interactivity. They are unable to capture complex character movements and rapidly changing performance details, resulting in a lack of vividness and interactivity in virtual displays. It is difficult to reproduce the artistic charm and cultural depth of the performance. The processing of sound effects relies on pre-recorded audio, which limits the natural expression of sound in different virtual environments and reduces the authenticity of the virtual experience. The effects on education and cultural inheritance are subject to certain constraints, and the potential of digital technology in cultural protection and dissemination cannot be fully tapped. Summary of the invention

[0005] In order to solve the technical problems of the lack of liveliness and interactivity in the prior art, the embodiment of the present invention provides a method and system for virtual display of intangible cultural heritage artworks based on VR technology. The technical solution is as follows:

[0006] On the one hand, a virtual display method of intangible cultural heritage artworks based on VR technology is provided, and the method includes:

[0007] S1: Based on the information of intangible cultural heritage types, video data of various intangible cultural heritage actions are collected. By analyzing the geometric structure and changes in the video sequence data, the change pattern of the character's actions is identified, the image data of costumes and handicrafts is recorded, and audio data is collected to generate basic data collection records;

[0008] S2: Based on the basic data collection records, analyze and configure various intangible cultural heritage performance scenes, construct three-dimensional digital models of various characters, costumes, and crafts, combine character movements, map them to performance scenes, and generate performance scene configuration records;

[0009] S3: Based on the performance scene configuration record, by simulating the propagation path of music sound waves in various environments and combining the attenuation characteristics of sound waves in various media, the audio data is mapped to the scene to generate spatial sound effect configuration information;

[0010] S4: configuring interactive effects for various scene objects according to the spatial sound effect configuration information and physical characteristics, simulating physical reactions of audiences when interacting with various scene elements, and generating scene interaction parameters;

[0011] S5: Analyze the cultural types and content characteristics of various intangible cultural heritage performances according to the scene interaction parameters, classify the intangible cultural heritage performances according to the cultural background, and plan the user's tour path according to the performance type to generate a cultural theme guided path.

[0012] As a further solution of the present invention, the basic data collection record includes video and image data, character motion analysis results, and audio data collection results; the performance scene configuration record includes scene light data, background object data, and a three-dimensional digital model; the spatial sound effect configuration information includes music sound wave propagation path, sound wave interaction parameters, and audio data space mapping results; the scene interaction parameters specifically refer to physical feedback information of the interaction position, sound wave change simulation data, and object physical property configuration parameters; the cultural theme guided tour path includes cultural type classification results, performance type classification results, and intangible cultural heritage visiting path.

[0013] As a further solution of the present invention, based on the intangible cultural heritage type information, video data of various intangible cultural heritage actions are collected, the geometric structure and changes in the video sequence data are analyzed, the change pattern of the character's action is identified, the image data of clothing and handicrafts is recorded, and audio data is collected. The steps of generating basic data collection records are specifically as follows:

[0014] S101: Based on the intangible cultural heritage type information, using an image acquisition device, collecting video data of intangible cultural heritage performance actions, recording image data of costumes and handicrafts, and generating visual data records;

[0015] S102: Analyze the video data based on the visual data record, detect the outline of the person in the multiple frames, identify the person's actions and record the change pattern of the action, and generate the action pattern analysis result;

[0016] S103: Based on the motion pattern analysis result, use an audio acquisition device to record the audio data of multiple musical instruments and performers in the intangible cultural heritage performance to generate a basic data acquisition record.

[0017] As a further solution of the present invention, the specific formula for detecting the contour of a person in multiple frames of images is:

[0018]

[0019] in, is the partial differential operator, I is the image brightness function, Indicates a slight change in the horizontal direction of the image. Indicates a slight change in the vertical direction of the image. Indicates the rate of change of image brightness in the horizontal direction. It represents the rate of change of image brightness in the vertical direction, and G represents the edge strength of the image.

[0020] As a further solution of the present invention, based on the basic data collection records, a variety of intangible cultural heritage performance scenes are analyzed and configured, a variety of three-dimensional digital models of characters, costumes, and handicrafts are constructed, and combined with character actions, they are mapped to the performance scenes. The steps of generating performance scene configuration records are specifically as follows:

[0021] S201: Using the basic data collection records, configuring the performance scene according to the type of intangible cultural heritage, including setting scene lighting and background objects, and generating basic scene setting information;

[0022] S202: Based on the basic scene setting information, by analyzing the images of people, clothing and crafts, extracting color, geometric shape, surface texture information, and creating a three-dimensional digital model, generating a three-dimensional model configuration record;

[0023] S203: According to the three-dimensional model configuration record and in combination with the character action information, the models of the characters, clothing and crafts are mapped to a variety of performance scenes to generate a performance scene configuration record.

[0024] As a further solution of the present invention, based on the performance scene configuration record, by simulating the propagation path of music sound waves in various environments, combined with the attenuation characteristics of sound waves in various media, the audio data is mapped to the scene, and the steps of generating spatial sound effect configuration information are specifically as follows:

[0025] S301: Based on the performance scene configuration record, by extracting physical structure data of multiple scenes, simulating the propagation paths of sound waves in multiple performance scenes, and generating sound wave propagation paths;

[0026] S302: Based on the sound wave propagation path, according to the reflectivity and absorptivity of the sound wave on the surfaces of various materials, the intensity values ​​of the sound wave at multiple positions are calculated to generate scene sound effect interaction information;

[0027] S303: Utilizing the scene sound effect interaction information, the audio data is mapped to multiple musical instruments and performer models in the scene to generate spatial sound effect configuration information.

[0028] As a further solution of the present invention, the specific formula for calculating the intensity values ​​of the sound wave at multiple positions is:

[0029] I=I0-20 log 10 (d) -αd;

[0030] Where I is the sound pressure level at the target point, I0 represents the initial sound pressure level of the sound source, log 10 (d) is the logarithmic function of the sound wave propagation distance, d is the distance from the sound source to the receiving point, and α is the absorption coefficient of the material.

[0031] As a further solution of the present invention, according to the spatial sound effect configuration information, according to the physical characteristics, the interactive effects are configured for various scene objects to simulate the physical reaction of the audience when interacting with various scene elements, and the steps of generating scene interaction parameters are specifically as follows:

[0032] S401: configuring physical property parameters for multiple materials in the scene, including cloth, wood, and metal, according to the spatial sound effect configuration information, to generate material physical property information;

[0033] S402: Based on the material physical property information, analyzing and identifying the interactive feedback effects required by various items, including the wave reaction of fabrics and the sound changes of musical instruments, and generating interactive feedback characteristic configurations;

[0034] S403: According to the interactive feedback characteristic configuration, feedback parameters are configured for a variety of objects, including the displacement caused by touch and the amplitude of sound waves, to generate scene interaction parameters.

[0035] As a further solution of the present invention, according to the scene interaction parameters, the cultural types and content characteristics of various intangible cultural heritage performances are analyzed, the intangible cultural heritage performances are classified according to the cultural background, and the user's tour path is planned according to the performance type. The steps of generating a cultural theme guided tour path are specifically as follows:

[0036] S501: Based on the scene interaction parameters, classify the types of intangible cultural heritage according to the cultural background and generate a cultural classification index;

[0037] S502: Based on the cultural classification index, according to the art forms of the performances, including singing and dancing, drama, and handicraft skills, the types of various intangible cultural heritage performances are identified, and a performance type analysis record is generated;

[0038] S503: According to the subdivided records of the performance types, a tour route is planned for the intangible cultural heritage display according to the types of culture and performance, and a cultural theme guided tour route is generated.

[0039] On the other hand, a virtual display system for intangible cultural heritage artworks based on VR technology is provided, and the system is applied to a virtual display method for intangible cultural heritage artworks based on VR technology, and the system comprises:

[0040] The performance data recording module collects action videos of intangible cultural heritage performances based on the intangible cultural heritage type information, records the image information of costumes and handicrafts, identifies the changing patterns of character actions through frame analysis of the collected videos, and records audio data to generate basic data collection records;

[0041] The intangible cultural heritage scene modeling module configures performance scenes for various intangible cultural heritages based on the basic data collection records, creates three-dimensional digital models of characters, costumes and crafts and maps them to the scenes in combination with action information, and generates performance scene configuration records;

[0042] The spatial sound effect simulation module simulates the propagation paths of music sound waves in multiple scenes based on the performance scene configuration records, calculates the reflection and absorption of sound waves and various objects in the scenes, maps the audio data to the simulated scenes, and generates spatial sound effect configuration information;

[0043] The interactive feedback design module sets the interactive feedback parameters of various scene objects according to the spatial sound effect configuration information, including touch response, physical collision effect and sound feedback, and generates scene interactive parameters;

[0044] The display tour path module analyzes the cultural types and performance types of various intangible cultural heritage performances based on the scene interaction parameters, plans the tour path, and generates a cultural theme guided tour path.

[0045] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0046] Through the collection of video, images and audio, the comprehensiveness and accuracy of the intangible cultural heritage display data are ensured. Combined with the construction of three-dimensional models and the mapping of performance movements, the realism of the visual effects is improved, and the accessibility and viewing value of the display are improved. Combined with the simulation of the sound propagation path in the scene and the setting of object interaction feedback parameters, the authenticity of the intangible cultural heritage display is improved, the user experience is enriched, and combined with the planning of cultural theme guided routes, it provides strong support for cultural education and inheritance. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 It is a schematic diagram of the workflow of the present invention;

[0049] Figure 2 This is a detailed flow chart of S1 of the present invention;

[0050] Figure 3 This is a detailed flow chart of S2 of the present invention;

[0051] Figure 4 This is a detailed flow chart of S3 of the present invention;

[0052] Figure 5 This is a detailed flow chart of S4 of the present invention;

[0053] Figure 6 This is a detailed flow chart of S5 of the present invention;

[0054] Figure 7 It is a system flow chart of the present invention. DETAILED DESCRIPTION

[0055] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0056] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0057] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0058] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0059] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0060] The embodiment of the present invention provides a virtual display method of intangible cultural heritage artworks based on VR technology, such as Figure 1 The flowchart of the method for virtual display of intangible cultural heritage artworks based on VR technology is shown, and the processing flow of the method may include the following steps:

[0061] S1: Based on the information of intangible cultural heritage types, video data of various intangible cultural heritage actions are collected. By analyzing the geometric structure and changes in the video sequence data, the change pattern of the character's actions is identified, the image data of costumes and handicrafts is recorded, and audio data is collected to generate basic data collection records;

[0062] S2: Based on the basic data collection records, analyze and configure various intangible cultural heritage performance scenes, build three-dimensional digital models of various characters, costumes, and crafts, combine character movements, map them to the performance scenes, and generate performance scene configuration records;

[0063] S3: Based on the performance scene configuration record, by simulating the propagation path of music sound waves in various environments and combining the attenuation characteristics of sound waves in various media, the audio data is mapped to the scene to generate spatial sound configuration information;

[0064] S4: Based on the spatial sound configuration information and physical characteristics, configure interactive effects for various scene objects, simulate the physical reactions of the audience when interacting with various scene elements, and generate scene interaction parameters;

[0065] S5: Analyze the cultural types and content characteristics of various intangible cultural heritage performances based on scene interaction parameters, classify intangible cultural heritage performances according to cultural background, and plan user tour paths based on performance types to generate cultural theme guided paths.

[0066] Basic data collection records include video and image data, character motion analysis results, and audio data collection results. Performance scene configuration records include scene light data, background object data, and three-dimensional digital models. Spatial sound effect configuration information includes music sound wave propagation path, sound wave interaction parameters, and audio data space mapping results. Scene interaction parameters specifically refer to physical feedback information of the interaction position, sound wave change simulation data, and object physical property configuration parameters. The cultural theme guided tour path includes cultural type classification results, performance type classification results, and intangible cultural heritage visiting path.

[0067] See also Figure 2 Based on the information of intangible cultural heritage types, video data of various intangible cultural heritage actions are collected. By analyzing the geometric structure and changes in the video sequence data, the change pattern of the character's actions is identified, the image data of clothing and handicrafts is recorded, and audio data is collected. The specific steps for generating basic data collection records are as follows:

[0068] S101: Based on the intangible cultural heritage type information, using an image acquisition device, collecting video data of intangible cultural heritage performance actions, recording image data of costumes and handicrafts, and generating visual data records;

[0069] In sub-step S101, cameras are deployed to capture dynamic video data during the performance. The camera settings will include ISO100-3200 and shutter speed 1 / 50 second to 1 / 2000 second to ensure excellent image quality. Automatic exposure lock and autofocus are combined to ensure image consistency. The performers' costumes and details of the crafts used are recorded to ensure that the resulting images can reflect subtle texture and color information. The collected videos are stabilized and optimized, including using SmoothTech technology to reduce jitter and using ClearView algorithm for image quality optimization. For each performance scene, exposure and focus parameters are automatically adjusted according to lighting changes to ensure consistency of image quality. All collected video and image data are stored in high-speed memory cards and synchronously backed up to cloud servers. Combined with AES-256 encryption to ensure data security, a complete visual data record is formed.

[0070] S102: Analyze the video data based on the visual data record, detect the outline of the person in the multiple frames, identify the person's actions and record the change pattern of the action, and generate the action pattern analysis result;

[0071] The specific formula for detecting the contour of a person in multiple frames of images is:

[0072]

[0073] in, is the partial differential operator, I is the image brightness function, Indicates a slight change in the horizontal direction of the image. Indicates a slight change in the vertical direction of the image. Indicates the rate of change of image brightness in the horizontal direction. It represents the rate of change of image brightness in the vertical direction, and G represents the edge strength of the image.

[0074] formula:

[0075]

[0076] Detailed explanation of the formula and the process of formula calculation and derivation:

[0077] The formula is used to calculate the edge strength of each pixel in the image and to identify and analyze the character movements and clothing boundaries in the intangible cultural heritage videos.

[0078] Parameter meaning and setting value:

[0079] Represents the gradient of image brightness in the x direction. Assuming that the brightness values ​​of adjacent pixels are I1=120 and I2=130,

[0080] Represents the gradient of image brightness in the y direction. Assuming that the brightness values ​​of adjacent pixels are I3=100 and I4=110, then

[0081] Substitute the parameters into the formula for calculation:

[0082]

[0083] Result 14.14 shows that the intensity of the edge of this pixel is more significant, indicating that this may be the edge of the character's outline. The result helps locate and track the movements and costumes of intangible cultural heritage performers, and reconstruct intangible cultural heritage performance scenes in a virtual reality environment.

[0084] S103: Based on the results of the motion pattern analysis, use an audio acquisition device to record audio data of multiple musical instruments and performers in the intangible cultural heritage performance to generate a basic data acquisition record;

[0085] In sub-step S103, a microphone array is deployed to capture audio data of various musical instruments and vocals involved in the performance. The audio equipment settings include audio receiving modes optimized for instruments of different ranges. The system uses real-time audio processing technologies implemented by Ableton Live software, such as dynamic range compression and echo cancellation, to ensure that the captured audio is clear and has high fidelity. The recorded audio data is classified and processed by musical instruments and performers. Each audio clip is subjected to sound wave morphology analysis to determine the audio quality and is labeled according to pitch and timbre. All audio data is stored and backed up in a lossless format to form a comprehensive basic data collection record, which provides raw data for subsequent sound effect processing and musicological analysis.

[0086] See also Figure 3 ,Based on the basic data collection records, analyze and configure a variety of intangible cultural heritage performance scenes, build a variety of three-dimensional digital models of characters, costumes, and crafts, combine character movements, and map them to the performance scenes. The specific steps for generating performance scene configuration records are as follows:

[0087] S201: using the basic data collection records, configuring the performance scene according to the type of intangible cultural heritage, including setting the scene light and background objects, and generating basic scene setting information;

[0088] In sub-step S201, Autodesk Maya software is used to set the scene. According to the type of each intangible cultural heritage, the scene lighting and background layout are designed. By analyzing the ambient light of the existing data, the dynamic lighting simulation function of Autodesk Maya is used to adjust the position and intensity of the light source in the virtual environment to match the lighting conditions during the actual performance. For example, if the data shows that soft diffuse light is often used in the performance venue, a soft light source is set and its angle and intensity are adjusted to ensure that the light projection is natural and uniform. Background objects are selected according to the characteristics of intangible cultural heritage, such as traditional architectural elements, natural landscape models, and cultural symbols, to ensure that the visual effect of each scene can reflect the environmental atmosphere of the intangible cultural heritage. Maya's advanced material and texture functions are used to enhance the realism of the scene, such as using high dynamic range imaging (HDRI) backgrounds to provide complex lighting effects and depth. The results are systematically recorded as basic scene setting information, which includes specific setting parameters for each element in the scene, including the color temperature, light intensity, source position of the light, layout coordinates of background objects, and detailed information on materials and textures used, providing a detailed foundation for subsequent scene rendering and refinement.

[0089] S202: Based on the basic scene setting information, by analyzing the images of people, clothing and crafts, extracting color, geometric shape, surface texture information, and creating a three-dimensional digital model, generating a three-dimensional model configuration record;

[0090] In sub-step S202, Pixologic ZBrush software is used to perform 3D modeling of characters, costumes, and crafts in intangible cultural heritage. The software performs color and geometric morphology analysis on the imported high-resolution image data, automatically extracts key surface texture features, and manually refines the details of the model through modeling tools to ensure that the 3D model can truly reproduce the visual characteristics of the physical object. Each model will set the corresponding texture map and material reflection characteristics according to the material properties. The generated 3D model configuration record lists the texture coordinates, color information, and material parameters of all models, which lays a precise foundation for subsequent dynamic simulation and interactive design.

[0091] S203: mapping the models of characters, clothing and handicrafts to various performance scenes according to the three-dimensional model configuration record and in combination with the character action information, and generating a performance scene configuration record;

[0092] In sub-step S203, the generated three-dimensional model and action information are integrated into the designed virtual scene using the Unity 3D engine. Through script automation processing, the engine reads the three-dimensional model configuration record, accurately positions each model according to the predetermined scene layout coordinates, and the motion capture data is converted into animation clips and bound to the character model to achieve synchronous display of each action and model. The interactive logic in the scene is programmed and implemented. When the audience enters the scene through the virtual reality device, they can interact with the elements in the scene in real time, such as virtually touching and operating handicrafts. The interactive responses and physical properties of all scene elements are recorded in the performance scene configuration record, providing rich interactive experience and educational significance for the intangible cultural heritage performances in virtual reality.

[0093] See also Figure 4 Based on the performance scene configuration record, by simulating the propagation path of music sound waves in various environments and combining the attenuation characteristics of sound waves in various media, the audio data is mapped to the scene, and the steps to generate spatial sound configuration information are as follows:

[0094] S301: Based on the performance scene configuration record, by extracting the physical structure data of multiple scenes, simulating the propagation paths of sound waves in multiple performance scenes, and generating the sound wave propagation paths;

[0095] In sub-step S301, COMSOL Multiphysics software is used to process the performance scene configuration record, including the scene geometry and physical structure data, to perform sound wave propagation simulation, and simulate the sound wave behavior in each scene. The software calculates the sound wave propagation path and scattering process based on the preset scene physical parameters, such as the density of the wall and the characteristics of the sound-absorbing material. During the process, the sound wave propagation path is accurately simulated to ensure the authenticity and accuracy of the acoustic simulation. The acoustic characteristics of each scene, such as echo and reverberation time, are also calculated to generate a sound wave propagation path record, which provides a scientific basis for the subsequent sound effect design.

[0096] S302: Based on the sound wave propagation path, according to the reflectivity and absorptivity of the sound wave on the surfaces of various materials, the intensity values ​​of the sound wave at multiple positions are calculated to generate scene sound effect interaction information;

[0097] The specific formula for calculating the intensity values ​​of sound waves at multiple locations is:

[0098] I=I0-20 log 10 (d) -αd;

[0099] Where I is the sound pressure level at the target point, I0 represents the initial sound pressure level of the sound source, log 10 (d) is the logarithmic function of the sound wave propagation distance, d is the distance from the sound source to the receiving point, and α is the absorption coefficient of the material.

[0100] formula:

[0101] I=I0-20log 10 (d)-αd

[0102] Detailed explanation of the formula and the process of formula calculation and derivation:

[0103] The formula is used to simulate and calculate the sound intensity values ​​at the audience's visitor location, accurately mapping the audio data for the scene.

[0104] Parameter meaning and setting value:

[0105] I0 is the initial intensity of the sound source, which is set to 94dB.

[0106] d is the distance from the sound source to the receiving point, which is set to 10 meters.

[0107] α is the absorption coefficient of the additional material, assumed to be 0.01dB / m.

[0108] Substitute the parameters into the formula for calculation:

[0109] I=94-20log 10 (10)-0.01×10;

[0110] I = 94-20 × 1-0.1;

[0111] I = 73.9dB;

[0112] The result 73.9dB indicates that the sound intensity at the target location is 73.9dB. The result is used to map the audio data to bring a more realistic audio-visual experience to the audience.

[0113] S303: using the scene sound effect interaction information, mapping the audio data to multiple musical instruments and performer models in the scene to generate spatial sound effect configuration information;

[0114] In sub-step S303, the recorded audio data is accurately mapped to the virtual environment through Steinberg Nuendo audio processing software. The software supports complex sound effect design, including volume, echo and directionality adjustment, ensuring that each audio output is perfectly synchronized with the position and movement of the virtual model. For models of different musical instruments and performers, the audio properties such as stereo distribution and dynamic range are adjusted according to the sound effect interaction information to adapt to their specific positions in the virtual scene. Through the audio configuration process, spatial sound effect configuration information is generated, which provides personalized sound effect settings for each musical instrument and performer model, enhancing the immersion and sense of presence of the virtual performance.

[0115] See also Figure 5 , according to the spatial sound configuration information, according to the physical characteristics, configure interactive effects for various scene objects, simulate the physical reaction of the audience when interacting with various scene elements, and generate scene interaction parameters in the following steps:

[0116] S401: configuring physical property parameters for multiple materials in the scene, including cloth, wood, and metal, according to the spatial sound effect configuration information, and generating material physical property information;

[0117] In sub-step S401, Ansys Workbench software is used to process the spatial sound configuration information and adjust the physical properties of various materials in the scene. By inputting the density, elastic modulus and sound absorption coefficient of materials such as cloth, wood and metal, the software simulates the physical response of each material under the action of sound waves. The process ensures the authenticity of the material response. The physical property parameters of each material are recorded, such as the fluttering characteristics of cloth and the resonant frequency of wood. The information is integrated to form a material physical property information database, which provides an accurate basis for the correct presentation of materials in the scene and interactive feedback.

[0118] S402: Based on the physical property information of the material, analyzing and identifying the interactive feedback effects required by various items, including the wave reaction of the cloth and the sound change of the musical instrument, and generating the interactive feedback characteristic configuration;

[0119] In sub-step S402, the interactive feedback characteristics are analyzed using MATLAB and Simulink environments. For different objects, such as clothing and musical instruments, the wave patterns of the fabric under the action of wind and the sonic response of the musical instrument when it is played are calculated, and the reactions of the objects in virtual interaction are simulated. The fabric model takes into account the effects of wind speed and fabric density on the amplitude of the wave, and the musical instrument model simulates the sonic characteristics generated by string vibration. The generated interactive feedback characteristic configuration records the parameter settings of each interactive effect, such as amplitude, frequency, and attenuation coefficient, providing quantifiable data support for dynamic interaction in virtual scenes.

[0120] S403: According to the interactive feedback characteristic configuration, feedback parameters are configured for the various items, including the displacement caused by touch and the amplitude of the sound wave, to generate scene interaction parameters;

[0121] In sub-step S403, the Unity 3D engine is used in conjunction with the PhysX physics engine to configure real-time interaction parameters. According to the interactive feedback characteristics, physical reaction parameters are set for various objects in the scene. Touch interaction takes into account touch strength and contact area, and calculates the displacement caused by touch and the corresponding physical deformation. Sound wave interaction processing includes the setting of sound wave amplitude and propagation speed to ensure the physical accuracy of sound waves interacting with objects. The configuration is automatically applied to the scene model through scripts. Each interaction produces the expected physical effect according to the set parameters, such as the fluttering of cloth and the vibration response of musical instruments. The feedback parameter setting makes the interaction of virtual scenes more vivid and realistic, enhancing the user experience.

[0122] See also Figure 6 ,According to the scene interaction parameters, the cultural types and content characteristics of various intangible cultural heritage performances are analyzed, the intangible cultural heritage performances are classified according to the cultural background, and the user's visiting path is planned according to the performance type. The specific steps for generating a cultural theme guided path are as follows:

[0123] S501: Based on the scene interaction parameters, classify the types of intangible cultural heritage according to the cultural background and generate a cultural classification index;

[0124] In sub-step S501, Python programming language and Pandas library are used for data processing and classification. The collected scene interaction parameters are processed and analyzed, and different intangible cultural heritage assets are automatically classified according to their cultural background and characteristics. In the process, the intangible cultural heritage assets are marked according to cultural regions, historical periods and art forms, and a cultural classification index is generated. The index includes specific classification labels for each intangible cultural heritage, such as "Asia-China-Drama" or "Europe-Italy-Sculpture", to ensure that all data can be accurately classified and used for further analysis and display preparation.

[0125] S502: Based on the cultural classification index, identify the types of various intangible cultural heritage performances according to the art forms of the performances, including singing and dancing, drama, and handicraft skills, and generate performance type analysis records;

[0126] In sub-step S502, machine learning models, such as the K-means clustering algorithm, are used to further analyze the subdivision types of various intangible cultural heritage performances based on the cultural classification index. Through the feature set of artistic performance elements provided by the education and training departments, such as tone, rhythm, and tools or materials used, the model can identify and classify various performance forms. The generated performance type analysis records show the categories of various intangible cultural heritage performances, such as traditional dance, classical music concerts, handicraft production, etc., providing accurate reference data for cultural communication and education.

[0127] S503: Segment the records according to the performance type, plan a tour route for the intangible cultural heritage display according to the type of culture and performance, and generate a cultural theme guided tour route;

[0128] In sub-step S503, a graph theory algorithm is used to design a tour route based on the performance type segmentation records. During the route planning process, the audience's interests and cultural background are taken into consideration to generate theme guided routes for various types of intangible cultural heritage performances, such as "A Journey into Traditional Chinese Drama" or "A Journey into European Classical Music". A series of exhibition routes are planned from one performance scene to another to ensure that visitors can navigate according to their cultural interests and artistic preferences. The generated cultural theme guided routes provide convenient navigation information and enhance the quality of education and experience for visitors.

[0129] See also Figure 7 , a virtual display system for intangible cultural heritage artworks based on VR technology, the virtual display system for intangible cultural heritage artworks based on VR technology is used to execute the above-mentioned virtual display method for intangible cultural heritage artworks based on VR technology, and the system includes:

[0130] The performance data recording module collects action videos of intangible cultural heritage performances based on the intangible cultural heritage type information, records the image information of costumes and handicrafts, identifies the changing patterns of character actions through frame analysis of the collected videos, and records audio data to generate basic data collection records;

[0131] The intangible cultural heritage scene modeling module configures performance scenes for various intangible cultural heritage based on basic data collection records, combines action information, creates three-dimensional digital models of characters, costumes and crafts, maps them to the scenes, and generates performance scene configuration records;

[0132] The spatial sound simulation module simulates the propagation path of music sound waves in multiple scenes based on the performance scene configuration record, calculates the reflection and absorption of sound waves and various objects in the scene, maps the audio data to the simulated scene, and generates spatial sound configuration information;

[0133] The interactive feedback design module sets the interactive feedback parameters of various scene objects according to the spatial sound effect configuration information, including touch response, physical collision effect and sound feedback, and generates scene interaction parameters;

[0134] The display tour path module analyzes the cultural types and performance types of various intangible cultural heritage performances based on scene interaction parameters, plans tour paths, and generates cultural theme guided paths.

[0135] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0136] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0137] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0138] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0139] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0140] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0141] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0142] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0144] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0145] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A virtual display method for intangible cultural heritage artworks based on VR technology, characterized in that: The method comprises: Based on the information of intangible cultural heritage types, video data of various intangible cultural heritage actions are collected. By analyzing the geometric structure and changes in the video sequence data, the change pattern of the character's actions is identified, the image data of clothing and handicrafts is recorded, and audio data is collected to generate basic data collection records; Based on the basic data collection records, analyze and configure various intangible cultural heritage performance scenes, build three-dimensional digital models of various characters, costumes, and crafts, combine character movements, map them to performance scenes, and generate performance scene configuration records; Based on the performance scene configuration record, by simulating the propagation path of music sound waves in various environments and combining the attenuation characteristics of sound waves in various media, the audio data is mapped to the scene to generate spatial sound effect configuration information; According to the spatial sound effect configuration information, according to the physical characteristics, interactive effects are configured for various scene objects, the physical reactions of the audience when interacting with various scene elements are simulated, and scene interaction parameters are generated; According to the scene interaction parameters, the cultural types and content characteristics of various intangible cultural heritage performances are analyzed, the intangible cultural heritage performances are classified according to the cultural background, and the user's visiting path is planned according to the performance type to generate a cultural theme guided path.

2. The method for virtual display of intangible cultural heritage artworks based on VR technology according to claim 1, characterized in that: The basic data collection records include video and image data, character motion analysis results, and audio data collection results. The performance scene configuration records include scene light data, background object data, and three-dimensional digital models. The spatial sound effect configuration information includes music sound wave propagation path, sound wave interaction parameters, and audio data space mapping results. The scene interaction parameters specifically refer to physical feedback information of the interaction position, sound wave change simulation data, and object physical property configuration parameters. The cultural theme guided tour path includes cultural type classification results, performance type classification results, and intangible cultural heritage visiting path.

3. The method for virtual display of intangible cultural heritage artworks based on VR technology according to claim 1, characterized in that: Based on the intangible cultural heritage type information, video data of various intangible cultural heritage actions are collected. By analyzing the geometric structure and changes in the video sequence data, the change pattern of the character's actions is identified, the image data of clothing and handicrafts is recorded, and audio data is collected. The specific steps for generating basic data collection records are as follows: Based on the information of intangible cultural heritage types, use image acquisition equipment to collect video data of intangible cultural heritage performance actions, record image data of costumes and handicrafts, and generate visual data records; Based on the visual data record, the video data is analyzed to detect the outline of the person in the multiple frames of images, the person's actions are identified and the change pattern of the action is recorded to generate an action pattern analysis result; According to the motion pattern analysis results, an audio acquisition device is used to record the audio data of multiple musical instruments and performers in the intangible cultural heritage performance to generate a basic data acquisition record.

4. The method for virtual display of intangible cultural heritage artworks based on VR technology according to claim 3 is characterized in that: The specific formula for detecting the contour of a person in multiple frames of images is: in, is the partial differential operator, I is the image brightness function, Indicates a slight change in the horizontal direction of the image. Indicates a slight change in the vertical direction of the image. Indicates the rate of change of image brightness in the horizontal direction. It represents the rate of change of image brightness in the vertical direction, and G represents the edge strength of the image.

5. The method for virtual display of intangible cultural heritage artworks based on VR technology according to claim 1, characterized in that: Based on the basic data collection records, a variety of intangible cultural heritage performance scenes are analyzed and configured, and a variety of three-dimensional digital models of characters, costumes, and crafts are constructed. Combined with character movements, they are mapped to the performance scenes. The specific steps of generating performance scene configuration records are as follows: Using the basic data collection records, according to the type of intangible cultural heritage, configure the performance scene, including setting scene lighting and background objects, and generate basic scene setting information; Based on the basic scene setting information, by analyzing the images of people, clothing and crafts, extracting color, geometric shape, surface texture information, and creating a three-dimensional digital model, generating a three-dimensional model configuration record; According to the three-dimensional model configuration record and in combination with character action information, the models of characters, clothing and crafts are mapped to a variety of performance scenes to generate a performance scene configuration record.

6. The method for virtual display of intangible cultural heritage artworks based on VR technology according to claim 1, characterized in that: Based on the performance scene configuration record, by simulating the propagation path of music sound waves in various environments, combined with the attenuation characteristics of sound waves in various media, the audio data is mapped to the scene, and the steps of generating spatial sound effect configuration information are specifically as follows: Based on the performance scene configuration record, by extracting the physical structure data of multiple scenes, simulating the propagation paths of sound waves in multiple performance scenes, and generating the sound wave propagation paths; Based on the sound wave propagation path, according to the reflectivity and absorptivity of the sound wave on the surfaces of various materials, the intensity values ​​of the sound wave at multiple positions are calculated to generate scene sound effect interaction information; The scene sound effect interaction information is used to map the audio data to multiple musical instruments and performer models in the scene to generate spatial sound effect configuration information.

7. The method for virtual display of intangible cultural heritage artworks based on VR technology according to claim 6 is characterized in that: The specific formula for calculating the intensity values ​​of the sound wave at multiple positions is: I=I0-20 log 10 (d)-αd; Where I is the sound pressure level at the target point, I0 represents the initial sound pressure level of the sound source, log 10 (d) is the logarithmic function of the sound wave propagation distance, d is the distance from the sound source to the receiving point, and α is the absorption coefficient of the material.

8. The method for virtual display of intangible cultural heritage artworks based on VR technology according to claim 1, characterized in that: According to the spatial sound effect configuration information, according to the physical characteristics, the interactive effects are configured for various scene objects, and the physical reactions of the audience when interacting with various scene elements are simulated. The steps of generating scene interaction parameters are specifically as follows: According to the spatial sound effect configuration information, physical property parameters are configured for a variety of materials in the scene, including cloth, wood, and metal, to generate material physical property information; Based on the material physical property information, analyze and identify the interactive feedback effects required by various items, including the wave reaction of fabrics and the sound changes of musical instruments, and generate interactive feedback characteristic configurations; According to the interactive feedback characteristic configuration, feedback parameters are configured for a variety of objects, including the displacement caused by touch and the amplitude generated by sound waves, to generate scene interaction parameters.

9. The method for virtual display of intangible cultural heritage artworks based on VR technology according to claim 1, characterized in that: According to the scene interaction parameters, the cultural types and content characteristics of various intangible cultural heritage performances are analyzed, the intangible cultural heritage performances are classified according to the cultural background, and the user's tour path is planned according to the performance type. The specific steps of generating a cultural theme guided tour path are as follows: Based on the scene interaction parameters, the types of intangible cultural heritage are classified according to the cultural background to generate a cultural classification index; Based on the cultural classification index, the types of various intangible cultural heritage performances are identified according to the art forms of the performances, including singing and dancing, drama, and handicraft skills, and performance type analysis records are generated; According to the records of the performance types, a tour route is planned for the intangible cultural heritage display according to the type of culture and performance, and a cultural theme guided tour route is generated.

10. The virtual display system of intangible cultural heritage artworks based on VR technology is characterized by: According to any one of claims 1 to 9, the method for virtual display of intangible cultural heritage artworks based on VR technology comprises: The performance data recording module collects action videos of intangible cultural heritage performances based on the intangible cultural heritage type information, records the image information of costumes and handicrafts, identifies the changing patterns of character actions through frame analysis of the collected videos, and records audio data to generate basic data collection records; The intangible cultural heritage scene modeling module configures performance scenes for various intangible cultural heritages based on the basic data collection records, creates three-dimensional digital models of characters, costumes and handicrafts and maps them to the scenes in combination with action information, and generates performance scene configuration records; The spatial sound effect simulation module simulates the propagation paths of music sound waves in multiple scenes based on the performance scene configuration records, calculates the reflection and absorption of sound waves and various objects in the scenes, maps the audio data to the simulated scenes, and generates spatial sound effect configuration information; The interactive feedback design module sets the interactive feedback parameters of various scene objects according to the spatial sound effect configuration information, including touch response, physical collision effect and sound feedback, and generates scene interactive parameters; The display tour path module analyzes the cultural types and performance types of various intangible cultural heritage performances based on the scene interaction parameters, plans the tour path, and generates a cultural theme guided tour path.

Images