Virtual reality-based intangible cultural heritage immersive experience method, device, equipment and medium

Abstract

The application relates to a virtual reality-based intangible cultural heritage immersive experience method, device, equipment and medium. The method comprises the following steps: acquiring multidimensional data of a target intangible cultural heritage scene; performing grid processing on original three-dimensional point cloud data to obtain an initial three-dimensional geometric model; performing spatial registration on a spectral reflection curve and the initial three-dimensional geometric model to obtain a three-dimensional geometric model containing material spectral properties; analyzing a facet spectral emission curve in combination with a scene space-time cultural database to generate a parameterized physical material library; generating a basic light image in combination with the parameterized physical material library and the scene space-time cultural database according to experience space-time information selected by a user; fusing environmental special effect physical description parameters and the basic light image to obtain a synthesized visual picture result; and generating an immersive experience instruction based on the synthesized visual picture result and environmental special effect data. The method can dynamically adapt to material characteristics and space-time changes of different intangible cultural heritage scenes, and improves the restoration degree of the intangible cultural heritage scene and the user immersion.

Description

Technical Field

[0001] This invention belongs to the field of virtual reality technology, and in particular relates to methods, devices, equipment and media for immersive experiences of intangible cultural heritage based on virtual reality. Background Technology

[0002] The application of virtual reality (VR) technology in the transmission of intangible cultural heritage is receiving increasing attention. Through immersive experiences, it allows users to deeply appreciate the charm of traditional culture, effectively protecting and disseminating VR, and stimulating public interest and identification with history and culture. In the virtual reproduction of VR scenes, how to recreate the authentic atmosphere of traditional scenes using technological means has become crucial for promoting the popularization of cultural experiences. VR not only needs to present the visual details of VR, but also needs to enhance users' immersion and sense of belonging to historical scenes through dynamic lighting effects and environmental atmosphere. However, current technological solutions still face many challenges in achieving this goal, and breakthroughs are urgently needed to meet the deeper needs of cultural transmission.

[0003] Existing methods often struggle to balance historical authenticity with adaptability to dynamic environments when generating the atmosphere of intangible cultural heritage scenes. Many solutions rely on static lighting settings or preset scene templates, making it difficult to flexibly adjust to the unique architectural structures and cultural backgrounds of different intangible cultural heritage scenes. For example, the wooden structures of traditional workshops and the gray bricks and tiles of ancient buildings present drastically different visual effects under different lighting conditions at different times of day, but existing technologies cannot adapt in real time based on the material properties and spatiotemporal changes of the scene. Furthermore, the color temperature and brightness characteristics of traditional light sources such as oil lamps and candles, as well as the decorative lighting and smoke effects unique to festival activities, are difficult to realistically reproduce due to the lack of intelligent generation mechanisms. These shortcomings result in a lack of depth in the expression of cultural atmosphere in virtual scenes, limiting the user's immersive experience.

[0004] In the construction of virtual reality for intangible cultural heritage scenes, the core technical challenges lie in the dynamic and intelligent generation capabilities of lighting and shadow rendering. Firstly, the architectural structures and material properties of scenes vary greatly. For example, the diffuse reflection of wooden beams and columns and the refraction of paper windows require precise calculations based on the angle of light and material characteristics, which current technologies struggle to achieve in real-time dynamic adjustments. This further leads to another technical challenge: how to automatically generate lighting atmospheres and environmental effects that conform to historical authenticity, according to the needs of cultural activities. For instance, in virtual festival and sacrificial scenes, the faint glow of incense and the dynamic effects of smoke particles need to be coordinated with the overall lighting of the scene. However, existing methods often fail to intelligently generate corresponding lighting and particle effects based on the temporal and spatial characteristics of cultural activities, thus affecting the cultural expressiveness of the scene. Summary of the Invention

[0005] Therefore, it is necessary to address the aforementioned technical issues by providing a virtual reality-based immersive experience method, device, equipment, and medium for intangible cultural heritage that can dynamically generate historically authentic lighting and environmental effects based on the architectural structure, material properties, and cultural activity characteristics of the intangible cultural heritage scene.

[0006] Firstly, this application provides a method for immersive intangible cultural heritage experiences based on virtual reality, including:

[0007] Acquire multi-dimensional data of the target intangible cultural heritage scene; the multi-dimensional data includes the original three-dimensional point cloud data and the spectral reflectance curves of different materials in the target intangible cultural heritage scene;

[0008] The original 3D point cloud data is meshed to obtain an initial 3D geometric model;

[0009] The spectral reflectance curve is spatially registered with the initial three-dimensional geometric model to obtain the three-dimensional geometric model; the three-dimensional geometric model includes the material spectral properties.

[0010] Based on a pre-defined scene spatiotemporal culture database, the spectral emission curves of each facet in the 3D geometric model are analyzed to generate a parametric physical material library; the scene spatiotemporal culture database includes environmental effect physical description parameters;

[0011] Obtain the spatiotemporal information of the user's selected experience, and generate basic lighting and shadow images based on the parametric physical material library and the scene spatiotemporal culture database;

[0012] Based on the physical description parameters of environmental effects and the basic lighting and shadow images, a composite visual image is generated; the composite visual image includes environmental effects data.

[0013] Based on the synthesized visual results and environmental effects data, immersive experience instructions are generated. These instructions are used to instruct preset devices to generate data that meets the user's spatiotemporal information requirements to achieve an immersive experience. The preset devices include tactile and auditory devices.

[0014] In one embodiment, the original 3D point cloud data is meshed to obtain an initial 3D geometric model, including:

[0015] The original 3D point cloud data is denoised to obtain denoised point cloud data;

[0016] An initial triangular mesh model is generated based on the denoised point cloud data.

[0017] The initial triangular mesh model is topologically repaired to obtain the repaired triangular mesh model.

[0018] The repaired triangular mesh model is simplified to obtain a simplified triangular mesh model.

[0019] The simplified triangular mesh model is smoothed to obtain the initial three-dimensional geometric model.

[0020] In one embodiment, the spectral reflectance curve is spatially registered with an initial three-dimensional geometric model to obtain a three-dimensional geometric model, including:

[0021] Material boundaries are identified from the initial 3D geometric model to obtain the material boundaries.

[0022] By spatially aligning the spectral reflectance curve with the material boundary, an aligned 3D geometric model is obtained.

[0023] Based on a pre-defined material property database, the spectral reflectance curves are mapped onto each facet of the aligned 3D geometric model, and the material spectral properties of each facet are calculated to obtain the 3D geometric model. The material property database contains material spectral property parameters corresponding to different spectral reflectance ranges.

[0024] In one embodiment, based on a preset material property database, spectral reflectance curves are mapped onto each facet of the aligned 3D geometric model, and the material spectral properties of each facet are calculated to obtain the 3D geometric model, including:

[0025] Each facet of the aligned 3D geometric model is numbered to obtain a set of numbered faces;

[0026] Based on the spectral reflectance curve, the spectral reflectance data corresponding to each numbered patch in the set of numbered patches is extracted; the spectral reflectance data includes the spectral reflectance of each patch.

[0027] Based on the preset material property database, obtain the initial material parameter set for each match;

[0028] Based on the initial material parameter set, the material spectral properties of each facet are calculated using the following formula:

[0029]

[0030] in, The material spectral properties for each facet. Indicates the first The weights of the initial material parameters; Indicates the first The first piece of the face Initial material parameters, Indicates the first Each patch at wavelength Spectral reflectance at that location Indicates the total number of initial material parameters;

[0031] By associating the material spectral properties of all facets with their corresponding numbered facets, a three-dimensional geometric model is obtained.

[0032] In one embodiment, based on a preset scene spatiotemporal cultural database, the spectral emission curves of each facet in the three-dimensional geometric model are analyzed to generate a parametric physical material library, including:

[0033] Based on the scene's spatiotemporal cultural database, the ambient lighting parameters are obtained;

[0034] Based on ambient lighting parameters, the emitted radiance of each facet is calculated using the following formula, yielding the spectral emission curves for each facet:

[0035]

[0036] in, Indicates wavelength The emitted radiation brightness at that location, Represents the bidirectional reflection distribution function. Indicates the incident radiation brightness. Indicates the direction of incidence. Indicates the direction of launch. Represents the normal vector of a surface. This represents the solid angular domain composed of all incident directions;

[0037] Convert the spectral emission curve into a parametric material descriptor to generate a parametric physical material library.

[0038] In one embodiment, the user-selected spatiotemporal information of the experience is obtained, and a basic lighting and shadow image is generated based on a parametric physical material library and a scene spatiotemporal cultural database, including:

[0039] Obtain the spatiotemporal information selected by the user for the experience; the spatiotemporal information includes time, weather, and geographical location;

[0040] Based on the spatiotemporal information of the experience and the spatiotemporal culture database of the scene, the corresponding lighting parameters are determined;

[0041] Based on the parametric physical material library and lighting parameters, the lighting intensity of each facet is obtained using the following formula:

[0042]

[0043] in, Represents a piece of dough Light intensity, Indicates the light source The radiation intensity, Represents the bidirectional reflection distribution function. Indicates the direction of observation. Indicates the direction of incidence. Represents a piece of dough The normal vector, The solid angle representing the direction of incidence. Indicates the total number of light sources;

[0044] Based on the light intensity and the spatial coordinate information of the three-dimensional geometric model, the three-dimensional geometric model is converted into a two-dimensional image to obtain the basic two-dimensional image.

[0045] Texture mapping is performed on a basic two-dimensional image to obtain a two-dimensional texture image;

[0046] Color rendering is performed on the basic 2D texture image to generate a basic lighting and shadow image.

[0047] In one embodiment, based on environmental effects physical description parameters and basic lighting and shadow images, a synthesized visual image result is generated, including:

[0048] Based on the physical description parameters of environmental effects, environmental effects are simulated to obtain environmental effect data;

[0049] The environmental effects data is fused with the basic lighting and shadow images to obtain the initial visual result;

[0050] Color correction is performed on the initial visual image to obtain the corrected visual image.

[0051] The contrast of the corrected visual image is enhanced to obtain the synthesized visual image.

[0052] Secondly, this application also provides an immersive intangible cultural heritage experience device based on virtual reality, comprising:

[0053] The data acquisition module is used to acquire multi-dimensional data of the target intangible cultural heritage scene; the multi-dimensional data includes the original three-dimensional point cloud data and the spectral reflectance curves of different materials in the target intangible cultural heritage scene;

[0054] The point cloud data processing module is used to perform meshing processing on the original 3D point cloud data to obtain the initial 3D geometric model.

[0055] The geometric model generation module is used to spatially register the spectral reflectance curve with the initial 3D geometric model to obtain the 3D geometric model; the 3D geometric model includes material spectral properties.

[0056] The parametric material library generation module is used to analyze the spectral emission curves of each facet in the 3D geometric model based on a preset scene spatiotemporal culture database, and generate a parametric physical material library; the scene spatiotemporal culture database includes environmental effect physical description parameters;

[0057] The lighting and shadow image generation module is used to obtain the spatiotemporal information of the user's selected experience and generate basic lighting and shadow images based on the parametric physical material library and the scene spatiotemporal culture database.

[0058] The visual image compositing module is used to generate composite visual images based on environmental effects physical description parameters and basic lighting and shadow images; the composite visual images include environmental effects data.

[0059] The immersive experience instruction generation module is used to generate immersive experience instructions based on the results of synthesized visual images and environmental effects data. Immersive experience instructions are used to instruct preset devices to generate data that meets the user's spatiotemporal information needs to achieve an immersive experience. The preset devices include tactile devices and auditory devices.

[0060] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0061] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0062] The aforementioned virtual reality-based immersive experience method, device, equipment, and medium for intangible cultural heritage (ICH) accurately acquires multi-dimensional data of ICH scenes through an immersive experience terminal. This data is then meshed and spatially registered to construct a 3D model containing material spectral attributes. A parametric physical material library is generated by analyzing a scene's spatiotemporal cultural database. Based on the user's selected spatiotemporal information, basic lighting and shadow images are generated and integrated with environmental effects, ultimately outputting immersive experience commands. This method dynamically adapts to the material characteristics and spatiotemporal changes of different ICH scenes, realistically reproducing traditional material lighting and shadow effects, as well as the atmosphere of festivals, weather, and other environmental elements. It ensures the historical authenticity and detailed integrity of ICH scenes and enhances user immersion and engagement through multi-sensory interaction. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a schematic diagram of the application environment of a virtual reality-based immersive experience method for intangible cultural heritage in one embodiment;

[0065] Figure 2This is a flowchart illustrating a virtual reality-based immersive experience method for intangible cultural heritage in one embodiment.

[0066] Figure 3 This is a schematic diagram of the structure of a virtual reality-based immersive experience device for intangible cultural heritage in one embodiment;

[0067] Figure 4 This is a schematic diagram of a computer device in one embodiment. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0069] The virtual reality-based immersive experience method for intangible cultural heritage provided in this application can be applied to, for example... Figure 1 In the application environment shown, the immersive experience terminal 101 establishes connections with the modeling device 102, the sound device 103, and the interactive device terminal 104 via a wireless network. After the user selects the spatiotemporal information to be experienced through the interactive device terminal 104, the immersive experience terminal 101 acquires the 3D data of the intangible cultural heritage scene generated by the modeling device 102 and drives the sound device 103 to generate audio content matching the spatiotemporal information. Finally, the user achieves multi-sensory interaction with the virtual intangible cultural heritage scene through the interactive device terminal 104 (including tactile, auditory, and other interactive modules), thereby completing the immersive experience. The immersive experience terminal 101 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc.; the modeling device 102 can be a 3D scanner, etc.; the sound device 103 can be bone conduction headphones and head-tracking audio, etc.; and the interactive device terminal 104 can be a head-mounted device, etc.

[0070] In one exemplary embodiment, such as Figure 2 As shown, a method for immersive intangible cultural heritage experience based on virtual reality is provided, which can be applied to... Figure 1 Taking the immersive experience terminal in the example, the following steps are included:

[0071] S1: Obtain multi-dimensional data of the target intangible cultural heritage scene.

[0072] The immersive experience terminal acquires multi-dimensional data of the target intangible cultural heritage scene through pre-set acquisition devices. These devices can include 3D laser scanners, spectrometers, etc. The raw 3D point cloud data is obtained by collecting the spatial geometry of the intangible cultural heritage scene using a 3D laser scanner. It contains the 3D coordinates of each point in the scene, accurately reconstructing its spatial form. The spectral reflectance curves of different materials within the target intangible cultural heritage scene are obtained by measuring the reflectance characteristics of various materials under different wavelengths of light using a spectrometer. These curves reflect the material's ability to reflect different wavelengths of light. The acquisition devices can include 3D laser scanners, spectrometers, etc.; the materials can include wooden utensils, textiles, metal ornaments, etc., from intangible cultural heritage crafts.

[0073] S2, the original 3D point cloud data is meshed to obtain the initial 3D geometric model.

[0074] For example, the immersive experience terminal first preprocesses the original 3D point cloud data to remove outliers caused by device noise and environmental interference, ensuring the accuracy of the point cloud data. Then, a meshing algorithm is used to convert the discrete 3D point cloud data into a continuous geometric surface structure. The meshing algorithm works by calculating the spatial topological relationships between point clouds to construct a mesh model with triangular facets as basic units, efficiently fitting the complex geometric forms of intangible cultural heritage scenes. During the meshing process, the terminal optimizes the distribution density of facets, preserving key geometric features of the scene while ensuring the efficiency of subsequent model processing, ultimately generating an initial 3D geometric model that fully presents the 3D form of the intangible cultural heritage scene. These geometric features can include architectural carvings, artifact textures, etc.

[0075] S3. Spatial registration of the spectral reflectance curve with the initial three-dimensional geometric model is performed to obtain the three-dimensional geometric model.

[0076] Optionally, the immersive experience terminal first divides the initial 3D geometric model into material regions. By analyzing the changes in geometric curvature and texture differences on the model surface, it identifies the distribution areas and boundaries of different materials on the model. Next, spatial registration is performed, establishing a spatial coordinate relationship between the previously acquired spectral reflectance curves of each material and the corresponding material regions on the model, ensuring that each spectral reflectance curve accurately matches the corresponding material region on the model. During this process, the terminal uses a coordinate system algorithm to eliminate any coordinate deviations that may exist between the spectral data and the geometric model during acquisition. Then, based on the spectral reflectance curves, it calculates the optical characteristic parameters of the corresponding material regions and assigns these parameters to the corresponding facets of the model, ultimately obtaining a 3D geometric model containing the spectral properties of the materials, giving the model a realistic foundation for material optical representation. These optical characteristic parameters include reflectivity and refractive index.

[0077] S4 analyzes the spectral emission curves of each facet in the 3D geometric model based on the preset scene spatiotemporal culture database, and generates a parametric physical material library.

[0078] The scene spatiotemporal culture database includes environmental effect physical description parameters, reflecting the lighting and environmental effect characteristics of different scenes under different time and space conditions. For example, the immersive experience terminal analyzes the spectral emission curves of each facet in the 3D geometric model based on the preset scene spatiotemporal culture database. By analyzing the spectral emission curves, the reflection and emission characteristics of each facet under different lighting conditions can be extracted and converted into parametric material descriptors, thereby generating a parametric physical material library.

[0079] S5 acquires the spatiotemporal information of the user's selected experience and generates basic lighting and shadow images based on the parametric physical material library and the scene spatiotemporal culture database.

[0080] Optionally, the immersive experience terminal receives the user's selected spatiotemporal information via an interactive interface. This information includes historical time, weather conditions, and specific location (such as the center of a courtyard or next to a workshop workbench). Based on this spatiotemporal information, the terminal retrieves the corresponding ambient lighting parameters from the scene spatiotemporal cultural database. It then calls the material parameters of each surface in the parametric physical material library, calculates the lighting response of each surface in conjunction with the lighting parameters, and converts the three-dimensional geometric model into a two-dimensional image using rasterization technology. Simultaneously, it overlays the inherent texture information of the materials, ultimately generating a basic light and shadow image that realistically reflects the light and shadow layers and material texture of the intangible cultural heritage scene within the target spatiotemporal context.

[0081] S6 generates composite visual results based on environmental effects physical description parameters and basic lighting and shadow images.

[0082] Specifically, the immersive experience terminal extracts physical description parameters of environmental effects from a scene spatiotemporal cultural database that match the user experience's spatiotemporal context, such as particle parameters for fireworks in a festive scene or water vapor concentration parameters for a humid scene. Based on these parameters, the terminal uses technologies such as particle systems and fluid dynamics simulation to generate corresponding environmental effect data. Subsequently, an image fusion algorithm is used to overlay the environmental effect data with the basic lighting and shadow images, ensuring that the effects and scene background are naturally coordinated in terms of spatial position and lighting interaction. After fusion, the terminal performs color correction on the image to make the colors conform to the visual characteristics of the historical scene, and then performs contrast enhancement processing to highlight scene details, ultimately obtaining a synthetic visual image containing environmental effects. The environmental effect data can include fireworks animations and water vapor diffusion effect data, while scene details include textures of utensils and architectural carvings.

[0083] S7 generates immersive experience commands based on synthesized visual results and environmental effects data.

[0084] Specifically, the immersive experience terminal integrates the synthesized visual image results with environmental effects data to construct immersive experience instructions encompassing multi-sensory control logic. These instructions instruct preset devices to generate data that meets the user's spatiotemporal information requirements, thus achieving an immersive experience. Visual control information explicitly displays the required screen resolution and frame rate to ensure smooth presentation of the synthesized visual image. Tactile control information is generated based on scene characteristics, such as texture feedback parameters when touching wooden components or pressure feedback parameters when feeling raindrops, guiding tactile devices to output corresponding tactile signals. Auditory control information is generated in conjunction with scene spatiotemporal features, such as the sounds of gongs and drums during festivals or the sounds of tools being struck in a workshop, instructing auditory devices to output matching audio; these devices can be surround sound headphones. The terminal encapsulates this control information into standardized instructions and sends them to preset tactile, auditory, and display devices. The devices then generate corresponding data outputs based on these instructions, allowing the user to simultaneously obtain an immersive experience consistent with the target intangible cultural heritage scene across visual, tactile, and auditory dimensions.

[0085] The aforementioned virtual reality-based immersive experience method for intangible cultural heritage (ICH) accurately acquires multi-dimensional data of ICH scenes, combining gridding, spatial registration, and material spectral attribute calculations to construct a 3D model that combines geometric accuracy and material realism. Furthermore, relying on a scene spatiotemporal cultural database, it achieves dynamic adaptation of lighting effects and environmental special effects, both restoring the authentic lighting atmosphere of ICH scenes under different materials and time periods, and intelligently generating traditional light sources, festival effects, and other cultural elements. This effectively solves the problems of insufficient historical authenticity and poor adaptability to dynamic environments caused by static lighting and scene templates, creating an immersive ICH experience for users that combines visual detail and cultural atmosphere.

[0086] In an optional embodiment, the original 3D point cloud data is meshed to obtain an initial 3D geometric model, including the following steps:

[0087] S11 performs denoising processing on the original 3D point cloud data to obtain denoised point cloud data.

[0088] Optionally, the immersive experience terminal uses a filtering algorithm to process the original 3D point cloud data. Since the original point cloud data may be affected by factors such as equipment precision and environmental interference during the acquisition process, it contains noise points. The filtering algorithm can identify and remove these noise points, allowing the point cloud data to more accurately reflect the true geometric structure of the intangible cultural heritage scene, resulting in denoised point cloud data. The filtering algorithm can be statistical filtering, radius filtering, etc. Noise points are points that do not conform to the geometric features of the real scene.

[0089] S12 generates an initial triangular mesh model based on the denoised point cloud data.

[0090] Optionally, the immersive experience terminal uses the denoised point cloud data as input, calculates the normal vector of the point cloud using the Poisson surface reconstruction algorithm, and then constructs an implicit function such that the function is positive inside the point cloud and negative outside. The zero isosurface of the function is the fitted scene surface, ultimately generating an initial triangular mesh model composed of triangular facets. This model initially presents the three-dimensional surface morphology of the intangible cultural heritage scene. The principle of the Poisson surface reconstruction algorithm is to fit the surface of the point cloud data by constructing an implicit function.

[0091] S13, perform topology repair on the initial triangular mesh model to obtain the repaired triangular mesh model.

[0092] Specifically, the initial triangular mesh model may have topological defects, such as overlapping faces, holes, and non-manifold edges (i.e., edges that belong to multiple faces simultaneously). The immersive experience terminal uses a topology repair algorithm to handle these topological defects. For example, for holes, new triangular faces are generated to fill them based on the geometric features and topological relationships of the surrounding faces; for non-manifold edges, the connection relationships of the faces are adjusted to conform to a manifold structure (each edge belongs to at most two faces), thus obtaining a topologically corrected triangular mesh model.

[0093] S14. The repaired triangular mesh model is simplified to obtain a simplified triangular mesh model.

[0094] Specifically, the repaired triangular mesh model may have a large number of faces, which is not conducive to subsequent real-time rendering and processing. The immersive experience terminal uses a mesh simplification algorithm to reduce the number of triangular faces while maintaining the overall geometric features of the model. The edge folding algorithm calculates the folding cost of each edge, prioritizing the folding of edges with lower costs, thus obtaining a simplified triangular mesh model with fewer faces but a similar geometric shape. Here, the mesh simplification algorithm can be an edge folding algorithm; the folding cost is the degree of change in the geometric features of the model after folding that edge.

[0095] S15, smooth the simplified triangular mesh model to obtain the initial three-dimensional geometric model.

[0096] Optionally, the simplified triangular mesh model may suffer from surface unevenness due to the simplification process. The immersive experience terminal addresses this issue using a smoothing algorithm. This smoothing algorithm can be a Laplacian smoothing algorithm. The Laplacian smoothing algorithm calculates the average position of each vertex's neighboring vertices, then moves that vertex a certain distance towards the average position. After multiple iterations, the model surface becomes smoother, ultimately yielding an initial 3D geometric model that can smoothly and accurately represent the 3D geometric form of the intangible cultural heritage scene.

[0097] In an optional embodiment, the spectral reflectance curve is spatially registered with the initial three-dimensional geometric model to obtain the three-dimensional geometric model, including the following steps:

[0098] S21, perform material boundary identification on the initial 3D geometric model to obtain the material boundary.

[0099] For example, the immersive experience terminal performs material boundary identification on the initial 3D geometric model to determine the boundaries between different materials in the model. Material boundary identification can be achieved by analyzing the geometric features and spectral reflectance characteristics of the geometric model. Specifically, material boundaries are identified by detecting changes in curvature and abrupt changes in spectral reflectance within the geometric model. The resulting material boundaries are shown. Geometric features include surface roughness and shape differences.

[0100] S22, spatially align the spectral reflectance curve with the material boundary to obtain an aligned 3D geometric model.

[0101] For example, the immersive experience terminal uses a spatial registration algorithm to spatially align the material region corresponding to the spectral reflectance curve with the material boundary identified on the initial 3D geometric model. Specifically, it matches the coordinate system of the spectral reflectance curve with the coordinate system of the geometric model to ensure that the spectral reflectance curve of each material can accurately correspond to the corresponding material region on the geometric model, thereby obtaining an aligned 3D geometric model. The aligned 3D geometric model is used to establish a spatial correspondence between the spectral information of the material and the geometric structure.

[0102] S23. Based on the preset material property database, the spectral reflectance curve is mapped to each facet of the aligned 3D geometric model, and the material spectral properties of each facet are calculated to obtain the 3D geometric model.

[0103] The material attribute database contains material spectral attribute parameters corresponding to different spectral reflectance ranges. Specifically, the immersive experience terminal maps the spectral reflectance curve to each facet of the aligned 3D geometric model based on the preset material attribute database and calculates the material spectral attributes of each facet. By searching the material attribute database, the spectral reflectance data in the spectral reflectance curve can be converted into specific material spectral attributes. The final 3D geometric model not only includes the geometry of the scene but also the material spectral attributes of each facet.

[0104] In an optional embodiment, based on a preset material property database, the spectral reflectance curve is mapped to each facet of the aligned 3D geometric model, and the material spectral properties of each facet are calculated to obtain the 3D geometric model, including the following steps:

[0105] S31, number each facet of the aligned 3D geometric model to obtain a set of numbered faces.

[0106] For example, the immersive experience terminal numbers each facet of the aligned 3D geometric model. By assigning a unique number to each facet, all facets can be organized into a numbered facet set, facilitating individual manipulation and analysis of each facet. The numbered facet set provides a clear data structure for subsequent spectral reflectance data extraction and material spectral property calculation, ensuring that each facet can be processed accurately.

[0107] S32, based on the spectral reflectance curve, extract the spectral reflectance data corresponding to each numbered patch in the set of numbered patches.

[0108] The spectral reflectance data includes the spectral reflectance of each surface patch. The immersive experience terminal extracts the spectral reflectance data for each numbered surface patch from the set of numbered patches based on the spectral reflectance curve. Spectral reflectance data reflects the reflectance characteristics of each surface patch under different wavelengths of light and is fundamental to calculating the spectral properties of the material. By analyzing the spectral reflectance curve, the spectral reflectance of each numbered surface patch at various wavelengths can be extracted, providing accurate input data for subsequent calculations of the material's spectral properties.

[0109] S33, based on the preset material property database, obtain the initial material parameter set for each matching material.

[0110] For example, the immersive experience terminal obtains each matching initial material parameter set based on a preset material attribute database. The material attribute database stores initial material parameter sets corresponding to different spectral reflectance ranges. By searching the material attribute database, a matching initial material parameter set can be found based on the extracted spectral reflectance data. These initial material parameter sets contain the basic physical properties of the material, such as reflectance and refractive index, providing fundamental parameters for subsequent calculations of the material's spectral properties.

[0111] S34, based on the initial material parameter set, calculate the material spectral properties of each facet using the following formula:

[0112]

[0113] in, The material spectral properties for each facet. Indicates the first The weights of the initial material parameters of the item. Indicates the first The first piece of the face Initial material parameters, Indicates the first Each patch at wavelength Spectral reflectance at that location This indicates the total number of initial material parameters.

[0114] Specifically, the immersive experience terminal calculates the material spectral properties of each surface patch using a preset formula based on an initial set of material parameters. These material spectral properties reflect the reflection and emission characteristics of the surface patch under different lighting conditions. The formula for calculating these material spectral properties includes... The material spectral properties of each facet comprehensively reflect the optical characteristics (such as reflection and refraction) of the facet material under specific wavelengths of light. Indicates the first The initial material parameter weights are used to adjust the influence of different material parameters on the final spectral properties, and all weights satisfy the following conditions: . Indicates the first The first piece of the face The initial material parameters are the basic parameters that describe the physical properties of the material and are obtained from the material property database. Indicates the first Each patch at wavelength The spectral reflectance at a certain point is used to reflect the ability of the sheet material to reflect light of a specific wavelength. This represents the total number of initial material parameters, specifically the total number of parameters used to describe material properties. By calculating the material spectral properties of each facet, more accurate material information can be provided for subsequent rendering and user experience.

[0115] S35 associates the material spectral properties of all facets with the corresponding numbered facets to obtain a three-dimensional geometric model.

[0116] For example, the immersive experience terminal associates the material spectral properties of all facets with their corresponding numbered facets to obtain the final 3D geometric model. By mapping the material spectral properties to the numbered facets, it can be ensured that each facet has accurate material information. The final 3D geometric model not only includes the geometry of the scene but also the material spectral properties of each facet, providing a complete data foundation for subsequent rendering and immersive experiences.

[0117] In an optional embodiment, based on a preset scene spatiotemporal cultural database, the spectral emission curves of each facet in the 3D geometric model are analyzed to generate a parametric physical material library, including the following steps:

[0118] S41, obtain the ambient lighting parameters based on the scene's spatiotemporal cultural database.

[0119] The scene spatiotemporal culture database stores the lighting characteristics of different scenes under various time and spatial conditions, including information such as the direction, intensity, and color of the light source. The immersive experience terminal obtains ambient lighting parameters based on this database. By querying the database, the corresponding ambient lighting parameters can be obtained based on the user's selected experience time and space information. These lighting parameters are fundamental to generating realistic lighting effects and reflect the lighting environment of the scene under specific spatiotemporal conditions.

[0120] S42, based on ambient lighting parameters, the emitted radiance of each facet is calculated using the following formula, yielding the spectral emission curves for each facet:

[0121]

[0122] in, Indicates wavelength The emitted radiation brightness at that location, Represents the bidirectional reflection distribution function. Indicates the incident radiation brightness. Indicates the direction of incidence. Indicates the direction of launch. Represents the normal vector of a surface. It represents the solid angular domain composed of all incident directions.

[0123] Specifically, the immersive experience terminal calculates the emitted radiance of each panel using a preset formula based on ambient lighting parameters, obtaining the spectral emission curve of each panel. The spectral emission curve reflects the reflection and emission characteristics of the panel under specific lighting conditions. In the formula for calculating the emitted radiance, Indicates wavelength The emitted radiation brightness at the point, the plate at the wavelength The intensity of radiation emitted in the downward emission direction is a key indicator for evaluating the optical performance of a surface plate. This represents the bidirectional reflectance distribution function, used to describe the material's reflection at wavelengths of light and shadow. Down, from the incident direction To the direction of launch The reflective properties reflect the material's reflection characteristics. Indicates the incident radiance, i.e., from the incident direction. Incident on the surface, wavelength is The radiation intensity. This indicates the incident direction, that is, the direction vector of the light ray striking the surface. It indicates the direction of emission, that is, the direction vector of the received light. The normal vector of the surface is a vector perpendicular to the surface of the surface and is used to determine the angle between the incident ray and the surface of the surface. It represents the solid angular domain consisting of all incident directions, and the integral range covers all possible incident directions of light received by the surface.

[0124] S43 converts the spectral emission curve into a parametric material descriptor, generating a parametric physical material library.

[0125] Optionally, the immersive experience terminal extracts and parametrically processes the spectral emission curves of each surface patch, converting them into parametric material descriptors. These descriptors are quantitative representations of the material's optical properties, including physical property parameters such as reflection and refraction under different lighting and viewing conditions. The parametric material descriptors of all surfaces are then stored to generate a parametric physical material library, which provides physical property data support for the subsequent generation of lighting and shadow images.

[0126] In an optional embodiment, the user-selected spatiotemporal information is obtained, and a basic lighting and shadow image is generated based on a parametric physical material library and a scene spatiotemporal cultural database, including the following steps:

[0127] S51, obtains the spatiotemporal information of the experience selected by the user.

[0128] Optionally, the immersive experience terminal acquires the spatiotemporal information selected by the user, including time, weather, and geographical location. This spatiotemporal information reflects the specific time and environmental conditions of the intangible cultural heritage scene the user wishes to experience. Time information can affect the intensity and direction of light in the scene, weather information can affect the lighting effects and environmental special effects, and geographical location information can affect the lighting conditions and background. By acquiring this spatiotemporal information, a personalized immersive experience can be provided to the user.

[0129] S52 determines the corresponding lighting parameters based on the experience spatiotemporal information and the scene spatiotemporal cultural database.

[0130] For example, the immersive experience terminal determines the corresponding lighting parameters based on the experience's spatiotemporal information and a scene spatiotemporal culture database. The scene spatiotemporal culture database stores the lighting characteristics of different scenes under different time and spatial conditions. By querying the database, the corresponding lighting parameters can be obtained based on the user's selected experience spatiotemporal information. These lighting parameters include information such as the direction, intensity, and color of the light source, reflecting the lighting environment of the scene under specific spatiotemporal conditions. By determining the lighting parameters, an accurate lighting foundation can be provided for subsequent rendering and experience.

[0131] S53, based on the parametric physical material library and lighting parameters, the lighting intensity of each facet is obtained using the following formula:

[0132]

[0133] in, Represents a piece of dough Light intensity, Indicates the light source The radiation intensity, Represents the bidirectional reflection distribution function. Indicates the direction of observation. Indicates the direction of incidence. Represents a piece of dough The normal vector, The solid angle representing the direction of incidence. This indicates the total number of light sources.

[0134] Specifically, the immersive experience terminal calculates the illumination intensity of each surface using a formula based on a parametric physical material library and lighting parameters. The illumination intensity reflects the amount of light energy received by the surface. In the formula for calculating the illumination intensity, Represents a piece of dough Light intensity, surface The brightness level under the current lighting conditions. Indicates the light source The radiation intensity, the first The intensity of radiation energy emitted by each light source. This represents the bidirectional reflection distribution function, used to describe the reflection of a surface material from the incident direction. To the direction of observation Its reflective properties. This indicates the viewing direction, i.e., the direction vector in which the user or virtual camera observes the patch. Indicates the incident direction, i.e., the first... A direction vector from which a light source strikes a surface. Represents a piece of dough The normal vector is perpendicular to the surface. The vector of the surface is used to determine the angular relationship between the incident ray and the surface of the patch. The solid angle represents the incident direction and is used to quantify the coverage area of ​​that incident direction in space. This represents the total number of light sources, i.e., the total number of light sources involved in the illumination calculation.

[0135] S54. Based on the illumination intensity and the spatial coordinate information of the three-dimensional geometric model, the three-dimensional geometric model is converted into a two-dimensional image to obtain the basic two-dimensional image.

[0136] For example, the immersive experience terminal converts the 3D geometric model into a 2D image based on the light intensity and the spatial coordinate information of the 3D geometric model, obtaining a base 2D image. This process is achieved through projection transformation, mapping the points and faces in the 3D geometric model onto a 2D plane according to the viewpoint and projection method. The base 2D image contains the geometric shape and lighting information of the scene and is the foundation for generating the final visual effect. By converting the 3D model into a 2D image, the terminal can provide data support for subsequent texture mapping and color rendering.

[0137] S55 performs texture mapping on the basic two-dimensional image to obtain a two-dimensional texture image.

[0138] Specifically, the immersive experience terminal performs texture mapping on a basic 2D image, applying texture information to the 2D image to enhance its detail and realism. Texture mapping is achieved by mapping pixel values ​​in a texture image to corresponding locations in the 2D image. The texture image can be a predefined texture map or a dynamically generated texture based on the scene. Through texture mapping, each facet of the 2D image possesses corresponding texture details, thus more realistically reflecting the material characteristics of the scene.

[0139] S56 performs color rendering on the basic two-dimensional texture image to generate a basic light and shadow image.

[0140] For example, the immersive experience terminal performs color rendering on a basic two-dimensional texture image to generate a basic lighting and shadow image. The color rendering process calculates the final color value of each pixel by considering factors such as light intensity, texture color, and material properties. During rendering, the propagation and reflection of light in the scene are simulated based on the lighting and material models, thereby generating an image with variations in brightness, shadow effects, and reflection characteristics. The basic lighting and shadow image is used to reflect the visual effects of the scene under specific lighting conditions.

[0141] In an optional embodiment, based on environmental effects physical description parameters and basic lighting and shadow images, a synthetic visual image result is generated, including the following steps:

[0142] S61 simulates environmental effects based on the physical description parameters of environmental effects to obtain environmental effect data.

[0143] Optionally, the immersive experience terminal simulates environmental effects based on physical description parameters of environmental effects to obtain environmental effect data. These physical description parameters include the physical characteristics of natural phenomena such as fog, rain, and snow, such as density, speed, and shape. By simulating these physical characteristics, the terminal can generate corresponding environmental effect data, including the concentration distribution of fog and the trajectory of raindrops. This environmental effect data enhances the realism and immersion of the scene.

[0144] S62 integrates environmental effects data with basic lighting and shadow images to obtain the initial visual result.

[0145] For example, the immersive experience terminal fuses environmental effects data with basic lighting and shadow images to organically combine environmental effects with the scene's lighting and shadow effects. This fusion process can be achieved through image compositing algorithms, overlaying the environmental effects data as an additional layer onto the basic lighting and shadow image, and adjusting the fusion parameters according to the physical characteristics of the environmental effects and the scene's lighting conditions. The initial visual result includes both the scene's lighting and shadow effects and the environmental effects.

[0146] S63 performs color correction on the initial visual image to obtain the corrected visual image.

[0147] Specifically, the immersive experience terminal performs color correction on the initial visual image to adjust the image's color balance, making it more consistent with the user's visual habits and the realistic colors of the scene. Color correction can be achieved by adjusting parameters such as color temperature, saturation, and contrast, ensuring that the image's colors remain consistent and accurate across different display devices. The corrected visual image appears more natural and realistic in its color representation.

[0148] S64, perform contrast enhancement on the corrected visual image result to obtain the synthesized visual image result.

[0149] Specifically, the immersive experience terminal enhances the contrast of the corrected visual image to highlight details and depth. Contrast enhancement is achieved by adjusting the brightness distribution of the image, making dark areas darker and bright areas brighter, thereby enhancing the visual effect. The synthesized visual image includes the scene's lighting effects, environmental effects, and optimized colors and contrast, providing users with a realistic and immersive visual experience.

[0150] The aforementioned virtual reality-based immersive experience method for intangible cultural heritage (ICH) achieves dynamic analysis of material spectral properties and construction of a parameterized physical material library by accurately collecting and processing multi-dimensional data of ICH scenes and combining it with a scene spatiotemporal cultural database. Based on the user's selected spatiotemporal information, it generates appropriate lighting and environmental effects, ultimately outputting multi-sensory immersive experience commands. Through spatial registration of the spectral reflectance curves of different materials and physical-based lighting calculations, it achieves precise adaptation of the architectural structure, material properties, and dynamic lighting of ICH scenes, breaking the limitations of static lighting settings or templated schemes. It can present realistic visual effects in real time based on different materials such as wood and bricks, as well as changes in time of day. Utilizing physical description parameters and intelligent generation mechanisms for environmental effects, it can dynamically simulate the characteristics of traditional light sources such as oil lamps and incense, as well as environmental effects such as festive smoke, making the cultural atmosphere of the virtual scene highly consistent with historical authenticity, thus enhancing the user's immersion and sense of belonging to the ICH scene.

[0151] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0152] Based on the same inventive concept, this application also provides a virtual reality-based immersive experience device for intangible cultural heritage, which implements the aforementioned virtual reality-based immersive experience method for intangible cultural heritage. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the virtual reality-based immersive experience device for intangible cultural heritage provided below can be found in the limitations of the virtual reality-based immersive experience method for intangible cultural heritage described above, and will not be repeated here.

[0153] In one exemplary embodiment, such as Figure 3 As shown, a virtual reality-based immersive experience device for intangible cultural heritage, 200, is provided, comprising:

[0154] The data acquisition module 201 is used to acquire multi-dimensional data of the target intangible cultural heritage scene; the multi-dimensional data includes the original three-dimensional point cloud data and the spectral reflectance curves of different materials in the target intangible cultural heritage scene;

[0155] Point cloud data processing module 202 is used to perform meshing processing on the original 3D point cloud data to obtain an initial 3D geometric model;

[0156] The geometric model generation module 203 is used to spatially register the spectral reflectance curve with the initial three-dimensional geometric model to obtain a three-dimensional geometric model; the three-dimensional geometric model includes material spectral properties.

[0157] The parametric material library generation module 204 is used to analyze the spectral emission curves of each facet in the 3D geometric model based on a preset scene spatiotemporal culture database to generate a parametric physical material library; the scene spatiotemporal culture database includes environmental effect physical description parameters;

[0158] The light and shadow image generation module 205 is used to obtain the spatiotemporal information of the experience selected by the user; and generate basic light and shadow images based on the parametric physical material library and the scene spatiotemporal culture database.

[0159] The visual image compositing module 206 is used to generate a composite visual image result based on the physical description parameters of the environmental effects and the basic lighting and shadow image; the composite visual image result includes environmental effects data.

[0160] The immersive experience instruction generation module 207 is used to generate immersive experience instructions based on the results of the synthesized visual image and environmental effects data. The immersive experience instructions are used to instruct preset devices to generate data that meets the user's spatiotemporal information needs in order to achieve an immersive experience. The preset devices include tactile devices and auditory devices.

[0161] Furthermore, the point cloud data processing module 202 is also used for:

[0162] The original 3D point cloud data is denoised to obtain denoised point cloud data;

[0163] An initial triangular mesh model is generated based on the denoised point cloud data.

[0164] The initial triangular mesh model is topologically repaired to obtain the repaired triangular mesh model.

[0165] The repaired triangular mesh model is simplified to obtain a simplified triangular mesh model.

[0166] The simplified triangular mesh model is smoothed to obtain the initial three-dimensional geometric model.

[0167] Furthermore, the geometric model generation module 203 is also used for:

[0168] Material boundaries are identified from the initial 3D geometric model to obtain the material boundaries.

[0169] By spatially aligning the spectral reflectance curve with the material boundary, an aligned 3D geometric model is obtained.

[0170] Based on a pre-defined material property database, the spectral reflectance curves are mapped onto each facet of the aligned 3D geometric model, and the material spectral properties of each facet are calculated to obtain the 3D geometric model. The material property database contains material spectral property parameters corresponding to different spectral reflectance ranges.

[0171] Furthermore, the geometric model generation module 203 is also used for:

[0172] Each facet of the aligned 3D geometric model is numbered to obtain a set of numbered faces;

[0173] Based on the spectral reflectance curve, the spectral reflectance data corresponding to each numbered patch in the set of numbered patches is extracted; the spectral reflectance data includes the spectral reflectance of each patch.

[0174] Based on the preset material property database, obtain the initial material parameter set for each match;

[0175] Based on the initial material parameter set, the material spectral properties of each facet are calculated using the following formula:

[0176]

[0177] in, The material spectral properties for each facet. Indicates the first The weights of the initial material parameters; Indicates the first The first piece of the face Initial material parameters, Indicates the first Each patch at wavelength Spectral reflectance at that location Indicates the total number of initial material parameters;

[0178] By associating the material spectral properties of all facets with their corresponding numbered facets, a three-dimensional geometric model is obtained.

[0179] Furthermore, the parametric material library generation module 204 is also used for:

[0180] Based on the scene's spatiotemporal cultural database, the ambient lighting parameters are obtained;

[0181] Based on ambient lighting parameters, the emitted radiance of each facet is calculated using the following formula, yielding the spectral emission curves for each facet:

[0182]

[0183] in, Indicates wavelength The emitted radiation brightness at that location, Represents the bidirectional reflection distribution function. Indicates the incident radiation brightness. Indicates the direction of incidence. Indicates the direction of launch. Represents the normal vector of a surface. This represents the solid angular domain composed of all incident directions;

[0184] Convert the spectral emission curve into a parametric material descriptor to generate a parametric physical material library.

[0185] Furthermore, the light and shadow image generation module 205 is also used for:

[0186] Obtain the spatiotemporal information selected by the user for the experience; the spatiotemporal information includes time, weather, and geographical location;

[0187] Based on the spatiotemporal information of the experience and the spatiotemporal culture database of the scene, the corresponding lighting parameters are determined;

[0188] Based on the parametric physical material library and lighting parameters, the lighting intensity of each facet is obtained using the following formula:

[0189]

[0190] in, Represents a piece of dough Light intensity, Indicates the light source The radiation intensity, Represents the bidirectional reflection distribution function. Indicates the direction of observation. Indicates the direction of incidence. Represents a piece of dough The normal vector, The solid angle representing the direction of incidence. Indicates the total number of light sources;

[0191] Based on the light intensity and the spatial coordinate information of the three-dimensional geometric model, the three-dimensional geometric model is converted into a two-dimensional image to obtain the basic two-dimensional image.

[0192] Texture mapping is performed on a basic two-dimensional image to obtain a two-dimensional texture image;

[0193] Color rendering is performed on the basic 2D texture image to generate a basic lighting and shadow image.

[0194] Furthermore, the visual image composition module 206 is also used for:

[0195] Based on the physical description parameters of environmental effects, environmental effects are simulated to obtain environmental effect data;

[0196] The environmental effects data is fused with the basic lighting and shadow images to obtain the initial visual result;

[0197] Color correction is performed on the initial visual image to obtain the corrected visual image.

[0198] The contrast of the corrected visual image is enhanced to obtain the synthesized visual image.

[0199] In one embodiment, such as Figure 4 A computer device 300 is provided, comprising:

[0200] At least one processor 301, and at least one memory 302 communicatively connected to said processor 301; said memory stores application code executable by said processor, said application code being executed by said processor to enable said processor to perform the steps of the virtual reality-based intangible cultural heritage immersive experience method described above;

[0201] The computer device may also include: sensor 303;

[0202] The processor 301, memory 301, and sensor 303 can be connected via bus 304 or other means; the diagram shows an example using bus 304. Figure 4 The character is represented by a single thick line, but this does not mean that there is only one bus or a type of bus.

[0203] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0204] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0205] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A method for immersive intangible cultural heritage experience based on virtual reality, characterized in that, The method includes: Acquire multi-dimensional data of the target intangible cultural heritage scene; the multi-dimensional data includes original three-dimensional point cloud data and spectral reflectance curves of different materials within the target intangible cultural heritage scene; The original 3D point cloud data is meshed to obtain an initial 3D geometric model; The spectral reflectance curve is spatially registered with the initial three-dimensional geometric model to obtain a three-dimensional geometric model; the three-dimensional geometric model includes material spectral properties. Based on a pre-defined scene spatiotemporal culture database, the spectral emission curves of each facet in the three-dimensional geometric model are analyzed to generate a parametric physical material library; the scene spatiotemporal culture database includes environmental effect physical description parameters; Obtain the spatiotemporal information of the user's selected experience, and generate basic light and shadow images based on the parameterized physical material library and the scene spatiotemporal culture database; Based on the environmental effects physical description parameters and the basic lighting and shadow images, a composite visual image result is generated; the composite visual image result includes environmental effects data. Based on the synthesized visual image results and the environmental effects data, an immersive experience instruction is generated; the immersive experience instruction is used to instruct a preset device to generate data that meets the user's spatiotemporal information needs to achieve an immersive experience; the preset device includes tactile devices and auditory devices; The step of obtaining the user-selected spatiotemporal information and generating a basic lighting image based on the parameterized physical material library and the scene spatiotemporal culture database includes: Obtain the spatiotemporal information of the user's selected experience; the spatiotemporal information includes time, weather, and geographical location; Based on the experience spatiotemporal information and the scene spatiotemporal culture database, the corresponding lighting parameters are determined; Based on the parametric physical material library and the lighting parameters, the lighting intensity of each of the facets is obtained using the following formula: ； in, Represents a piece of dough Light intensity, Indicates the light source The radiation intensity, Represents the bidirectional reflection distribution function. Indicates the direction of observation. Indicates the direction of incidence. Represents a piece of dough The normal vector, The solid angle representing the direction of incidence. Indicates the total number of light sources; Based on the light intensity and the spatial coordinate information of the three-dimensional geometric model, the three-dimensional geometric model is converted into a two-dimensional image to obtain a basic two-dimensional image. The basic two-dimensional image is texture mapped to obtain a two-dimensional texture image; The basic two-dimensional texture image is rendered with color to generate a basic light and shadow image.

2. The method according to claim 1, characterized in that, The step of meshing the original 3D point cloud data to obtain an initial 3D geometric model includes: The original three-dimensional point cloud data is denoised to obtain denoised point cloud data. Based on the denoised point cloud data, an initial triangular mesh model is generated; The initial triangular mesh model is subjected to topology repair processing to obtain the repaired triangular mesh model; The repaired triangular mesh model is then simplified to obtain a simplified triangular mesh model. The simplified triangular mesh model is smoothed to obtain the initial three-dimensional geometric model.

3. The method according to claim 1, characterized in that, The step of spatially registering the spectral reflectance curve with the initial three-dimensional geometric model to obtain the three-dimensional geometric model includes: Material boundaries are identified by performing material boundary identification on the initial three-dimensional geometric model to obtain the material boundaries; The spectral reflectance curve is spatially aligned with the material boundary to obtain an aligned three-dimensional geometric model. Based on a preset material property database, the spectral reflectance curve is mapped to each facet of the aligned 3D geometric model, and the material spectral properties of each facet are calculated to obtain the 3D geometric model; wherein, the material property database contains material spectral property parameters corresponding to different spectral reflectance ranges.

4. The method according to claim 3, characterized in that, The step of mapping the spectral reflectance curve to each facet of the aligned 3D geometric model according to a preset material property database, calculating the material spectral properties of each facet, and obtaining the 3D geometric model includes: Each facet of the aligned 3D geometric model is numbered to obtain a set of numbered faces; Based on the spectral reflectance curve, the spectral reflectance data corresponding to each numbered patch in the set of numbered patches is extracted; the spectral reflectance data includes the spectral reflectance of each patch; Based on the preset material property database, obtain the initial material parameter set for each match; Based on the initial material parameter set, the material spectral properties of each facet are calculated using the following formula: ； in, For each of the aforementioned facets, the material spectral properties, Indicates the first The weights of the initial material parameters; Indicates the first The first piece of the face Initial material parameters, Indicates the first Each patch at wavelength Spectral reflectance at that location Indicates the total number of initial material parameters; The material spectral properties of all facets are associated with the corresponding numbered facets to obtain the three-dimensional geometric model.

5. The method according to claim 1, characterized in that, The step involves analyzing the spectral emission curves of each facet in the 3D geometric model based on a preset scene spatiotemporal cultural database to generate a parametric physical material library, including: Based on the aforementioned scene spatiotemporal cultural database, the ambient lighting parameters are obtained; Based on the ambient lighting parameters, the emitted radiance of each of the facets is calculated using the following formula, resulting in the spectral emission curves of each facet: ； in, Indicates wavelength The emitted radiation brightness at that location, Represents the bidirectional reflection distribution function. Indicates the incident radiation brightness. Indicates the direction of incidence. Indicates the direction of launch. Represents the normal vector of a surface. This represents the solid angular domain composed of all incident directions; The spectral emission curve is converted into a parameterized material descriptor to generate the parameterized physical material library.

6. The method according to claim 1, characterized in that, The process of generating a synthesized visual image based on the environmental effects physical description parameters and the basic lighting and shadow image includes: Based on the physical description parameters of the environmental effects, environmental effects are simulated to obtain environmental effect data; The environmental effects data is fused with the basic lighting and shadow image to obtain the initial visual image result; The initial visual image result is color-corrected to obtain the corrected visual image result. The contrast of the corrected visual image is enhanced to obtain the synthesized visual image.

7. An immersive intangible cultural heritage experience device based on virtual reality, characterized in that, The device includes: The data acquisition module is used to acquire multi-dimensional data of the target intangible cultural heritage scene; the multi-dimensional data includes original three-dimensional point cloud data and spectral reflectance curves of different materials in the target intangible cultural heritage scene; The point cloud data processing module is used to perform meshing processing on the original three-dimensional point cloud data to obtain an initial three-dimensional geometric model. A geometric model generation module is used to spatially register the spectral reflectance curve with the initial three-dimensional geometric model to obtain a three-dimensional geometric model; the three-dimensional geometric model includes material spectral properties. The parametric material library generation module is used to analyze the spectral emission curves of each facet in the three-dimensional geometric model based on a preset scene spatiotemporal culture database to generate a parametric physical material library; the scene spatiotemporal culture database includes environmental effect physical description parameters; The light and shadow image generation module is used to acquire the spatiotemporal information of the experience selected by the user; the spatiotemporal information of the experience includes time, weather and geographical location; Based on the experience spatiotemporal information and the scene spatiotemporal culture database, the corresponding lighting parameters are determined; Based on the parametric physical material library and the lighting parameters, the lighting intensity of each of the facets is obtained using the following formula: ； in, Represents a piece of dough Light intensity, Indicates the light source The radiation intensity, Represents the bidirectional reflection distribution function. Indicates the direction of observation. Indicates the direction of incidence. Represents a piece of dough The normal vector, The solid angle representing the direction of incidence. Indicates the total number of light sources; Based on the light intensity and the spatial coordinate information of the three-dimensional geometric model, the three-dimensional geometric model is converted into a two-dimensional image to obtain a basic two-dimensional image. The basic two-dimensional image is texture mapped to obtain a two-dimensional texture image; The basic two-dimensional texture image is rendered with color to generate a basic light and shadow image; A visual image compositing module is used to generate a composite visual image result based on the environmental effects physical description parameters and the basic lighting and shadow image; the composite visual image result includes environmental effects data; An immersive experience instruction generation module is used to generate immersive experience instructions based on the synthesized visual image results and the environmental effects data; the immersive experience instructions are used to instruct preset devices to generate data that meets the user's spatiotemporal information to achieve an immersive experience; the preset devices include tactile devices and auditory devices.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images