Immersive experience system and method for scenic spot

Abstract

The invention discloses an immersive experience method for a scenic spot. The method specifically comprises the following steps of: 1, generating a three-dimensional virtual environment diagram according to a tourist scenic spot design drawing; 2, generating a three-dimensional model for cultural relics and buildings in the scenic area through a high-precision scanning technology; 3, arranging a tourist detection module and a data acquisition module at each scenic spot position; step 4, the tourist carries out data and position data interaction with the tourist detection module when arriving at a specific scenic spot through a handheld device or a wearing device, and selects actual experience and a virtual model system to carry out tourism experience or alternate experience of the actual experience and the virtual model system through the handheld device or the wearing device; 5, a data acquisition module acquires image video data and voice data of tourists in the area, and the experience pleasure of the tourists is judged according to information of character expressions and voice keywords; and step 6, generating a recommended touring route by a route planning module according to the basic information of the user, including gender, age and pleasure of the user.

Description

Technical Field

[0001] The present invention belongs to the field of cultural tourism digital technology, and specifically relates to a scenic spot immersive experience system and method. Background Art

[0002] As a key sector for enhancing the national economy and its soft power, the cultural tourism industry is undergoing a transformation and upgrade from traditional to digital and then to intelligent. Smart tourism systems leverage new technologies such as mobile cloud computing and the internet, using portable internet-connected devices to proactively perceive tourism-related information and promptly arrange and adjust travel plans. These systems primarily provide navigation, tour guides, guides, and shopping guides.

[0003] With the rapid development of information technology, the application of the metaverse in digital and intelligent cultural tourism has become a hot topic of research (Feng Xuegang and Cheng Xin, 2022). The metaverse integrates cutting-edge technologies such as virtual reality (VR), artificial intelligence (AI), and blockchain, driving the transformation of traditional tourism towards digitalization and intelligentization (Zhang Ning et al., 2023). Virtual tourism on the cultural and tourism metaverse platform has long been proven to be effective for users and tourists (J Han and Y Ji, 2022). Furthermore, the application of metaverse interactive technology has a positive impact on the development and preservation of historical and cultural relics (Yu Jie et al.; Styliadis AD et al., 2023). Numerous museums and tourist attractions around the world have begun using AR technology to provide virtual guided tours. For example, the Palace Museum in Beijing uses VR to provide visitors with an immersive online experience (Li Meng, 2023). Disney offers virtual theme park experiences through the metaverse platform, attracting tourists from around the world (Bob Chapek, 2023). The metaverse can create a borderless space for cultural exchange, which is of great significance for promoting the protection and dissemination of global cultural heritage.

[0004] In current technology, the main research direction is to digitize tourism resources and display them using software or virtual systems. This cannot replace the experience that real tourism brings to users. Therefore, how to cross-integrate virtual and real tourism, and how to collect and evaluate user experience information, becomes particularly important. Summary of the Invention

[0005] In response to the shortcomings of existing technologies, this invention proposes a method for immersive scenic spot experience that combines on-site experience with virtual experience, can identify tourists' emotional information, and intelligently recommend tour routes and attractions. The specific technical solution is as follows:

[0006] A method for immersive experience in a scenic spot, specifically:

[0007] Step 1: Generate a 3D virtual environment map based on the design drawing of the cultural and tourism scenic spot;

[0008] Step 2: Generate a 3D model of the scenic area’s cultural relics and buildings using high-precision scanning technology;

[0009] Step 3: Set up a visitor detection module and a data collection module at each scenic spot;

[0010] Step 4: When arriving at a specific scenic spot, the tourist interacts with the tourist detection module through a handheld device or wearable device to exchange data and location data. Through the handheld device or wearable device, the tourist chooses to experience the tourism through the actual experience, the virtual model system, or alternately experience the two.

[0011] Step 5: The data collection module collects image, video and voice data of tourists in the area, and judges the tourists' experience pleasure through information such as facial expressions and voice keywords;

[0012] Step 6: Route planning module generates recommended tour routes based on user basic information, including gender, age, and user happiness.

[0013] As an optimization: Step 2 is to use 3DGIS and BIM technology to build a highly detailed and dynamically updated three-dimensional model of the scenic area, which includes static information such as topography, building layout, road network, etc., and integrates dynamic data such as real-time changing environmental parameters, visitor distribution, and equipment status. It uses advanced rendering technology and physical engine simulation to realistically reproduce the physical environment of the scenic area, support multi-scale observation and analysis, and support dynamic update and expansion of the model.

[0014] As an optimization: in step 4, the handheld device includes a smart phone, a tablet computer or a dedicated handheld device, and the wearable device is a VR device.

[0015] As an optimization: In step 5, the specific method of identifying the tourist's happiness through the facial expressions is:

[0016] 5.1. Establish a deep learning-based convolutional neural network for emotion estimation;

[0017] 5.2. Use the camera module to capture facial images and obtain emotional information, specifically

[0018] 5.2.1. Collect the coordinates of the upper left and lower right corners of the face area in the image;

[0019] 5.2.2. Calculate the length and width of the rectangle enclosed by the two corner points based on the position coordinates;

[0020] 5.2.3. Extract the face area based on the length and width of the rectangle;

[0021] 5.3. Compare the face area image with the facial emotion dataset to identify facial emotions.

[0022] As an optimization: in the step 5, the method for identifying the tourist experience pleasure by voice keywords is to collect the tourist voice, separate the tourist voice data, and identify the voice data of a single individual;

[0023] Segmenting individual voice data audio files into several voice segments;

[0024] Compare the speech clips with the speech emotion database to identify emotional information.

[0025] The beneficial effects of the present invention are: 1. By combining physical on-site tours and virtual tours, the advantages of each are brought into play, the tour needs of different groups of people are met, and the visitor experience is improved; 2. During the tour, the user's emotions are judged through image video data and audio data, and a recommended tour route is automatically generated based on the emotion and the specific attractions, thereby improving the user experience and realizing intelligent and precise management. DETAILED DESCRIPTION

[0026] The preferred embodiments of the present invention are described in detail below so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more precise definition of the protection scope of the present invention.

[0027] A method for immersive experience in a scenic spot, specifically:

[0028] Step 1: Generate a 3D virtual environment map based on the design drawing of the cultural and tourism scenic spot;

[0029] Through a variety of sensors and IoT devices, various data within the scenic area are collected in real time and accurately, including the real-time location of visitors, environmental parameters such as temperature and humidity, air quality, and light intensity, as well as equipment status such as parking space availability, public facility usage, and security surveillance video streams;

[0030] Step 2: Generate a 3D model of the scenic area’s cultural relics and buildings using high-precision scanning technology;

[0031] Specifically, 3DGIS and BIM technologies are used to build a highly detailed and dynamically updated three-dimensional model of the scenic area, which includes static information such as topography, building layout, road network, and integrates dynamic data such as real-time changing environmental parameters, visitor distribution, and equipment status. Advanced rendering technology and physical engine simulation are used to realistically reproduce the physical environment of the scenic area, support multi-scale observation and analysis, and support dynamic updating and expansion of the model.

[0032] Step 3: Set up a visitor detection module and a data collection module at each scenic spot;

[0033] Step 4: When arriving at a specific attraction, the tourist uses a handheld device or wearable device. The handheld device includes a smartphone, tablet computer or dedicated handheld device, and the wearable device is a VR device. The tourist exchanges data and location data with the tourist detection module and chooses to experience the tourism through the actual experience and the virtual model system, or alternately through the two experiences.

[0034] Step 5: The data collection module collects image, video and voice data of tourists in the area, and judges the tourists' experience pleasure through information such as facial expressions and voice keywords;

[0035] The specific method of identifying tourists' happiness through facial expressions is:

[0036] 5.1. Establish a deep learning-based convolutional neural network for emotion estimation;

[0037] 5.2. Use the camera module to capture facial images and obtain emotional information, specifically

[0038] 5.2.1. Collect the coordinates of the upper left and lower right corners of the face area in the image;

[0039] 5.2.2. Calculate the length and width of the rectangle enclosed by the two corner points based on the position coordinates;

[0040] 5.2.3. Extract the face area based on the length and width of the rectangle;

[0041] 5.3. Compare the face area image with the facial emotion dataset to identify facial emotions.

[0042] The method of identifying the tourist experience pleasure through voice keywords is to collect the tourist voice, separate the tourist voice data, and identify the voice data of individual individuals;

[0043] Segmenting individual voice data audio files into several voice segments;

[0044] Compare the speech clips with the speech emotion database to identify emotional information.

[0045] Step 6: Route planning module generates recommended tour routes based on user basic information, including gender, age, and user happiness.

Claims

1. A scenic spot immersive experience method, characterized in that: The specific method is: Step 1: Generate a 3D virtual environment map based on the design drawing of the cultural and tourism scenic spot; Step 2: Generate a 3D model of the scenic area’s cultural relics and buildings using high-precision scanning technology; Step 3: Set up a visitor detection module and a data collection module at each scenic spot; Step 4: When arriving at a specific attraction, the tourist interacts with the tourist detection module through a handheld device or wearable device to exchange data and location data. Through the handheld device or wearable device, the tourist chooses to experience the tourism through the actual experience, the virtual model system, or alternately through the two experiences. Step 5: The data collection module collects image, video and voice data of tourists in the area, and judges the tourists' experience pleasure through information such as facial expressions and voice keywords; Step 6: Route planning module generates recommended tour routes based on user basic information, including gender, age, and user happiness.

2. The scenic spot immersive experience method according to claim 1, characterized in that: The second step is to use 3DGIS and BIM technology to build a highly detailed and dynamically updated three-dimensional model of the scenic area, which includes static information such as topography, building layout, road network, etc., and integrates dynamic data such as real-time changing environmental parameters, visitor distribution, and equipment status. It uses advanced rendering technology and physical engine simulation to realistically reproduce the physical environment of the scenic area, support multi-scale observation and analysis, and support dynamic updating and expansion of the model.

3. The scenic spot immersive experience method according to claim 1, characterized in that: In step 4, the handheld device includes a smart phone, a tablet computer or a dedicated handheld device, and the wearable device is a VR device.

4. The scenic spot immersive experience method according to claim 1, characterized in that: In step 5, the specific method of identifying the tourist's happiness through the facial expressions is: 5.

1. Establish a deep learning-based convolutional neural network for emotion estimation; 5.

2. Use the camera module to capture facial images and obtain emotional information. Specifically, 5.2.

1. Capture the position coordinates of the upper left corner and lower right corner of the facial area in the image; 5.2.

2. Calculate the length and width of the rectangle enclosed by the two corner points based on the position coordinates; 5.2.

3. Extract the face area based on the length and width of the rectangle; 5.

3. Compare the face area image with the facial emotion dataset to identify facial emotions.

5. The scenic spot immersive experience method according to claim 1, characterized in that: In the step 5, the method for identifying the tourist experience pleasure by voice keywords is to collect the tourist voice, separate the tourist voice data, and identify the voice data of a single individual; Segmenting individual voice data audio files into several voice segments; Compare the speech clips with the speech emotion database to identify emotional information.