Scenic spot navigation system and method based on AR technology and live-action roaming

Abstract

The invention provides a scenic spot guide system and method based on AR technology and live-action roaming, the guide system comprises a navigation module, a strategy module and a footprint module, the three modules are in network connection, the navigation module is used for intelligent route planning and AR virtual tour guide, the strategy module is used for rapidly obtaining scenic spot information and personalized route recommendation, and the footprint module is used for providing a guide system for the scenic spot. The footprint module is used for forming a personal experience path. Through flexible combination and application of the AR technology, the big language model technology and the big data technology, the tour efficiency of the user is improved, the tour experience of the user is reserved, the interactivity, knowledge and interestingness in the experience of the user in the tour process are enriched, the individual requirement of each user is met, the in-depth experience of the user on Chinese culture is achieved, and the user experience is improved. And inheritance and development of culture are promoted.

Description

Technical Field

[0001] The present invention relates to the fields of AR technology and big data technology, and in particular to a scenic spot guide system and method based on AR technology and real-scene roaming. Background Art

[0002] The realization of AR virtual tour guide technology is based on the rapidly developing AR technology and big data technology. It interacts with users through voice and pictures, which can enrich the knowledge and fun of the tour experience.

[0003] However, the tour guide systems currently in use have some problems: the voice introduction mode is single and fixed, lacking interaction with users and failing to stimulate users' interest in visiting and understanding; the navigation system functions are monotonous and unable to reflect the characteristics of each attraction and attract users to visit. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a scenic spot guidance system and method based on AR technology and real-scene roaming, which are specifically introduced as follows:

[0005] The present invention firstly proposes a scenic spot guide system based on AR technology and real-life roaming, including: a navigation module, a strategy module and a footprint module, the three modules are network-connected, the navigation module is used for intelligent route planning and AR virtual tour guide, the strategy module is used for quickly obtaining scenic spot information and personalized route recommendations, and the footprint module is used for forming a personal experience path.

[0006] The guide module includes a text introduction module and a picture overview module. The text introduction module and the picture overview module are connected to the network. The text introduction module is used to obtain the historical and cultural information behind the scenic spots. The picture overview module can generate a custom map in combination with the user's location information and can recommend user routes.

[0007] The navigation module includes a map navigation module and an AR guidance module. The map navigation module and the AR guidance module are connected to a network. The map navigation module is based on a custom map, user location and recommended routes for scenic spots. The AR guidance module uses a digital human to achieve real-time user interaction and personalized answering of questions.

[0008] The footprint module includes a cultural story recording module, a UI animation module, and an audio explanation module. The three modules are connected to a network. The cultural story recording module is used to record the user's tour footprints, the UI animation module is used to display the user's tour progress, and the audio explanation module is used to provide voice explanations.

[0009] The process of customizing map drawing in the map navigation module is as follows:

[0010] Convert the actual latitude and longitude coordinates of the user into screen coordinates on the map to locate the user's position, obtain the latitude and longitude coordinates corresponding to each scenic spot, and draw a recommended route based on the latitude and longitude coordinates of the user and the scenic spots.

[0011] The AR guidance module includes an AR virtual tour guide module and an AR photo-taking module. The AR virtual tour guide module combines the AR scene with a digital human. The digital human provides virtual tour guide services for users, conducts real-time interactions, and can also use big data for personalized problem-solving. The AR photo-taking module provides AR photo-taking services, and the digital human can choose different models and costumes.

[0012] The AR photo-taking module retrieves the virtual image of the digital human and realizes AR photo-taking by using the function of the camera of the usage terminal through AR and taking a screenshot of the current screen.

[0013] The AR virtual tour guide module realizes the movement of the AR digital human in the AR scene and the conversion with static animations and conversation animations through the AR network. At the same time, the audio data of the user is converted into text by remote transmission, and the text is sent to the back end of the big data model. The parsed text data is synthesized into voice data for return, and finally played through the digital human.

[0014] Based on the above system, the present invention also proposes a corresponding scenic spot tour guide method, including the following steps:

[0015] Before or during the tour, use the strategy module to display the introductions of the main scenic spots. The user selects a scenic spot according to the pictures and the introduction text of the scenic spot is displayed; the user clicks on the picture to jump to the navigation page for picture navigation and a recommended route for the user is given;

[0016] During the tour, according to the recommended route, the AR guidance module can be entered for real-time interactive guidance by the AR virtual tour guide, and a group photo can be taken with the digital human during AR photo-taking;

[0017] After the tour, the footprint module records the tour experiences of the user at each scenic spot to form a unique personal experience path.

[0018] Further, the process of using the AR guidance module for AR guidance includes:

[0019] Step 1: The user clicks the "AR Navigation" button to enter the AR guidance interface;

[0020] Step 2: The user uses the camera for AR guidance. Clicking on the ground will cause an AR character to appear on the ground;

[0021] Step 3: Communicate with the character through the microphone in the interface, and the character gives corresponding responses;

[0022] Step 4: Click the camera button on the right side of the microphone to take a group photo with the AR character in the scenic area.

[0023] Beneficial effects:

[0024] By flexibly combining and applying AR technology, large language model technology, and big data technology, the present invention improves the user's tour efficiency, retains the user's travel experience, enriches the interactivity, knowledge, and fun in the user's tour process experience, meets the personalized needs of each user, realizes the user's in-depth experience of Chinese culture, and promotes the inheritance and development of culture. Description of the drawings

[0025] Figure 1 It is a schematic structural diagram of the tour guide system of the present invention;

[0026] Figure 2 It is a schematic diagram of the custom map drawing process of the present invention;

[0027] Figure 3 It is a schematic structural diagram of the AR guidance module of the present invention;

[0028] Figure 4 It is the custom map of the implementation case of the present invention;

[0029] Figure 5 It is the digital human virtual image diagram of the implementation case of the present invention. Specific implementation manners

[0030] The following will describe in detail the specific implementation manners of the present invention with reference to the accompanying drawings of the specification.

[0031] The present invention aims at the Huangdi Mausoleum Scenic Area and designs the "Intelligent Tour on the Mausoleum" tour guide system.

[0032] As Figure 1 shown, the system includes: a navigation module, a strategy module, and a footprint module. The three modules are connected to the network. The navigation module is used for intelligent route planning and AR virtual tour guide. The strategy module is used for quickly obtaining scenic spot information and personalized route recommendations. The footprint module is used for forming a personal experience path.

[0033] The strategy module includes a text introduction module and a picture overview module. The text introduction module and the picture overview module are connected to the network. The text introduction module is used for obtaining the historical and cultural information behind the scenic spots. The picture overview module can generate a custom map in combination with the user's location information and can recommend user routes.

[0034] The navigation module includes a map navigation module and an AR guidance module. The map navigation module and the AR guidance module are connected to the network. The map navigation module is based on the custom map, user location, and scenic spot recommended routes. The AR guidance module uses digital humans to achieve real-time interaction and personalized doubt-solving for users.

[0035] The schematic diagram of the custom map drawing process of the map navigation module is as Figure 2 shown below:

[0036] Convert the actual longitude and latitude coordinates of the user into screen coordinates on the map to locate the user's position, obtain the longitude and latitude coordinates corresponding to each scenic spot, and draw a recommended route based on the longitude and latitude coordinates of the user and the scenic spots. The custom map in this example is as Figure 4 shown.

[0037] The actual longitude and latitude coordinates of the user and specific scenic spots are obtained, and the positioning function of the Amap SDK is used when drawing the recommended route.

[0038] The schematic diagram of the AR guidance module structure is as Figure 3 shown. The AR guidance module includes an AR virtual tour guide module and an AR photo-taking module. The AR virtual tour guide module realizes the combination of the AR scene and the digital human. The digital human provides virtual tour guide services for users and conducts real-time interactions, and can also use big data for personalized problem-solving. The AR photo-taking module provides AR photo-taking services, and the digital human can choose different models and costumes. The virtual image of the digital human is as Figure 5 shown.

[0039] The AR photo-taking module includes a Texture2D texture rendering module and an album access plugin saving module. The working process of this AR photo-taking module is as follows:

[0040] Step 1: Convert the rendering texture of the main camera into a Texture2D object;

[0041] Step 2: Save the photo content and use the album access plugin, specifically the nativegallery plugin.

[0042] The function of the mobile phone camera is called through AR, and the AR photo-taking is realized by intercepting the current screen.

[0043] Specifically, the AR virtual tour guide module includes the Lightship ARDK kit, the large model interface, Microsoft Azure, SkinMesh and Avatar. The AR virtual tour guide process includes: Step 1: Introduce the AR function in Unity using the Lightship ARDK kit; Step 2: Connect to the DeepSeek interface; Step 3: Connect to the speech recognition and speech synthesis of Microsoft Azure.

[0044] With the above technical solution, a dynamic AR mesh is generated through the ARMeshManager script, the movement of the AR character in the AR scene is realized through the ARMeshAgent script, and the conversion of the character's static animation, movement animation, and conversation animation is realized by writing an animation state machine and a character movement state machine. The user's audio data is converted into text by remote transmission, and the text is sent to the DeepSeek backend to parse the user's audio data, and the returned voice data is obtained through Azure speech synthesis, processed as an audioclip in Unity, and played through the digital human.

[0045] The SkinMesh and Avatar modules are used to realize the dressing and animation processing of the model.

[0046] With the above technical solution, through the avatar tools plugin, the dressing of the digital human is processed, and different clothing models are attached to a set of body models.

[0047] The AR group photo module retrieves the virtual image of the digital human and realizes the AR group photo by using the function of the terminal camera through AR and taking a screenshot of the current screen. By taking a group photo with the virtual tour guide of the digital human, the user's travel memories are retained, making each tour a precious personal experience.

[0048] The AR virtual tour guide module realizes the movement of the AR digital human in the AR scene and the conversion with the static animation and conversation animation through the AR network. At the same time, the user's audio data is converted into text by remote transmission, and the text is sent to the big data model backend to parse the obtained text data and obtain the returned voice data through speech synthesis, and finally played through the digital human. The AR virtual tour guide module can provide personalized answers and suggestions by implanting DeepSeek or other AI technologies.

[0049] Through the above technical solution, the combination of the AR scene and the digital human is realized, allowing users to interact with the virtual tour guide in real time.

[0050] The footprint module includes a cultural story recording module, a UI animation module, and an audio explanation module. The three modules are network-connected. The cultural story recording module is used to record the user's tour footprint, the UI animation module is used to display the user's tour progress, and the audio explanation module is used to provide voice commentary.

[0051] Based on the above system, the present invention also proposes a corresponding scenic spot tour guide method, including the following steps:

[0052] Before or during the tour, use the strategy module to display the introduction of the main scenic spots. The user selects the scenic spots according to the pictures and the introduction text of the scenic spots is displayed; when the user clicks on the picture, it jumps to the navigation page for picture navigation and a recommended route is given to the user.

[0053] During the tour, according to the recommended route, the user can enter the AR guidance module to conduct real-time interactive guidance of the AR virtual tour guide. When taking an AR group photo, the user can take a photo with the digital human;

[0054] After the tour, the footprint module records the user's tour experience at each scenic spot, forming a unique personal experience path.

[0055] Among them, the process of using the AR guidance module for AR guidance includes:

[0056] Step 1: The user clicks the "AR Navigation" button to enter the AR guidance interface;

[0057] Step 2: The user uses the camera for AR guidance. Clicking on the ground will cause an AR character to appear on the ground;

[0058] Step 3: Communicate with the character through the microphone in the interface, and the character gives corresponding responses;

[0059] Step 4: Click the camera button on the right side of the microphone to take a group photo with the AR character in the scenic area.

Claims

1. A scenic spot guide system based on AR technology and real-life roaming, comprising: The navigation module, strategy module and footprint module are connected by a network. The navigation module is used for intelligent route planning and AR virtual tour guide, the strategy module is used to quickly obtain scenic spot information and personalized route recommendations, and the footprint module is used to form a personal experience path.

2. The scenic spot guide system based on AR technology and real-life roaming according to claim 1, characterized in that: The guide module includes a text introduction module and a picture overview module. The text introduction module and the picture overview module are connected to the network. The text introduction module is used to obtain the historical and cultural information behind the scenic spots. The picture overview module can generate a custom map in combination with the user's location information and can recommend user routes.

3. The scenic spot guide system based on AR technology and real-life roaming according to claim 1, characterized in that: The navigation module includes a map navigation module and an AR guidance module. The map navigation module and the AR guidance module are connected to a network. The map navigation module is based on a custom map, user location and recommended routes for scenic spots. The AR guidance module uses a digital human to achieve real-time user interaction and personalized answering of questions.

4. The scenic spot guide system based on AR technology and real-life roaming according to claim 1, characterized in that: The footprint module includes a cultural story recording module, a UI animation module, and an audio explanation module. The three modules are connected to a network. The cultural story recording module is used to record the user's tour footprints, the UI animation module is used to display the user's tour progress, and the audio explanation module is used to provide voice explanations.

5. The scenic spot guide system based on AR technology and real-life roaming according to claim 3, characterized in that: The process of customizing map drawing in the map navigation module is as follows: The user's actual longitude and latitude coordinates are converted into screen coordinates on the map to locate the user, obtain the longitude and latitude coordinates corresponding to each scenic spot, and draw a recommended route based on the longitude and latitude coordinates of the user and the scenic spot.

6. The scenic spot guide system based on AR technology and real-life roaming according to claim 3, characterized in that: The AR guidance module includes an AR virtual tour guide module and an AR group photo module. The AR virtual tour guide module realizes the combination of AR scenes and digital humans. The digital humans provide virtual tour guide services to users and interact in real time. They can also use big data for personalized answering of questions. The AR group photo module provides AR group photo services, and digital humans can choose different models and costumes.

7. The scenic spot guide system based on AR technology and real-life roaming according to claim 6, characterized in that: The AR group photo module retrieves the virtual image of the digital human, and uses the function of the terminal camera through AR call to realize AR group photo by capturing the current screen.

8. The scenic spot guide system based on AR technology and real-life roaming according to claim 6, characterized in that: The AR virtual tour guide module realizes the movement of the AR digital human in the AR scene and the conversion between static animation and conversation animation through the AR network. At the same time, it converts the user's audio data into text by remote transmission, and sends the text to the big data model backend, parses the obtained text data and obtains the returned voice data through speech synthesis, which is finally played through the digital human.

9. A method for guiding scenic spots based on the AR technology and real-life roaming of the system of claims 1 to 8, comprising the following steps: Before or during a tour, use the guide module to display introductions to major attractions. Users can select attractions based on pictures and see the introduction text. During the tour, users select specific attractions as destinations, and a custom map is generated based on the user's specific location to give personalized recommended routes; Users click on the image to jump to the navigation page and navigate on the map. During navigation, they can switch to the AR guidance module to receive real-time interactive guidance from the AR virtual tour guide, and can take photos with the digital human when taking AR photos. After the tour, the footprint module records the user's visit experience at each attraction, forming a unique personal experience path.

10. The navigation method according to claim 9, characterized in that: The process of using the AR guidance module for AR guidance includes: Step 1: The user clicks the "AR Navigation" button to enter the AR guidance interface; Step 2: The user uses the camera for AR guidance. When the user clicks the ground, an AR character will appear on the ground. Step 3: Communicate with the character through the microphone in the interface, and the character will give a corresponding reply; Step 4: Click the camera button on the right side of the microphone to take a photo with the AR character in the scenic area.

Images