Intelligent tour guide robot and AR scenic spot explanation system

Abstract

The invention relates to the technical field of scenic spot explanation systems, in particular to an intelligent tour guide robot and AR scenic spot explanation system, and the system comprises a cloud server side which comprises a scenic spot 3D model database, an augmented reality resource package database, a deep learning semantic understanding service module and a multi-language text translation service module; the intelligent tour guide terminal comprises an intelligent tour guide robot, and the robot is provided with a walking mechanism, a voice recognition module, a voice synthesis module, a natural language processing module and a human-computer interaction interface; the AR glasses terminal comprises AR glasses, and the glasses are provided with a camera, a display screen, a processor, a memory and a communication module; through the combination of deep learning semantic understanding and neural machine translation, the accuracy of cross-language communication is improved, and the ability of the system to understand the cultural background is enhanced. The AR technology and the knowledge graph are fused, so that unprecedented knowledge visualization experience is created, and abstract historical and cultural knowledge becomes vivid and perceptible.

Description

Technical Field

[0001] The present invention relates to the technical field of scenic spot interpretation systems, in particular to an intelligent tour guide robot and an AR scenic spot interpretation system. Background Art

[0002] With the booming tourism industry and the rapid advancement of information technology, traditional tourist attraction guides have been unable to meet the growing personalized needs of modern tourists. In recent years, intelligent tour guide systems have gradually become the focus of industry attention, but the solutions currently on the market still have many shortcomings.

[0003] The closest existing technologies usually use simple voice guides or mobile applications. Although they have improved the tourist experience to a certain extent, they still have obvious limitations. First, these systems often lack intelligent interaction capabilities and cannot accurately understand and respond to tourists' complex questions. Secondly, multilingual support usually relies on pre-recorded content or simple machine translation, making it difficult to provide smooth and natural multilingual services. Furthermore, existing AR technology applications are often limited to simple image overlays and lack the ability to deeply reconstruct historical scenes of scenic spots. In addition, these systems generally lack personalized recommendation functions and cannot dynamically adjust the guided content according to tourists' interests and behaviors.

[0004] More importantly, existing technologies have obvious deficiencies in integrating multiple advanced technologies. Functions such as voice recognition, natural language processing, and AR presentation are often fragmented, making it difficult to achieve seamless integration and collaborative work. This leads to slow system response, fragmented user experience, and difficulty in providing tourists with an immersive guided tour experience. At the same time, due to the lack of strong knowledge graph support and dynamic content generation capabilities, existing systems are unable to meet tourists' needs for in-depth cultural experiences. Summary of the invention

[0005] In view of the above problems, the present invention proposes an intelligent tour guide robot and AR scenic spot interpretation system, aiming to provide a comprehensive, intelligent and personalized scenic spot guidance solution by integrating multiple advanced technologies. The core goal of the present invention is to solve the problems existing in the prior art, such as insufficient interactive intelligence, limited multi-language support, shallow AR application, lack of personalized services and insufficient technology integration.

[0006] The present invention proposes an intelligent tour guide robot and an AR scenic spot interpretation system, including:

[0007] The cloud server side includes a 3D model database of scenic spots, an augmented reality resource package database, a deep learning semantic understanding service module, and a multilingual text translation service module;

[0008] An intelligent tour guide terminal, which includes an intelligent tour guide robot having a walking mechanism, a speech recognition module, a speech synthesis module, a natural language processing module and a human-computer interaction interface;

[0009] An AR glasses terminal, comprising an AR glasses having a camera, a display screen, a processor, a memory and a communication module;

[0010] The deep learning semantic understanding service module is used to convert the interactive voice request data of tourists into text form, and translate the text into a specific language, and then input the text translated into the specific language into the deep learning algorithm to identify the text type, thereby converting the voice into a scenic spot explanation text or operation instruction;

[0011] The multilingual text translation service module is used to convert speech into a specific language, identify the text type, and perform machine translation;

[0012] The intelligent tour guide robot and the AR glasses interact with each other through the cloud server to realize scenic spot guidance and AR content display;

[0013] When tourists give voice commands to the intelligent tour guide robot, the intelligent tour guide robot encodes and encrypts the voice data and transmits it to the cloud server. The cloud server decrypts and decodes the received data, and then processes it through the deep learning semantic understanding service module to identify the command type and call the corresponding functional module based on the recognition result.

[0014] Preferably, the deep learning semantic understanding service module adopts a bidirectional long short-term memory network Bi-LSTM model based on an attention mechanism, and the processing flow of the model includes:

[0015] First, the input text is converted into word vectors through the embedding layer;

[0016] Next, a bidirectional LSTM layer is used to capture contextual information;

[0017] Then, an attention layer is used to weight important information;

[0018] Finally, the final semantic understanding result is output through the fully connected layer;

[0019] The softmax function is used to calculate the weights of the attention layer, ensuring that the sum of all weights is 1, thereby highlighting the importance of key information.

[0020] Preferably, the multilingual text translation service module adopts a neural machine translation model based on the Transformer architecture, and the processing flow of the model includes:

[0021] First, the encoder encodes the source language text into a vector representation;

[0022] Then, the decoder generates the target language text based on the encoder’s output;

[0023] In both encoding and decoding processes, a multi-head attention mechanism is used to improve translation quality;

[0024] Among them, the multi-head attention mechanism captures language features at different levels by calculating multiple attention functions in parallel, thereby improving the accuracy and fluency of translation.

[0025] Preferably, a scenic spot data reconstruction module based on the HDF5 deep learning framework is also included, and the workflow of the module includes:

[0026] First, the data preprocessing submodule cleans, standardizes, and enhances the historical data of scenic spots;

[0027] Then, the model training submodule uses a method combining convolutional neural network (CNN) and generative adversarial network (GAN) to train the scenic spot reconstruction model;

[0028] Next, the trained model parameters are compressed and optimized to adapt to the computing power of the Android terminal;

[0029] Finally, the inference submodule runs the optimized model on the Android terminal to achieve real-time historical reconstruction of scenic spots;

[0030] Among them, the generator and discriminator of the GAN network are trained alternately to continuously improve the authenticity and detail richness of the reconstruction effect.

[0031] Preferably, the AR glasses use Vuforia technology to achieve AR experience, and its workflow includes:

[0032] First, the image recognition module uses a feature point matching algorithm to identify target objects in the real world;

[0033] Then, the 3D object tracking module uses SLAM technology to achieve precise positioning of AR content;

[0034] Next, the AR content rendering module uses OpenGL ES technology to merge virtual content with real scenes in real time;

[0035] Finally, the fused image is presented through the display screen of AR glasses;

[0036] Among them, SLAM technology achieves stable overlay and accurate positioning of AR content by simultaneously performing camera positioning and environmental map construction.

[0037] Preferably, it also includes an intelligent recommendation system based on deep reinforcement learning, the workflow of which includes:

[0038] First, collect and analyze users’ historical behavior data;

[0039] Then, adjust the recommendation strategy based on real-time feedback;

[0040] Next, the deep Q learning network DQN is used to select the best recommended action;

[0041] Finally, provide users with personalized attraction recommendations and guided tours;

[0042] Among them, the DQN network stabilizes the training process through the experience replay mechanism and the target network, thereby improving the performance and stability of the recommendation system.

[0043] Preferably, a scenic spot information retrieval system based on knowledge graph is also included, and the workflow of the system includes:

[0044] First, build a knowledge graph containing entities such as scenic spots, historical events, and people;

[0045] Then, the knowledge graph is converted into a low-dimensional vector representation using graph embedding technology;

[0046] Next, according to the user query, vector similarity calculation is used to quickly retrieve relevant information;

[0047] Finally, the retrieval results are expanded through relational reasoning to provide in-depth background knowledge of scenic spots;

[0048] Among them, the construction of the knowledge graph adopts a remote supervision method to automatically extract entity relationships from large-scale text corpora, and ensure the accuracy of knowledge through manual review.

[0049] Preferably, a dynamic explanation generation module based on natural language generation (NLG) technology is also included, and the workflow of the module includes:

[0050] First, select appropriate content from the knowledge graph based on the user’s interests and knowledge level;

[0051] Then, a Seq2Seq model based on the attention mechanism is used to generate preliminary explanation text;

[0052] Next, a rule-based post-processing method is applied to ensure the coherence and correctness of the generated content;

[0053] Finally, the generated commentary content is transmitted to the intelligent tour guide robot or AR glasses for display;

[0054] Among them, the encoder and decoder of the Seq2Seq model both adopt LSTM networks, and use the beam search algorithm to generate diverse commentary content.

[0055] Preferably, a real-time data processing module based on edge computing is also included, which is deployed on the intelligent tour guide robot and AR glasses, and its workflow includes:

[0056] First, data preprocessing and feature extraction are performed on the local device;

[0057] Then, a lightweight neural network model is used for preliminary data analysis and decision making;

[0058] Next, the processing results are synchronized and verified with the cloud server;

[0059] Finally, according to the local processing results and cloud feedback, corresponding operations are performed or content is displayed;

[0060] Among them, the lightweight neural network model adopts model compression and knowledge distillation technology to reduce computational complexity while ensuring performance and adapt to the resource limitations of edge devices.

[0061] Preferably, it also includes a user emotion recognition module based on multimodal fusion, and the workflow of the module includes:

[0062] First, the user's voice, facial expression and physiological signal data are collected simultaneously; then, the respective feature extraction networks are used to process the data of different modalities; then, the features of different modalities are weightedly fused through the attention mechanism; after that, the fused features are classified using the multi-layer perceptron (MLP) to derive the user's emotional state; finally, the content and rhythm of the tour are dynamically adjusted according to the recognition results; among them, multimodal fusion adopts a late strategy to integrate the prediction results of different modalities at the decision-making level to improve the accuracy and robustness of emotion recognition.

[0063] The beneficial effects of the present invention are mainly reflected in the following aspects:

[0064] Through the organic combination of core technologies such as deep learning semantic understanding, neural machine translation, AR scene reconstruction, and knowledge graph construction, the present invention has achieved significant breakthroughs in many aspects. First, the Bi-LSTM model based on the attention mechanism has greatly improved the system's semantic understanding ability, enabling it to accurately grasp the true intentions of tourists. Secondly, the neural machine translation model using the Transformer architecture, combined with the multi-head attention mechanism, achieves high-quality real-time multilingual translation, breaking the barriers to language communication.

[0065] In terms of AR technology application, the present invention combines Vuforia technology and SLAM algorithm to not only improve the stability and realism of AR content, but also achieve in-depth reconstruction of historical scenes. This innovative application greatly enhances the tourists' sense of immersion and cultural experience. In addition, the introduction of the information retrieval system based on the knowledge graph and the dynamic interpretation generation module enables the system to provide in-depth and personalized cultural content to meet the diverse needs of different tourists.

[0066] It is worth mentioning that the present invention has made a major breakthrough in technology integration and optimization. Through the application of edge computing technology and deep reinforcement learning, the system achieves rapid response and intelligent recommendation, greatly improving the user experience. The introduction of multimodal fusion emotion recognition technology enables the system to adjust the navigation strategy in real time and provide a truly user-centric service.

[0067] These technological innovations not only perform well in their respective fields, but more importantly, they achieve synergy in this system. For example, the combination of deep learning semantic understanding and neural machine translation not only improves the accuracy of cross-language communication, but also enhances the system's ability to understand cultural background. The integration of AR technology and knowledge graphs creates an unprecedented "knowledge visualization" experience, making abstract historical and cultural knowledge vivid and tangible.

[0068] In general, this invention has successfully solved many challenges faced by existing intelligent guide systems by innovatively integrating and optimizing multiple advanced technologies. It not only greatly improves the intelligence and personalization level of guide services, but also opens up new ways for the display and dissemination of cultural heritage. The application of this system is expected to completely change the way tourists experience scenic spots, provide a strong impetus for the intelligent upgrading of the tourism industry, and also bring new possibilities to the fields of cultural communication and education. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is the overall logic block diagram of the system of the present invention.

[0070] Figure 2 The figure is a logic block diagram of the intelligent tour guide terminal of the present invention.

[0071] Figure 3 This is a logic block diagram of the AR glasses terminal of the present invention.

[0072] Figure 4 It is a logic block diagram of the scenic spot data reconstruction module of the present invention.

[0073] Figure 5 The following is a logic block diagram of the AR experience module implemented by the Vuforia technology of the present invention. DETAILED DESCRIPTION

[0074] In order to further explain the technical means and effects of the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments thereof. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0075] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0076] Example 1

[0077] See also Figure 1-5 First, the intelligent tour guide robot and AR scenic spot interpretation system of the present invention includes three main components: a cloud server 1, an intelligent tour guide terminal 2 and an AR glasses terminal 3. The cloud server 1 is the core of the system, responsible for storing and processing a large amount of data, including a scenic spot 3D model database 11, an augmented reality resource package database 12, a deep learning semantic understanding service module 13 and a multilingual text translation service module 14. The intelligent tour guide terminal 2 includes an intelligent tour guide robot, which has a walking mechanism 211, a speech recognition module 212, a speech synthesis module 213, a natural language processing module 214 and a human-computer interaction interface 215. The AR glasses terminal 3 includes AR glasses 31, which are equipped with a camera 311, a display screen 312, a processor 313, a memory 314 and a communication module 315.

[0078] In actual applications, when tourists come to a scenic spot, they can choose to use an intelligent tour guide robot or wear AR glasses31 for a tour. For example, a tourist from France comes to China's West Lake Scenic Area. He can say to the intelligent tour guide robot in French: "Please tell me about the story of Broken Bridge and Remaining Snow." The intelligent tour guide robot will encode and encrypt this voice command and then transmit it to the cloud server1.

[0079] Next, after receiving the encrypted data, the cloud server 1 first decrypts and decodes it. Then, the deep learning semantic understanding service module 13 processes the text. This module uses a bidirectional long short-term memory network (Bi-LSTM) model based on the attention mechanism, and its processing flow includes four steps: first, the French text is converted into a word vector through the embedding layer; then, the bidirectional LSTM layer is used to capture contextual information; then, the attention layer is used to weight important information; finally, the final semantic understanding result is output through the fully connected layer. In this example, the system recognizes that the tourist wants to know about "Broken Bridge and Remaining Snow".

[0080] Preferably, the weight calculation of the attention layer adopts the softmax function to ensure that the sum of all weights is 1. The advantage of this is that the importance of key information can be highlighted. For example, when processing the request "Story of Broken Bridge and Residual Snow", the system will pay more attention to the keywords "Broken Bridge", "Residual Snow" and "Story", thereby improving the accuracy of semantic understanding.

[0081] Secondly, the system calls the multilingual text translation service module 14 to translate the French text into Chinese. This module uses a neural machine translation model based on the Transformer architecture, and its processing flow includes: first, the encoder encodes the French text into a vector representation; then, the decoder generates Chinese text based on the output of the encoder. In both the encoding and decoding processes, a multi-head attention mechanism is used to improve the translation quality. For example, the system may accurately translate the French request "Racontez-moi l'histoire du pont brisésous la neige" into "Please tell the story of the broken bridge and the lingering snow."

[0082] In one embodiment of the present invention, the multi-head attention mechanism captures language features at different levels by computing multiple attention functions in parallel. In practical applications, this may mean that the system can simultaneously focus on multiple aspects such as word meaning, grammatical structure, and cultural background, thereby improving the accuracy and fluency of translation. For example, when translating the phrase "broken bridge and residual snow", the system can not only understand the literal meaning, but also capture its cultural connotation as one of the ten scenic spots of West Lake.

[0083] Thirdly, the system also includes a scenic spot data reconstruction module 4 based on the HDF5 deep learning framework. The workflow of this module is firstly that the data preprocessing submodule 41 cleans, standardizes and enhances the historical data of the scenic spot. For example, for the historical image data of the broken bridge, the system will remove noise, adjust the resolution, and may improve the clarity of details through image enhancement technology. Then, the model training submodule 42 uses a method combining convolutional neural network (CNN) and generative adversarial network (GAN) to train the scenic spot reconstruction model. Then, the trained model parameters are compressed and optimized to adapt to the computing power of the Android terminal; finally, the reasoning submodule 43 runs the optimized model on the Android terminal to realize real-time scenic spot historical reconstruction; wherein, the generator and discriminator of the GAN network are alternately trained to continuously improve the authenticity and detail richness of the reconstruction effect. In this process, CNN may be used to extract the features of the broken bridge, while GAN is used to generate realistic images of the broken bridge in different periods.

[0084] Preferably, the generator and discriminator of the GAN network are trained alternately to continuously improve the authenticity and detail richness of the reconstruction effect. For example, the generator may gradually learn how to accurately present the landscape changes of the Broken Bridge in different seasons, while the discriminator will become more and more adept at distinguishing between real photos of the Broken Bridge and the generated images. The advantage of this method is that it can generate highly realistic historical scene reconstructions, greatly enhancing the immersion of tourists.

[0085] Next, the AR glasses 31 use Vuforia technology to realize the AR experience. Its workflow includes: First, the image recognition module uses a feature point matching algorithm to identify target objects in the real world. For example, when a tourist looks at the Broken Bridge, the system will recognize the outline and features of the bridge. Then, the 3D object tracking module uses SLAM technology to achieve precise positioning of AR content. This means that no matter how the tourist moves, the AR content can be accurately superimposed on the real scene. Next, the AR content rendering module uses OpenGL ES technology to merge virtual content with the real scene in real time.

[0086] In another embodiment of the present invention, SLAM technology achieves stable overlay and accurate positioning of AR content by simultaneously performing camera positioning and environmental map construction. This enables the system to continuously and stably display AR content related to the Broken Bridge, such as historical figures, text descriptions or animation effects, while tourists are strolling around West Lake.

[0087] In addition, the system also includes an intelligent recommendation system based on deep reinforcement learning. The workflow of the system first collects and analyzes the user's historical behavior data. For example, the system records the tourists' tour routes, stay time, and attractions of interest in the West Lake Scenic Area. Then, the recommendation strategy is adjusted based on real-time feedback. If it is found that tourists are particularly interested in literary allusions, the system may add recommended attractions related to legends such as the Legend of the White Snake.

[0088] Preferably, the system uses a deep Q-learning network (DQN) to select the best recommended action. The DQN network stabilizes the training process through the experience replay mechanism and the target network, improving the performance and stability of the recommendation system. For example, the system may learn to recommend tourists to visit the cool Yuewang Temple first on a hot summer day, and then recommend visiting the Leifeng Pagoda under the scorching sun to optimize the tourist experience.

[0089] Next, the present invention also includes a scenic spot information retrieval system based on the knowledge graph. The system first constructs a knowledge graph containing entities such as scenic spots, historical events, and characters. For example, the relationship between entities such as "Broken Bridge", "White Snake", and "Xu Xian" will be clearly represented. Then, the system uses graph embedding technology to convert the knowledge graph into a low-dimensional vector representation. The advantage of this is that the similarity between entities can be quickly calculated, improving retrieval efficiency.

[0090] In a preferred embodiment of the present invention, the knowledge graph is constructed using a remote supervision method to automatically extract entity relationships from a large-scale text corpus and ensure the accuracy of the knowledge through manual review. This method can efficiently build a large and accurate knowledge base and provide tourists with rich and reliable background knowledge.

[0091] The system also includes a dynamic explanation generation module based on natural language generation (NLG) technology. This module first selects appropriate content from the knowledge graph based on the user's interests and knowledge level. For example, for foreign tourists interested in Chinese literature, the system may focus on introducing literary works and allusions related to the Broken Bridge. Then, a sequence-to-sequence (Seq2Seq) model based on the attention mechanism is used to generate preliminary explanation text.

[0092] Preferably, the encoder and decoder of the Seq2Seq model both use LSTM networks, and use the beam search algorithm to generate diverse commentary content. This ensures that the generated commentary is both coherent and varied, avoiding giving tourists a monotonous feeling. For example, the system may generate multiple versions of the Broken Bridge introduction, including a concise version, a detailed version, and an interesting version, to meet the needs of different tourists.

[0093] Finally, the present invention also includes a real-time data processing module based on edge computing and a user emotion recognition module based on multimodal fusion. The edge computing module first performs data preprocessing and feature extraction on the local device, and then uses a lightweight neural network model for preliminary data analysis and decision-making. This design can significantly reduce the system response time and improve the user experience. For example, when a tourist suddenly turns his head to look at Leifeng Pagoda, the AR glasses can respond immediately and display relevant information without waiting for cloud processing.

[0094] The user emotion recognition module uses multimodal fusion technology to accurately judge the user's emotional state by simultaneously collecting the user's voice, facial expression and physiological signal data. For example, if the system detects that the tourist seems to be very interested after listening to the legend of the Broken Bridge, it may recommend more related legends; on the contrary, if the tourist seems to be tired, the system may suggest going to a nearby teahouse to rest.

[0095] Through the above detailed description, it can be seen that the intelligent tour guide robot and AR scenic spot interpretation system of the present invention not only integrates a variety of advanced technologies, but also fully considers the actual application scenarios and user experience. It can provide tourists with a personalized and immersive tour experience, greatly enhancing the attraction of the scenic spot and the cultural communication effect. At the same time, the modular design of the system and the application of advanced algorithms also provide convenience for future expansion and upgrading, making this system have broad application prospects and development potential.

[0096] In order to verify the superiority of the intelligent tour guide robot and the AR scenic spot interpretation system of the present invention, we conducted a series of comparative experiments. The following is a detailed comparison of the embodiment and the comparative example:

[0097] Example 1: We deployed the complete system of the present invention in the West Lake Scenic Area, including intelligent tour guide robots, AR glasses, cloud servers and all advanced algorithm modules. The system uses core technologies such as deep learning semantic understanding, multi-language real-time translation, AR technology, and knowledge graphs.

[0098] Comparative Example 1: In the same West Lake scenic area, we deployed a traditional tour guide service system, including a human tour guide and an ordinary voice guide. The system does not have AR functions and does not use advanced artificial intelligence technology.

[0099] We selected the following key indicators to evaluate system performance:

[0100] 1. Semantic understanding accuracy: measures the system’s understanding of tourists’ questions.

[0101] 2. Multilingual translation quality: evaluate the accuracy and fluency of system translation.

[0102] 3. AR content presentation effect: measure the stability and realism of AR content.

[0103] 4. User satisfaction: a comprehensive score obtained through a questionnaire survey.

[0104] 5. Average response time: the speed at which the system responds to user requests.

[0105] Detection method:

[0106] 1. Semantic understanding accuracy: 100 questions from tourists are randomly selected, and professionals are asked to judge whether the system’s understanding is accurate.

[0107] 2. Multilingual translation quality: Use BLEU score to evaluate translation quality, ranging from 0-100.

[0108] 3. AR content presentation effect: Professional evaluators will score based on stability and realism, with a maximum score of 10.

[0109] 4. User satisfaction: 100 tourists were randomly sampled and a questionnaire survey was conducted, with a full score of 10.

[0110] 5. Average response time: measures the average time from when the system receives a user request to when it responds, in seconds.

[0111] Test results:

[0112] index Example 1 Comparative Example 1 Semantic understanding accuracy 95％ 70％ Multilingual Translation Quality (BLEU) 85 60 AR content presentation effect 9.2 N / A Customer satisfaction 9.5 7.2 Average response time (seconds) 0.5 0.5

[0113] Analysis and interpretation:

[0114] 1. Semantic understanding accuracy: The system of the present invention has achieved a high accuracy of 95%, far exceeding the 70% of the traditional system. This is due to the Bi-LSTM model based on the attention mechanism we adopted, which can better capture contextual information and understand the real intentions of tourists. High accuracy means that the system can better meet the information needs of tourists and reduce communication barriers.

[0115] 2. Multi-language translation quality: The proposed method achieves 85 points in BLEU score, which is significantly higher than the 60 points of the traditional system. This is due to the Transformer-based neural machine translation model and multi-head attention mechanism we use. Higher-quality translation can not only help foreign tourists better understand tourist attraction information, but also provide a more fluent and authentic language experience.

[0116] 3. AR content presentation effect: The AR system of the present invention received a high score of 9.2 points, while the traditional system has no such function at all. Using Vuforia technology and SLAM algorithm, our system can present AR content stably and realistically, greatly enhancing the immersion and interactive experience of tourists. This technology makes it possible to reproduce historical scenes and opens up new ways to display cultural heritage.

[0117] 4. User satisfaction: The present invention achieved a high satisfaction score of 9.5, significantly higher than the 7.2 of the traditional system. This comprehensive indicator reflects that our system has achieved excellent results in information accuracy, experience interactivity, personalized services and other aspects. High satisfaction means that tourists are more likely to reuse our system and recommend it to others.

[0118] 5. Average response time: The response time of the present invention is only 0.5 seconds, which is much faster than the 5 seconds of the traditional system. This is due to the edge computing technology and optimized algorithm design we use. Fast response can provide a smoother user experience, especially in scenarios such as AR content presentation and real-time translation, where low latency is crucial.

[0119] In general, these test results fully demonstrate the significant advantages of the invention in terms of intelligence, personalization and user experience. Our system not only surpasses traditional solutions in terms of technical indicators, but more importantly, it provides tourists with a new and attractive way to navigate scenic spots. This innovation is expected to bring changes to the tourism industry, enhance the display of cultural heritage, and create a richer and deeper cultural experience for tourists.

[0120] Based on the above test results, we believe that Example 1 represents the best implementation of the present invention. It makes full use of advanced artificial intelligence technology and AR technology, and provides excellent user experience while ensuring high performance. In particular, the excellent performance in semantic understanding, multilingual translation and AR content presentation enables this system to meet the needs of modern tourists for intelligent and personalized guide services. In addition, the system's fast response time and high user satisfaction further prove its superiority in practical applications.

[0121] This optimal embodiment not only demonstrates the technical innovation of the present invention, but also reflects its great potential in enhancing tourism experience and promoting cultural communication. We believe that with further optimization and promotion, this intelligent tour guide system will be widely used in various scenic spots around the world, injecting new vitality into the development of the tourism industry.

[0122] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. Intelligent tour guide robot and AR scenic spot interpretation system, characterized by: include: The cloud server side includes a 3D model database of scenic spots, an augmented reality resource package database, a deep learning semantic understanding service module, and a multilingual text translation service module; An intelligent tour guide terminal, which includes an intelligent tour guide robot having a walking mechanism, a speech recognition module, a speech synthesis module, a natural language processing module and a human-computer interaction interface; An AR glasses terminal, comprising an AR glasses having a camera, a display screen, a processor, a memory and a communication module; The deep learning semantic understanding service module is used to convert the interactive voice request data of tourists into text form, and translate the text into a specific language, and then input the text translated into the specific language into the deep learning algorithm to identify the text type, thereby converting the voice into a scenic spot explanation text or operation instruction; The multilingual text translation service module is used to convert speech into a specific language, identify the text type, and perform machine translation; The intelligent tour guide robot and the AR glasses interact with each other through the cloud server to realize scenic spot guidance and AR content display; When tourists give voice commands to the intelligent tour guide robot, the intelligent tour guide robot encodes and encrypts the voice data and transmits it to the cloud server. The cloud server decrypts and decodes the received data, and then processes it through the deep learning semantic understanding service module to identify the command type and call the corresponding functional module based on the recognition result.

2. The intelligent tour guide robot and AR scenic spot interpretation system according to claim 1, characterized in that: The deep learning semantic understanding service module adopts a bidirectional long short-term memory network Bi-LSTM model based on the attention mechanism. The processing flow of the model includes: First, the input text is converted into word vectors through the embedding layer; Next, a bidirectional LSTM layer is used to capture contextual information; Then, an attention layer is used to weight important information; Finally, the final semantic understanding result is output through the fully connected layer; The softmax function is used to calculate the weights of the attention layer, ensuring that the sum of all weights is 1, thereby highlighting the importance of key information.

3. The intelligent tour guide robot and AR scenic spot interpretation system according to claim 1, characterized in that: The multilingual text translation service module adopts a neural machine translation model based on the Transformer architecture. The processing flow of the model includes: First, the encoder encodes the source language text into a vector representation; Then, the decoder generates the target language text based on the encoder’s output; In both encoding and decoding processes, a multi-head attention mechanism is used to improve translation quality; Among them, the multi-head attention mechanism captures language features at different levels by calculating multiple attention functions in parallel, thereby improving the accuracy and fluency of translation.

4. The intelligent tour guide robot and AR scenic spot interpretation system according to claim 1, characterized in that: It also includes a scenic spot data reconstruction module based on the HDF5 deep learning framework. The workflow of this module includes: First, the data preprocessing submodule cleans, standardizes, and enhances the historical data of scenic spots; Then, the model training submodule uses a method combining convolutional neural network (CNN) and generative adversarial network (GAN) to train the scenic spot reconstruction model; Next, the trained model parameters are compressed and optimized to adapt to the computing power of the Android terminal; Finally, the inference submodule runs the optimized model on the Android terminal to achieve real-time historical reconstruction of scenic spots; Among them, the generator and discriminator of the GAN network are trained alternately to continuously improve the authenticity and detail richness of the reconstruction effect.

5. The intelligent tour guide robot and AR scenic spot interpretation system according to claim 1, characterized in that: The AR glasses use Vuforia technology to achieve AR experience, and its workflow includes: First, the image recognition module uses a feature point matching algorithm to identify target objects in the real world; Then, the 3D object tracking module uses SLAM technology to achieve precise positioning of AR content; Next, the AR content rendering module uses OpenGL ES technology to merge virtual content with real scenes in real time; Finally, the fused image is presented through the display screen of AR glasses; Among them, SLAM technology achieves stable overlay and accurate positioning of AR content by simultaneously performing camera positioning and environmental map construction.

6. The intelligent tour guide robot and AR scenic spot interpretation system according to claim 1, characterized in that: It also includes an intelligent recommendation system based on deep reinforcement learning. The workflow of the system includes: First, collect and analyze users’ historical behavior data; Then, adjust the recommendation strategy based on real-time feedback; Next, the deep Q learning network DQN is used to select the best recommended action; Finally, provide users with personalized attraction recommendations and guided tours; Among them, the DQN network stabilizes the training process through the experience replay mechanism and the target network, thereby improving the performance and stability of the recommendation system.

7. The intelligent tour guide robot and AR scenic spot interpretation system according to claim 1, characterized in that: It also includes a scenic spot information retrieval system based on knowledge graph, the workflow of which includes: First, build a knowledge graph containing entities such as scenic spots, historical events, and people; Then, the knowledge graph is converted into a low-dimensional vector representation using graph embedding technology; Next, according to the user query, vector similarity calculation is used to quickly retrieve relevant information; Finally, the retrieval results are expanded through relational reasoning to provide in-depth background knowledge of scenic spots; Among them, the construction of the knowledge graph adopts a remote supervision method to automatically extract entity relationships from large-scale text corpora, and ensure the accuracy of knowledge through manual review.

8. The intelligent tour guide robot and AR scenic spot interpretation system according to claim 1, characterized in that: It also includes a dynamic commentary generation module based on natural language generation (NLG) technology. The workflow of this module includes: First, select appropriate content from the knowledge graph based on the user’s interests and knowledge level; Then, a Seq2Seq model based on the attention mechanism is used to generate preliminary explanation text; Next, a rule-based post-processing method is applied to ensure the coherence and correctness of the generated content; Finally, the generated commentary content is transmitted to the intelligent tour guide robot or AR glasses for display; Among them, the encoder and decoder of the Seq2Seq model both adopt LSTM networks, and use the beam search algorithm to generate diverse commentary content.

9. The intelligent tour guide robot and AR scenic spot interpretation system according to claim 1, characterized in that: It also includes a real-time data processing module based on edge computing, which is deployed on the smart tour guide robot and AR glasses. Its workflow includes: First, data preprocessing and feature extraction are performed on the local device; Then, a lightweight neural network model is used for preliminary data analysis and decision making; Next, the processing results are synchronized and verified with the cloud server; Finally, according to the local processing results and cloud feedback, corresponding operations are performed or content is displayed; Among them, the lightweight neural network model adopts model compression and knowledge distillation technology to reduce computational complexity while ensuring performance and adapt to the resource limitations of edge devices.

10. The intelligent tour guide robot and AR scenic spot interpretation system according to claim 1, characterized in that: It also includes a user emotion recognition module based on multimodal fusion. The workflow of this module includes: First, the user's voice, facial expression and physiological signal data are collected simultaneously; then, the respective feature extraction networks are used to process the data of different modalities; then, the features of different modalities are weightedly fused through the attention mechanism; after that, the fused features are classified using the multi-layer perceptron (MLP) to derive the user's emotional state; finally, the content and rhythm of the tour are dynamically adjusted according to the recognition results; among them, multimodal fusion adopts a late strategy to integrate the prediction results of different modalities at the decision-making level to improve the accuracy and robustness of emotion recognition.

Images