Portable tour guide method, system and equipment based on artificial intelligence and storage medium

Abstract

The invention relates to the technical field of AI tour guide, and discloses a portable tour guide method, system and device based on artificial intelligence and a storage medium, and the method comprises the steps: receiving a unique scenic spot identifier or a unique exhibit identifier through a portable tour guide terminal device, and sending the unique scenic spot identifier or the unique exhibit identifier to an AI artificial intelligence management server; tourist question texts are collected; the dialogue historical record and the tourist question text are input into a large language model for intention rewriting, a target intention text is obtained, an intention recognition result is generated, and a multi-path recall candidate set corresponding to the target intention text and the unique identifier of the scenic spot or the unique identifier of the exhibit is searched in parallel; the answer text is determined based on the multi-path recall candidate set, the audio waveform is generated and pushed to the portable tour guide terminal device to be played, the portable tour guide terminal device is combined with the AI artificial intelligence management server, the complete intelligent tour guide interaction system is constructed, and the personalized tour guide service is achieved.

Description

Technical Field

[0001] This invention relates to the field of AI tour guide technology, and in particular to a portable tour guide method, system, device and storage medium based on artificial intelligence. Background Technology

[0002] Existing tour guide equipment uses a one-way broadcasting method. It can only play pre-stored information locally or in the cloud, unable to listen to tourists' questions or search for relevant content in a database based on very limited keywords. This makes it difficult to understand and explain many user questions, hindering effective interaction and a better tour experience. Furthermore, human tour guides have limited resources, and while their group tours offer more professional explanations of attractions and exhibits, they are more expensive and rarely answer individual questions or accompany tourists for extended periods of free time, thus failing to provide differentiated services. Tourists can only obtain information about services within attractions through guides or online searches, but this doesn't fully cover local specialties and services, as the most detailed information remains with the local attractions and museums. Without a guide, tourists struggle to access this information promptly. Furthermore, the traditional AI large language model LLM working method only uses the basic capabilities of the AI ​​large language model to interact with users, but it cannot understand the user's intentions well, nor can it switch smoothly between the content of the attraction introduction and the user's ordinary questions. As a result, the user's questions cannot be answered satisfactorily, and therefore it is impossible to recommend various service facilities in the attraction, which affects the user's travel experience and the cultural promotion of various cultural and tourism attractions. Summary of the Invention

[0003] This invention provides a portable tour guide method, system, device, and storage medium based on artificial intelligence. By combining a portable tour guide terminal device with an AI artificial intelligence management server, this invention constructs a complete intelligent tour guide interaction system, realizing personalized tour guide services.

[0004] In a first aspect, the present invention provides a portable tour guide method based on artificial intelligence, the portable tour guide method based on artificial intelligence comprising: The system receives unique identifiers for attractions or exhibits through portable tour guide terminal devices and sends these unique identifiers to the AI ​​artificial intelligence management server. Collect tourist questions and answers; The dialogue history and the tourist's question text are input into a large language model for intent rewriting to obtain the target intent text, and intent recognition results are generated based on the target intent text. Based on the intent recognition results, a multi-path recall candidate set corresponding to the target intent text and the unique identifier of the attraction or the unique identifier of the exhibit is searched in parallel. The response text is determined based on the multi-channel recall candidate set, and an audio waveform is generated and streamed to the portable tour guide terminal device for playback.

[0005] In conjunction with the first aspect, in a first implementation of the first aspect of the present invention, before receiving the unique identifier of the attraction or the unique identifier of the exhibit through a portable tour guide terminal device and sending the unique identifier of the attraction or the unique identifier of the exhibit to the AI ​​artificial intelligence management server, the method further includes: The mini-program obtains the list of configurable parameters for the current service from the AI ​​artificial intelligence management server, and selects the IP tone parameters and language type according to the needs of tourists; Tourist audio recordings are collected and uploaded to a large model server for voiceprint feature extraction and language type detection. A configuration data package containing the IP timbre parameters, language type, and voiceprint feature vectors of the settable parameter list is constructed. The mobile phone transmits the configuration data packet to the setting information acquisition module of the portable tour guide terminal device via the NFC near-field communication protocol. The central processing unit of the portable tour guide terminal device sequentially detects the working status of the wireless radio frequency receiving circuit and NFC scanning circuit of the beacon information acquisition module, the working status of the camera module, the network connection status of the communication module, the MIC microphone acquisition sensitivity and speaker output power in the user interaction module, and the battery level of the power module to obtain the device self-test results.

[0006] In conjunction with the first aspect, in a second implementation of the first aspect of the present invention, the step of receiving a unique identifier for a scenic spot or a unique identifier for an exhibit via a portable tour guide terminal device, and sending the unique identifier for the scenic spot or the unique identifier for the exhibit to an AI artificial intelligence management server, includes: The portable tour guide terminal device's wireless radio frequency receiving circuit receives the wireless signal emitted by the scenic spot beacon and performs signal demodulation and data unpacking to extract the scenic spot's unique identifier; or the portable tour guide terminal device's NFC scanning circuit transmits a carrier signal to the exhibit's NFC tag to supply power and receives the exhibit's unique identifier through load modulation; or the portable tour guide terminal device's camera reads the exhibit, scenic spot image, or QR code containing a unique identifier to obtain the exhibit or scenic spot's characteristic parameters; the central processing unit obtains the scenic spot's unique identifier or the exhibit's unique identifier. The central processing unit encapsulates the unique identifier of the scenic spot or the unique identifier of the exhibit, the image features of the exhibit or scenic spot, the current GPS positioning coordinates of the device, the unique serial number of the device, and the current timestamp to obtain a location information data packet. The portable tour guide terminal device sends the location information data packet to the AI ​​artificial intelligence management server through the communication module. The AI ​​artificial intelligence management server stores the unique identifier of the attraction or the unique identifier of the exhibit, or the unique identifier of the attraction or exhibit obtained after recognizing the image features of the exhibit or attraction, in the dialogue context state record to obtain the device information that records the current location identifier.

[0007] In conjunction with the first aspect, in a third implementation of the first aspect of the present invention, the step of collecting tourist question text includes: The headphone in-line control button is connected to the voltage divider acquisition control circuit. When the user presses the headphone in-line control button, the voltage divider acquisition control circuit detects a change in the pin voltage, or the central processing unit detects that the physical button GPIO pin level changes from high level to low level. At this time, the central processing unit sends a stop playback command to the AI ​​artificial intelligence management server and starts the MIC microphone to enter the recording state. The MIC microphone collects tourists' voice signals, and the central processing unit calls the Voice Activity Detection (VAD) algorithm to determine the start and end points of the tourists' voice signals and generate audio data packets. The portable tour guide terminal device transmits the audio data packet to the AI ​​artificial intelligence management server through the communication module. The AI ​​artificial intelligence management server calls the ASR speech recognition model to decode and convert the audio data packet to obtain the tourist question text.

[0008] In conjunction with the first aspect, in the fourth implementation of the first aspect of the present invention, the step of inputting the dialogue history and the visitor's question text into a large language model for intent rewriting to obtain target intent text, and generating intent recognition results based on the target intent text, includes: The AI ​​artificial intelligence management server reads the dialogue history from the dialogue history database and concatenates the unique identifier of the scenic spot or exhibit in the device information, the dialogue history, and the tourist question text into a first prompt word. The first prompt word is input into a large language model for semantic understanding to obtain the target intent text; The target intent text is input into an intent classification model fine-tuned based on the large language model for intent reasoning, and intent recognition results are obtained by classifying the question type as either a scenic area-related question or a general knowledge question.

[0009] In conjunction with the first aspect, in a fifth implementation of the first aspect of the present invention, the step of searching in parallel, based on the intent recognition result, the multi-path recall candidate set corresponding to the target intent text and the unique identifier of the scenic spot or the unique identifier of the exhibit, includes: The target intent text is converted into a semantic vector, and the text fragment vectors corresponding to the unique identifiers of the attractions or exhibits are read from the vector database to obtain a vector retrieval result set. The maximum boundary correlation (MMR) algorithm is performed on the selected text segment vectors in the vector database to calculate the MMR retrieval result set. The target intent text is segmented into words to obtain query keywords, and the BM25 algorithm is executed in the full-text search database to calculate the fuzzy search result set. The vector retrieval result set, the MMR retrieval result set, and the fuzzy search result set are summarized to obtain a multi-path recall candidate set.

[0010] In conjunction with the first aspect, in a sixth implementation of the first aspect of the present invention, the step of determining the answer text based on the multi-channel recall candidate set and generating an audio waveform to be streamed to the portable tour guide terminal device for playback includes: Calculate the relevance score between the tourist question text and the candidate text fragments in the multi-path recall candidate set, and select a set of selected content based on the relevance score; The selected content set, the tourist question text, and the dialogue history are concatenated into a second prompt word. The second prompt word is input into the large language model for inference and generation. Based on the IP timbre parameters, the tone and expression style of the corresponding IP character are matched to generate the answer text. The answer text is input into the TTS speech synthesis model for text regularization and phoneme sequence conversion, and reconstructed into an audio waveform. Based on the IP timbre parameters, the corresponding timbre model weight file is loaded to generate personalized speech corresponding to the audio waveform. The personalized voice is streamed to the portable tour guide terminal device via the WebRTC protocol and played through the user interaction module.

[0011] Secondly, the present invention provides a portable tour guide system based on artificial intelligence, the portable tour guide system based on artificial intelligence comprising: The receiving unit is used to receive the unique identifier of the attraction or the unique identifier of the exhibit through a portable tour guide terminal device, and send the unique identifier of the attraction or the unique identifier of the exhibit to the AI ​​artificial intelligence management server. The data collection unit is used to collect text of tourist questions. The intent recognition unit is used to input the dialogue history and the tourist question text into the large language model for intent rewriting to obtain the target intent text, and generate intent recognition results based on the target intent text; A parallel search unit is used to search in parallel, based on the intent recognition result, the target intent text and the unique identifier of the attraction or the unique identifier of the exhibit, in a multi-way recall candidate set. The playback unit is used to determine the answer text based on the multi-channel recall candidate set and generate an audio waveform to be streamed to the portable tour guide terminal device for playback.

[0012] A third aspect of the present invention provides a portable tour guide device based on artificial intelligence, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the portable tour guide device based on artificial intelligence to perform the aforementioned portable tour guide method based on artificial intelligence.

[0013] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the aforementioned portable guide method based on artificial intelligence.

[0014] The technical solution provided by this invention constructs a complete intelligent tour guide interaction system by combining a portable tour guide terminal device with an AI artificial intelligence management server. First, by detecting voltage changes in the interrupt button on the headphone remote through a voltage divider acquisition control circuit, tourists can instantly interrupt and ask questions about the AI-generated content, breaking through the limitations of traditional one-way playback and establishing a true two-way interactive mechanism. Second, a two-stage processing flow combining intent rewriting and intent classification is adopted. The large language model first performs semantic simplification and keyword extraction on the tourist's questions, and then the intent classification model accurately determines whether the question belongs to a relevant question within the scenic area or a general knowledge question, effectively solving the technical problem that the AI ​​system cannot understand tourist intent and question type switching. Third, through a multi-path parallel recall strategy using vector similarity retrieval, maximum boundary relevance (MMR) algorithm retrieval, and ElasticSearch fuzzy search, candidate content related to attractions or exhibits is obtained from different dimensions. Then, a cross-encoder re-ranking model calculates deep semantic relevance scores and filters a selected set of content, improving the accuracy and diversity of content retrieval. Fourth, by dynamically loading the weight file of the TTS speech synthesis model based on the IP timbre parameters and adjusting the prosody parameters, differentiated and personalized voice broadcasts are generated for adult and child tourists, realizing personalized tour guide services and enhancing the tourist experience and the dissemination of scenic area culture.

[0015] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of one embodiment of the portable tour guide method based on artificial intelligence in this invention. Figure 2 This is a schematic diagram of one embodiment of the portable tour guide system based on artificial intelligence in this invention. Figure 3 This is a schematic diagram of one embodiment of a portable tour guide device based on artificial intelligence, as described in this invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The terms "comprising" and "having," and any variations thereof, used in the embodiments of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0020] To facilitate understanding of this embodiment, a portable tour guide method based on artificial intelligence, disclosed in this embodiment of the invention, will first be described in detail. For example... Figure 1 As shown, this method includes the following steps: 101. Receive the unique identifier of the attraction or exhibit through the portable tour guide terminal device, and send the unique identifier of the attraction or exhibit to the AI ​​artificial intelligence management server. It is understood that the executing entity of this invention can be a portable tour guide system based on artificial intelligence, or it can be a terminal or a server; the specific implementation is not limited here. This embodiment of the invention will be described using a server as an example.

[0021] Specifically, in outdoor scenic areas, the portable tour guide terminal's radio frequency receiving circuit receives wireless signals transmitted by attraction beacons in the 2.4GHz or 5.8GHz frequency band. Through signal demodulation and data unpacking, it extracts a unique attraction identifier in 16-bit hexadecimal encoding format. In indoor museum environments, the portable tour guide terminal's NFC scanning circuit transmits a 13.56MHz carrier signal to a passive NFC tag next to the exhibit to power it. The tag transmits the exhibit's unique identifier in 20-bit hexadecimal encoding format back to the terminal device via load modulation. After obtaining the attraction or exhibit unique identifier, the central processing unit encapsulates it with the device's current GPS coordinates, unique serial number, and current timestamp to form a location information data packet. This packet is then sent to the AI ​​management server via the communication module using HTTPS encrypted transmission protocol. The AI ​​management server stores the attraction or exhibit unique identifier in the dialogue context state record.

[0022] 102. Collect text of tourist questions; Specifically, when a tourist needs to ask a question, pressing the interrupt button on the headphone remote, connected via a Type-C interface, triggers the voltage divider acquisition control circuit to detect a change in the CC pin voltage from 0.5V to 1.2V. Alternatively, pressing the physical button on the terminal device's casing changes the GPIO pin level from high to low. Upon detecting the button signal, the central processing unit (CPU) immediately sends a stop playback command to the AI ​​management server and activates the microphone in the user interaction module to enter recording mode. The microphone acquires the tourist's voice signal at a 16kHz sampling rate and 16-bit depth. After preamplification and ADC conversion, the CPU uses the Voice Activity Detection (VAD) algorithm to analyze the energy envelope of the voice signal in real time, determining the start and end points of the speech. The speech is considered complete when the voice signal energy remains below a preset threshold for 500 milliseconds. The recorded audio data is compressed using an Opus encoder to form audio data packets. The portable tour guide terminal device transmits these audio data packets to the AI ​​management server via the WebRTC real-time audio / video transmission protocol through the communication module. The server then uses an ASR speech recognition model to decode the audio data packets and convert them into text format containing the tourist's question.

[0023] 103. Input the dialogue history and tourist question text into the large language model for intent rewriting to obtain the target intent text, and generate intent recognition results based on the target intent text; Specifically, the AI ​​management server reads the most recent five rounds of dialogue from the dialogue history database as the dialogue history record. It then concatenates the unique identifiers of attractions or exhibits stored in the dialogue context state record, the dialogue history record, and the tourist question text according to a preset format to construct a first prompt word containing complete contextual information. This first prompt word is input into a large language model for semantic understanding and text simplification. The large language model uses natural language processing techniques to remove colloquial expressions and redundant words from the tourist question text, extracting core intent keywords to generate concise and clear target intent text. Subsequently, the target intent text is input into an intent classification model fine-tuned based on the large language model for inference calculation. The intent classification model determines the question type based on the semantic features of the target intent text, classifying it into two main categories: scenic area-related questions and general knowledge questions. Scenic area-related questions are further subdivided into subcategories such as requests for attraction or exhibit information, inquiries about scenic area services and facilities, and restaurant recommendations, generating intent recognition results with specific category labels.

[0024] 104. Based on the intent recognition results, search in parallel the target intent text and the unique identifier of the attraction or exhibit for a multi-path recall candidate set. Specifically, the AI ​​management server initiates a multi-path recall strategy based on the intent recognition results. First, the target intent text is input into a text embedding model and converted into a semantic vector. Pre-segmented and embedded text fragment vectors corresponding to the unique identifiers of attractions or exhibits are read from the vector database. The cosine similarity between the semantic vector and each text fragment vector is calculated, and the top 20 text fragments are selected as the vector retrieval result set, sorted from highest to lowest similarity. Simultaneously, the Maximum Boundary Relevance (MMR) algorithm is performed on the selected text fragments in the vector database. By balancing the relevance of text fragments to the query and the diversity among text fragments through hyperparameters, the 15 text fragments with the highest MMR scores are selected as the MMR retrieval result set. In parallel, the target intent text is segmented into words and used as query keywords. The BM25 algorithm is executed in the ElasticSearch full-text retrieval database to calculate the term frequency-inverse document frequency (IF-IVF) score, and the 15 text fragments with the highest scores are selected as the fuzzy search result set. The vector retrieval result set, the MMR retrieval result set, and the fuzzy search result set are then aggregated and merged to form a multi-path recall candidate set containing multiple candidate text fragments.

[0025] 105. Determine the response text based on the multi-channel recall candidate set, and generate an audio waveform to be streamed to a portable tour guide terminal device for playback.

[0026] Specifically, the AI ​​management server concatenates the tourist's question text with candidate text fragments from a multi-channel recall candidate set using a preset delimiter, and then inputs the concatenation into a cross-encoder reordering model. This model calculates a deep semantic relevance score through a multi-layer attention mechanism. The server sets a relevance threshold of 0.6, filtering out candidate text fragments with scores below the threshold, and selecting the top 10 candidate text fragments with the highest scores as a curated content set. Subsequently, the curated content set, the tourist's question text, the dialogue history, and IP voice parameters are concatenated to construct a second prompt word, which is then input into a large language model for inference generation. The model generates the response text by matching the tone and expression style of the corresponding IP character based on the IP voice parameters. The response text is then input into a TTS speech synthesis model for text regularization and phoneme sequence conversion. The acoustic model predicts the Mel spectrum frame sequence, which is then reconstructed into an audio waveform by a vocoder. Based on the IP voice parameters, the corresponding voice model weight file is loaded to generate personalized speech. The personalized speech is then streamed to a portable tour guide terminal device via the WebRTC real-time audio and video transmission protocol, and played to the tourist through the speaker or headphone jack of the user interaction module.

[0027] In one specific embodiment, before receiving the unique identifier of the attraction or exhibit through the portable tour guide terminal device and sending the unique identifier of the attraction or exhibit to the AI ​​artificial intelligence management server, the following steps are also included: The mini-program obtains the list of configurable parameters for the current service from the AI ​​artificial intelligence management server, and selects the IP tone parameters and language type according to the needs of tourists; Collect tourist audio recording samples and upload them to a large model server for voiceprint feature extraction and language type detection. Construct a configuration data package containing IP timbre parameters, language type, and voiceprint feature vectors with a list of configurable parameters. The mobile phone transmits the configuration data packet to the setting information acquisition module of the portable tour guide terminal device via the NFC near-field communication protocol. The central processing unit of the portable tour guide terminal device sequentially detects the working status of the wireless radio frequency receiving circuit and NFC scanning circuit of the beacon information acquisition module, the working status of the camera module, the network connection status of the communication module, the MIC microphone acquisition sensitivity and speaker output power in the user interaction module, and the battery level of the power module to obtain the device self-test results.

[0028] Specifically, before receiving the unique identifier of a scenic spot or exhibit through the portable tour guide terminal device, the device needs to undergo an initial configuration process. Rental service personnel open the rental mini-program, which retrieves a tenant list from the AI ​​management server via a network connection. This list includes various IP voice types and language options. IP voice types are divided into two main categories: adult tour guide voices and children's tour guide voices. Each category contains multiple specific voice actor roles to choose from. Language options include Chinese, English, and other foreign languages. Rental service personnel select the appropriate IP voice parameters and language type identifiers on the mini-program interface based on the tourist's actual needs. For example, a professional-style adult IP voice is selected for adult tourists, and a lively children's IP voice is selected for children. After selection, the mini-program enters the detection interface and establishes a WebSocket connection with the large model server. The tourist uploads an audio sample via their mobile phone microphone. The large model server extracts voiceprint features and automatically detects the language type from this audio sample, providing real-time feedback to the mini-program interface. After confirming that the voiceprint information is normal and the language type is accurately identified, the mini-program encapsulates the selected IP voice parameters, language type identifier, and extracted voiceprint feature vectors into a configuration data package. Tourists use their mobile phones' NFC function to touch the NFC contact area on the portable tour guide terminal's casing. This allows for rapid transmission of configuration data packets to the terminal's settings acquisition module over a short distance via the NFC near-field communication protocol. Upon receiving the configuration data packet, the portable tour guide terminal's central processing unit immediately initiates a self-test program. This program sequentially checks the following: the operational status of the wireless radio frequency receiving circuit and NFC scanning circuit in the beacon information acquisition module; the stability of the 4G / 5G / WIFI network connection in the communication module; the audio acquisition sensitivity of the microphone and the audio output power of the speaker in the user interaction module; and the remaining battery power and supply voltage stability in the power module. The program generates a self-test result and communicates the self-test status to the user via indicator light color changes.

[0029] In this embodiment, when the device self-test results show that all modules are working normally, the following steps are also included: The portable tour guide terminal device establishes a WebRTC bidirectional communication connection with the AI ​​artificial intelligence management server through the communication module, creates a PeerConnection object and configures the audio track parameters, completes ICE candidate address negotiation and DTLS key exchange through signaling exchange, and establishes an end-to-end encrypted audio transmission channel; The portable tour guide terminal device uploads the IP timbre parameters and language type identifier to the AI ​​artificial intelligence management server, and the AI ​​artificial intelligence management server loads the corresponding TTS speech synthesis model parameter file from the model repository according to the IP timbre parameters. The TTS speech synthesis model parameter file includes an acoustic model weight file and a vocoder weight file; The AI ​​artificial intelligence management server calls the loaded TTS speech synthesis model to generate a verification audio with a preset verification text. The preset verification text includes the IP name corresponding to the IP timbre parameters and the current attraction name. The generated verification audio is pushed to the portable tour guide terminal device through the WebRTC audio transmission channel; The user interaction module of the portable tour guide terminal device receives the verification audio and plays it through the speaker or headphone interface. At the same time, the IP decorative component performs an eye-opening action and triggers a blinking action at preset time intervals during playback, completing the warm-up process of the TTS speech synthesis model and verifying the device's working status.

[0030] In one specific embodiment, the process of performing step 101 may specifically include the following steps: The portable tour guide terminal device's wireless radio frequency receiving circuit receives the wireless signal transmitted by the scenic spot beacon and performs signal demodulation and data unpacking to extract the scenic spot's unique identifier. Alternatively, the portable tour guide terminal device's NFC scanning circuit transmits a carrier signal to the exhibit's NFC tag to supply power and receives the exhibit's unique identifier through load modulation. Or, the portable tour guide terminal device's camera reads the exhibit or scenic spot's image, QR code containing a unique identifier, etc., to obtain the exhibit or scenic spot's characteristic parameters. The central processing unit then obtains the scenic spot's unique identifier or the exhibit's unique identifier. The central processing unit encapsulates the unique identifier of the attraction or exhibit, the current GPS positioning coordinates of the device, the image features of the exhibit or attraction, the unique serial number of the device, and the current timestamp to obtain a location information data packet; The portable tour guide terminal device sends location information data packets to the AI ​​artificial intelligence management server through the communication module. The AI ​​artificial intelligence management server stores the unique identifier of the attraction or exhibit, or the unique identifier of the attraction or exhibit obtained after recognizing the image features of the exhibit or attraction, in the dialogue context state record, thus obtaining the device information that records the current location identifier.

[0031] Specifically, when tourists move within the scenic area carrying portable tour guide terminal devices, the devices automatically acquire the location's attraction or exhibit identification information in two ways. For outdoor scenic areas, directional outdoor radio frequency beacons are deployed at various attractions. These beacons emit wireless signals containing unique identifiers of the attractions in the 2.4GHz or 5.8GHz frequency band, effectively covering a fan-shaped area with a radius of 10 to 30 meters around each attraction. When a tourist enters the beacon coverage area of ​​an attraction, the radio frequency receiving circuit in the portable tour guide terminal's beacon information acquisition module automatically receives the wireless signal emitted by that beacon. Through signal demodulation and data unpacking, the unique identifier of the attraction is extracted. This identifier uses a 16-bit hexadecimal encoding format, where the first 8 bits represent the scenic area code, and the last 8 bits represent the specific attraction number. For indoor museum exhibits, each exhibit is accompanied by a passive NFC tag containing the exhibit's unique identifier. This identifier uses a 20-bit hexadecimal encoding format, where the first 8 bits represent the museum code, the middle 8 bits represent the exhibition hall number, and the last 4 bits represent the exhibit number. Tourists bring the NFC scanning area of ​​their portable tour guide terminal device close to the NFC tag next to the exhibit. When the distance is less than 5 centimeters, the terminal device's NFC scanning circuit transmits a 13.56MHz carrier signal to the tag to power it. The tag then transmits its stored unique exhibit identifier data back to the terminal device via load modulation. After obtaining the unique identifier of the attraction or exhibit, the central processing unit immediately encapsulates this identifier along with the device's current location coordinates obtained via GPS, the device's unique serial number, and the current timestamp to generate a location information data packet containing complete location context information. The portable tour guide terminal device securely sends the location information data packet to the AI ​​artificial intelligence management server via a 4G / 5G / WIFI communication module using HTTPS encrypted transmission protocol. Upon receiving the location information data packet, the AI ​​artificial intelligence management server extracts the unique identifier of the attraction or exhibit and stores it in the dialogue context state record database, forming device information recording the current location.

[0032] In this embodiment, the AI ​​management server stores the unique identifier of the attraction or exhibit in the dialogue context state record. Obtaining the device information recording the current location identifier also includes: the AI ​​management server queries the management backend database for the corresponding content resource package based on the unique identifier of the attraction or exhibit. The unique identifier of the attraction includes the first 8 digits of the scenic area code and the last 8 digits of the attraction number; the unique identifier of the exhibit includes the first 8 digits of the museum code, the middle 8 digits of the exhibition hall number, and the last 4 digits of the exhibit number. The management backend database returns the corresponding content resource package based on the unique identifier of the attraction or exhibit. The content resource package includes textual background information about the attraction or exhibit, historical story text data, storage path data for related photos and videos, and story script data customized for the IP character corresponding to the IP voice parameters. The AI ​​management server inputs the story script data from the content resource package into a large language model for content generation, matches the explanation style of the corresponding IP character according to the IP voice parameters, and generates automatic broadcast text for the attraction or exhibit. The automatic broadcast text is input into a TTS speech synthesis model to convert it into automatic broadcast audio, which is then streamed to a portable tour guide terminal device via the WebRTC audio transmission channel for automatic playback, proactively introducing the current attraction or exhibit content to tourists.

[0033] In this embodiment, before receiving the unique identifier of the attraction or exhibit through the portable tour guide terminal device, the process includes the step of the scenic area or museum management uploading the content information of the attraction or exhibit and constructing a multimodal RAG database: The scenic area or museum management uploads the documentary materials, advertising materials, and introductory documents of the attraction or exhibit to the AI ​​artificial intelligence management server through the management backend. The AI ​​artificial intelligence management server performs token-level fragmentation processing on the uploaded text content, with each text fragment set to a length of 200 to 500 tokens, and adjacent text fragments overlapping by 50 tokens to maintain semantic coherence; The AI ​​artificial intelligence management server inputs the tokenized text fragments into a text embedding model to convert them into semantic vectors. Text fragments and their corresponding semantic vectors are associated and stored in a vector database. Simultaneously, an index mapping relationship is established between text fragments and unique identifiers for attractions or exhibits. The AI ​​management server performs paragraph-level and sentence-level text segmentation on the uploaded text content, stores the segmented text content in the ElasticSearch full-text search database, and builds a BM25 inverted index, supporting keyword-based fuzzy search. The AI ​​management server calls a large language model to extract knowledge from the uploaded text content, extracting knowledge entities and relationships in subject-verb-object triple structures. The extracted triple knowledge is stored in a graph database, and a knowledge graph is built, completing the construction process of the multimodal RAG database.

[0034] In one specific embodiment, the process of performing step 102 may specifically include the following steps: The headphone in-line control button is connected to the voltage divider acquisition control circuit. When the user presses the headphone in-line control button, the voltage divider acquisition control circuit detects a change in the pin voltage, or the central processing unit detects that the physical button GPIO pin level changes from high level to low level. The central processing unit then sends a stop playback command to the AI ​​artificial intelligence management server and starts the MIC microphone to enter the recording state. The MIC microphone collects tourists' voice signals, and the central processing unit calls the Voice Activity Detection (VAD) algorithm to determine the start and end points of the tourists' voice signals and generate audio data packets. The portable tour guide terminal device transmits audio data packets to the AI ​​artificial intelligence management server through the communication module. The AI ​​artificial intelligence management server calls the ASR speech recognition model to decode and convert the audio data packets to obtain the text of the tourist's questions.

[0035] Specifically, when the portable tour guide terminal is playing information about attractions or exhibits through the speaker or headphones of the user interaction module, tourists can trigger the interruption question mode by asking questions. The first method is using the wired headphones that come with the terminal. These headphones connect to the terminal via a Type-C interface. The interrupt button on the headphone's in-line control is connected to the CC configuration pin of the Type-C interface via a special voltage divider acquisition control circuit. When the tourist presses the interrupt button, the internal resistor changes the voltage division ratio. The voltage divider acquisition control circuit detects the voltage on the CC pin changing from the standard 0.5V to 1.2V. The central processing unit (CPU) acquires this voltage value through an ADC (Analog-to-Digital Converter) and determines it as an interrupt button press event. The second method is when the headphones are not used, the tourist directly presses the physical button on the terminal's casing. This button is connected to the CPU via a GPIO pin. When the button is pressed, the GPIO pin level changes from high to low, and the CPU detects this level change. Regardless of the method used, after detecting the interruption signal, the CPU immediately sends a stop playback command to the AI ​​artificial intelligence management server through the communication module and simultaneously activates the microphone in the user interaction module to enter recording mode. The microphone (MIC) captures tourists' voice signals in real time at a 16kHz sampling rate and 16-bit depth. The acquired analog audio signals are amplified by a preamplifier circuit and then sent to an ADC (Analog-to-Digital Converter) to convert them into digital audio data. The central processing unit (CPU) uses a locally integrated Voice Activity Detection (VAD) algorithm to analyze the digital audio data in real time. This algorithm determines the start and end points of the voice signal by calculating the short-time energy and zero-crossing rate. The voice signal is considered to have ended when its energy is below a preset threshold for 500 consecutive milliseconds. The recorded audio data is compressed using an Opus encoder to generate audio data packets. The portable tour guide terminal device transmits these audio data packets to an AI management server via a WebRTC real-time communication protocol. The server then uses an end-to-end ASR (Automatic Speech Recognition) model based on a Transformer architecture to decode the audio data packets, converting the voice signals into corresponding text to obtain the tourists' questions.

[0036] In this embodiment, the headphone in-line control button is connected to the voltage divider acquisition control circuit. When a tourist presses the headphone in-line control button, the voltage divider acquisition control circuit detects a change in the pin voltage. This includes: the headphone in-line control button is connected to the voltage divider resistor network of the voltage divider acquisition control circuit through the CC configuration pin of the Type-C interface. The voltage divider resistor network includes a voltage divider circuit composed of pull-up resistors and the internal resistance of the headphone in-line control button. When the tourist presses the headphone in-line control button, the internal resistance of the headphone in-line control button changes the voltage division ratio of the voltage divider circuit. The central processing unit (CPU) acquires the voltage change of the CC configuration pin from the standard value of 0.5V to 1.2V through the ADC analog-to-digital converter. The CPU determines that the button press event is interrupted based on the voltage change, immediately sends a stop playback command to the AI ​​artificial intelligence management server, and starts the MIC microphone to enter the recording state. At the same time, it stops the audio output of the portable tour guide terminal device. After receiving the stop playback command, the AI ​​artificial intelligence management server interrupts the currently streaming audio data and clears the audio buffer, entering a listening state to wait for tourist questions.

[0037] In this embodiment, the central processing unit (CPU) calls the Voice Activity Detection (VAD) algorithm to determine the start and end points of the tourist's voice signal and generate audio data packets. This includes: the CPU performing a Short Time Fourier Transform (STFT) on the tourist's voice signal collected by the microphone, converting the time-domain voice signal into a frequency-domain energy spectrum. The window length for each frame is set to 25 milliseconds, and the frame shift is set to 10 milliseconds. The VAD algorithm calculates the logarithmic energy value of the frequency-domain energy spectrum for each frame, compares the logarithmic energy value with a preset energy threshold, and determines the start point of the voice signal when the logarithmic energy value of three consecutive frames is higher than the energy threshold, recording the timestamp of the start point. The VAD algorithm continuously monitors the logarithmic energy value of subsequent frames, and determines the end point of the voice signal when the logarithmic energy value of 50 consecutive frames (500 milliseconds) is lower than the energy threshold, recording the timestamp of the end point. The CPU extracts a complete tourist voice segment based on the timestamps of the start and end points of the voice signal, compresses and encodes it using an Opus encoder at a bitrate of 32 kbps to obtain an audio data packet.

[0038] In one specific embodiment, the process of performing step 103 may specifically include the following steps: The AI ​​management server reads the dialogue history from the dialogue history database and combines the unique identifier of the attraction or exhibit in the device information, the dialogue history, and the text of the visitor's question into the first prompt word. The first prompt word is input into a large language model for semantic understanding to obtain the target intent text; The target intent text is input into an intent classification model fine-tuned based on a large language model for intent reasoning, and the intent recognition results are obtained by classifying the question type as either a relevant question within the scenic area or a general knowledge question.

[0039] Specifically, after receiving the tourist's question text, the AI ​​management server first reads the device's most recent five rounds of dialogue history from the dialogue history database. Each round of dialogue history includes the tourist's past questions and the AI ​​guide's past answers. The unique identifier of the attraction or exhibit stored in the device information, the read dialogue history, and the current tourist question text are concatenated according to a preset format template to form the first prompt word. The structure of this first prompt word is "Current Location: [Attraction or Exhibit Identifier], Dialogue History: [Past Dialogue Content], User Question: [Tourist Question Text]". This format provides complete contextual information for the large language model. The AI ​​management server inputs the first prompt word into the large language model for semantic understanding and text simplification. The large language model performs in-depth analysis of the tourist question text, removing redundant information such as colloquial expressions, interjections, and repetitive words. It then extracts core intent keywords by combining the current location information and the dialogue history context, generating concise and clear target intent text. For example, when a tourist says "I'm hungry, I want to eat something," the large language model will rewrite it as the target intent text "Find nearby restaurants." This intent rewriting process standardizes tourists' natural language expressions into a regulated intent description that facilitates subsequent processing. The AI ​​management server inputs the rewritten target intent text into an intent classification model for intent inference calculation. This intent classification model is a specialized classification model fine-tuned and trained on a large language model using an intent classification dataset. It accurately categorizes tourist questions into two main types: scenic area-related questions and general knowledge questions. Scenic area-related questions are further subdivided into several subcategories, such as requests for information on attractions or exhibits, inquiries about scenic area services and facilities, restaurant recommendations, inquiries about local specialties, and restroom locations. General knowledge questions include subcategories such as historical knowledge questions, scientific knowledge answers, and everyday conversational questions. The intent classification model uses a multi-layer neural network to extract features and perform classification inference on the target intent text, outputting the probability distribution of each category and selecting the category with the highest probability as the intent recognition result.

[0040] In this embodiment, the first prompt word is input into a large language model for semantic understanding to obtain the target intent text. This includes: inputting the first prompt word into the encoder of the large language model for contextual semantic encoding; extracting semantic association features between attraction or exhibit identifiers in the tourist question text, dialogue history, and device information through a multi-layer self-attention mechanism; the decoder of the large language model generates a simplified intent description text based on the semantic association features; removing modifiers, repeated words, and redundant modifiers from the tourist question text; extracting core verbs, nouns, and key entities to obtain the target intent text; the length of the target intent text is controlled between 30% and 50% of the length of the original tourist question text, retaining the core semantics of the tourist question text and eliminating colloquial expressions; and passing the target intent text and the unique attraction identifier or exhibit identifier in the first prompt word together to the intent classification model as input data for intent classification.

[0041] In this embodiment, the input of the target intent text into an intent classification model fine-tuned based on a large language model for intent reasoning, to obtain intent recognition results categorized as either scenic area-related issues or general knowledge-based issues, further includes: the AI ​​artificial intelligence management server calling the Agent planning module to select tools and plan execution paths based on the intent recognition results. When the intent recognition result is a scenic area-related issue, the Agent planning module selects the RAG retrieval tool and the geolocation service tool as the execution tool combination; when the intent recognition result includes service facility query subcategories such as restaurant recommendations, specialty shopping consultations, or restroom location queries, the Agent planning module adds the geolocation service tool to the execution tool combination, retrieving relevant service facility information within a 1-kilometer radius of the current GPS positioning coordinates of the portable tour guide terminal device from the service facility database; when the intent recognition result is a general knowledge-based issue, the Agent planning module selects a web search tool or a knowledge graph retrieval tool as the execution tool, without calling the RAG retrieval tool and the geolocation service tool; the Agent planning module then passes the execution tool combination and the target intent text to the corresponding retrieval module, initiating a multi-path parallel retrieval processing flow.

[0042] In one specific embodiment, the process of performing step 104 may specifically include the following steps: The target intent text is converted into a semantic vector, and the text fragment vectors corresponding to the unique identifiers of attractions or exhibits are read from the vector database to obtain a vector retrieval result set. The maximum boundary correlation (MMR) algorithm is performed on the selected text fragment vectors in the vector database to calculate the MMR retrieval result set. The target intent text is segmented into words to obtain query keywords, and the BM25 algorithm is executed in the full-text search database to calculate the fuzzy search result set; The vector retrieval result set, the MMR retrieval result set, and the fuzzy search result set are combined to obtain a multi-path recall candidate set.

[0043] Specifically, the AI ​​management server determines whether to invoke the RAG retrieval enhancement generation tool based on the intent recognition results. This tool employs a multi-path recall strategy to retrieve relevant content in parallel from multiple databases within the backend management server cluster. The vector similarity retrieval path is then executed, inputting the target intent text into a text embedding model for vectorization. This embedding model uses a BERT-based sentence-transformer model to convert the target intent text into a 768-dimensional semantic vector. Simultaneously, all pre-stored text fragment vectors associated with the unique identifiers of attractions or exhibits are read from the vector database. These text fragments are created by segmenting the descriptions of attractions or exhibits into segments of 200 to 500 tokens, with adjacent segments overlapping by 50 tokens to maintain semantic coherence. Each segment is converted into a 768-dimensional vector using the same embedding model and stored in the vector database. The server calculates the cosine similarity between the semantic vector of the target intent text and each text fragment vector. The cosine similarity value ranges from -1 to 1; a value closer to 1 indicates greater semantic relevance. The top 20 most relevant text fragments are selected as the vector retrieval result set after sorting by similarity from highest to lowest.

[0044] Among them, vector similarity retrieval (vector database): ; in It is a query text vector. It is the sentence vector being searched. The algorithm calculates the similarity based on the cosine value of the two vectors.

[0045] In parallel, the maximum boundary relevance (MMR) algorithm is executed in the same corpus of the vector database to retrieve the path. This algorithm considers not only the relevance of the text fragments to the query intent, but also the diversity among the selected text fragments. By introducing a hyperparameter λ, these two factors are balanced. The MMR score of each candidate fragment is calculated iteratively, and those text fragments that are highly relevant to the query intent and can provide new information are selected first, avoiding a large amount of duplicate or similar content in the retrieval results. Finally, the 15 text fragments with the highest MMR scores are selected as the MMR retrieval result set.

[0046] Among them, the Maximum Boundary Correlation Algorithm (MMR) (Vector Database): ; Where Q is the query text, C is the set of documents to be searched, R is the similarity set, and k is the index of the k sentences returned by the search. The algorithm adjusts the weights of the similarity between sentences and the overall similarity between the sentences and document C through hyperparameters, and then obtains the k sentences with the highest similarity.

[0047] Simultaneously, a third ElasticSearch fuzzy search path is executed. The target intent text is segmented using a Chinese word segmenter to extract core query keywords. The BM25 algorithm is then applied to the ElasticSearch full-text database for keyword matching. This database stores text containing attraction and exhibit descriptions processed using different segmentation strategies. The BM25 algorithm considers both the term frequency (TF) of the keywords within the document and the inverse document frequency (IDF) across the entire document set, calculating the relevance score of each document to the query keywords. The 15 text fragments with the highest scores are returned as the fuzzy search result set. After the three-way search is completed, the AI ​​management server merges and summarizes the vector search result set, the MMR search result set, and the fuzzy search result set, removing any potentially duplicate fragments to form a multi-path recall candidate set containing 30 to 50 candidate text fragments.

[0048] In this embodiment, the maximum boundary relevance (MMR) algorithm is performed on the selected text fragment vectors in the vector database to select the MMR retrieval result set. This includes: selecting the text fragment vector with the highest cosine similarity to the semantic vector of the target intent text from the vector retrieval result set as the first selected text fragment, and adding the first selected text fragment to the candidate result set; for the remaining unselected text fragment vectors in the vector retrieval result set, calculating the similarity between each unselected text fragment vector and the semantic vector of the target intent text as a relevance score, and calculating the maximum cosine similarity between each unselected text fragment vector and all selected text fragment vectors in the candidate result set as a diversity penalty score; calculating the MMR score for each unselected text fragment vector according to the preset hyperparameter λ value, where the MMR score is equal to the hyperparameter λ value multiplied by the relevance score minus F multiplied by the diversity penalty score, and F is set to 0.3; selecting the unselected text fragment vector with the highest MMR score and adding it to the candidate result set; repeating the MMR score calculation and text fragment vector selection process until the candidate result set contains G text fragment vectors, and using the candidate result set as the MMR retrieval result set, where G is set to 15.

[0049] In one specific embodiment, the process of performing step 105 may specifically include the following steps: Calculate the relevance score between the tourist question text and the candidate text fragments in the multi-way recall candidate set, and select the best content set based on the relevance score; The selected content set, tourist question text and dialogue history are concatenated into a second prompt word. The second prompt word is input into a large language model for reasoning and generation. Based on the IP timbre parameters, the tone and expression style of the corresponding IP character are matched to generate the answer text. The answer text is input into the TTS speech synthesis model for text regularization and phoneme sequence conversion, and then reconstructed into an audio waveform. Based on the IP timbre parameters, the corresponding timbre model weight file is loaded to generate personalized speech corresponding to the audio waveform. Personalized voice messages are streamed to portable tour guide terminal devices via the WebRTC protocol and played through the user interaction module.

[0050] Specifically, the AI ​​management server invokes a cross-encoder re-ranking model to fine-tune candidate text fragments in the multi-path recall candidate set. This re-ranking model employs a lightweight Cross-Encoder architecture, finely tuned based on the DistilBERT distillation model, with approximately 60 million parameters. The visitor's question text is concatenated with each candidate text fragment in the candidate set according to the format "[Question Text][SEP][Candidate Fragment]" and input into the cross-encoder re-ranking model. The model's multi-layer attention mechanism calculates the deep semantic relevance score between the question and the candidate fragments, with the score ranging from 0 to 1. The server sets a relevance threshold of 0.6, filtering out irrelevant candidate text fragments with scores below this threshold, and retaining the top 10 candidate text fragments with the highest scores to form a curated content set. By integrating a curated collection of content, visitor question text, dialogue history, and IP voice parameters, a generative prompt template is constructed: "You are [IP Name], introducing [Attraction or Exhibit Name] to visitors. Visitor Question: [Original Question]. Reference Materials: [Current Collection of Content]. Dialogue History: [Dialogue Record]. Based on the reference materials, please generate a natural and fluent response using the tone and style of [IP Character], with a response length controlled between 150 and 300 words." This second prompt is input into a large language model for content generation. The large language model automatically matches the tone and expression style of the corresponding IP character based on the character type in the IP voice parameters. For adult IPs, a professional and detailed expression style is used; for children's IPs, a vivid, interesting, and easy-to-understand expression style is used, ultimately generating response text that matches the characteristics of the IP character. The server calls the TTS (Text-to-Speech) module, loading the corresponding voice model weight file based on the IP voice parameters uploaded during device initialization. This voice model uses an end-to-end speech synthesis model based on FastSpeech2 or VITS architecture. The TTS module first performs text regularization on the response text, converting numbers into Chinese character pronunciations and punctuation marks into prosodic pause markers. Then, a text encoder converts the processed text sequence into a phoneme sequence. The phoneme sequence is input into an acoustic model to predict the Mel spectrum frame sequence corresponding to each phoneme. The Mel spectrum has 80 dimensions and a frame shift of 12.5 milliseconds. The acoustic model simultaneously predicts phoneme duration and pitch profile to control speech rate and intonation. Prosodic parameters are adjusted according to the characteristics of the IP character: the speech rate for adult IPs is set to 180 to 220 words per minute, and the speech rate for children's IPs is set to 150 to 180 words per minute with increased pitch variation.The predicted Mel spectrum sequence is input into a vocoder for waveform reconstruction. The vocoder uses a HiFiGAN model to convert the Mel spectrum into a 16kHz sampling rate PCM audio waveform. After loudness normalization and Opus audio compression, the synthesized audio waveform is streamed in real time through a WebRTC audio transmission channel established between the server and the terminal device. The network transmission bitrate is 32kbps, and the latency is controlled within 200 milliseconds. The central processing unit of the portable tour guide terminal device receives and decodes the audio stream data packets through the communication module and sends them to the audio playback buffer. The audio is then played to tourists through the speaker or headphone jack of the user interaction module. During playback, the IP decoration part synchronously executes speaking animations, controlling mouth opening and closing movements and blinking frequency according to the energy envelope of the audio waveform to enhance the vividness of the interactive experience.

[0051] In this embodiment, the relevance score between the tourist question text and the candidate text fragments in the multi-path recall candidate set is calculated, and a selected content set is selected based on the relevance score. This includes: concatenating the tourist question text and each candidate text fragment in the multi-path recall candidate set into an input sequence pair using a special delimiter SEP; inputting the input sequence pair into a cross-encoder reordering model for encoding; the multi-layer cross-attention mechanism of the cross-encoder reordering model simultaneously processes the token sequences of the tourist question text and the candidate text fragments, calculates the word-level semantic interaction features and sentence-level semantic matching features between the two, and outputs a relevance score between 0 and 1; sorting all candidate text fragments in the multi-path recall candidate set from high to low according to the relevance score, setting a relevance threshold, and filtering candidate text fragments with a relevance score lower than the relevance threshold; selecting the top E candidate text fragments with the highest relevance scores to form a selected content set, and passing the selected content set to a large language model for answer generation processing, where E is 10.

[0052] In this embodiment, the response text is input into the TTS speech synthesis model for text regularization and phoneme sequence conversion, reconstructing it into an audio waveform. Based on the IP timbre parameters, the corresponding timbre model weight file is loaded to generate personalized speech corresponding to the audio waveform. This includes: the TTS speech synthesis model performing text regularization on the response text, converting numbers to Chinese character pronunciations, converting punctuation marks to prosodic pauses, and converting the regularized text sequence into a phoneme sequence using a text encoder; the TTS speech synthesis model determining whether the current tour guide role is an adult tour guide or a child tour guide based on the IP timbre parameters, and when it is determined to be an adult tour guide... The acoustic model's speech rate parameter is set to 180 to 220 words per minute, and the fundamental frequency range of pitch is 80 to 200 Hz. When the character is identified as a child tour guide, the speech rate parameter is set to 150 to 180 words per minute, the fundamental frequency range of pitch is 150 to 300 Hz, and the pitch variation amplitude parameter is increased. The acoustic model predicts the Mel spectrum frame sequence and phoneme duration sequence corresponding to the phoneme sequence based on the speech rate parameter and the fundamental frequency range of pitch. The vocoder loads the corresponding timbre model weight file according to the IP timbre parameter, reconstructs the Mel spectrum frame sequence into an audio waveform with a sampling rate of 16 kHz, and generates personalized speech that conforms to the IP timbre parameter.

[0053] The above describes the portable tour guide method based on artificial intelligence in the embodiments of the present invention. The following describes the portable tour guide system based on artificial intelligence in the embodiments of the present invention. Please refer to [link / reference]. Figure 2 One embodiment of the portable tour guide system based on artificial intelligence in this invention includes: The receiving unit 201 is used to receive the unique identifier of the attraction or the unique identifier of the exhibit through the portable tour guide terminal device, and send the unique identifier of the attraction or the unique identifier of the exhibit to the AI ​​artificial intelligence management server. Collection unit 202 is used to collect text of tourist questions; The intent recognition unit 203 is used to input the dialogue history and the tourist question text into the large language model for intent rewriting, obtain the target intent text, and generate the intent recognition result based on the target intent text; Parallel search unit 204 is used to search in parallel the target intent text and the unique identifier of the attraction or exhibit based on the intent recognition result; The playback unit 205 is used to determine the answer text based on the multi-channel recall candidate set and generate an audio waveform to be streamed to the portable tour guide terminal device for playback.

[0054] Through the collaborative efforts of the aforementioned components, this invention constructs a complete intelligent tour guide interaction system by combining a portable tour guide terminal device with an AI artificial intelligence management server. By detecting voltage changes on the CC pin of the Type-C interface using a voltage divider acquisition control circuit and a VAD voice activity detection algorithm, the system accurately determines the start and end points of speech, enabling tourists to instantly interrupt and ask questions about the AI-generated content. This overcomes the limitations of traditional one-way playback, establishing a true two-way interactive mechanism that allows tourists to ask questions and receive immediate answers. A two-stage processing flow combining intent rewriting and intent classification is employed. The large language model first semantically simplifies tourist questions, removing colloquial expressions and redundant words, and extracts core intent keywords. Then, the intent classification model accurately determines whether the question belongs to a scenic area-related question or a general knowledge question. The Agent planning module intelligently selects the execution combination of RAG retrieval tools, geolocation service tools, or web search tools based on the intent recognition results. This effectively solves the technical challenge of the AI ​​system's inability to understand tourist intent and the switching of question types, achieving seamless switching between attraction introductions, service facility inquiries, and knowledge Q&A. A multi-parallel recall strategy, employing vector similarity retrieval, maximum boundary correlation (MMR) algorithm retrieval, and ElasticSearch's BM25 algorithm fuzzy search, retrieves candidate text fragments associated with unique identifiers of attractions or exhibits from three different dimensions: semantic similarity, content diversity, and keyword matching. A multi-layered cross-attention mechanism using a cross-encoder reordering model then calculates the deep semantic relevance score between the candidate text fragments and the tourist's question text, and a relevance threshold is set for filtering. This significantly improves the accuracy and diversity of content retrieval, ensuring that the answers are both relevant and rich. Based on IP timbre parameters, the weight file of the TTS speech synthesis model is dynamically loaded to generate differentiated personalized voice broadcasts for tourists of different age groups, achieving truly personalized tour guide services. An end-to-end encrypted real-time audio transmission channel is established using the WebRTC protocol, coupled with a model warm-up process to ensure initial response speed, comprehensively improving the system's real-time interactive performance and user experience, and promoting the effective dissemination of scenic area culture.

[0055] above Figure 2 The portable tour guide system based on artificial intelligence in the embodiments of the present invention will be described in detail from the perspective of modular functional entities. The portable tour guide device based on artificial intelligence in the embodiments of the present invention will be described in detail from the perspective of hardware processing.

[0056] Figure 3This is a schematic diagram of the structure of a portable tour guide device based on artificial intelligence provided in an embodiment of the present invention. The portable tour guide device 300 based on artificial intelligence can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 333 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the portable tour guide device 300 based on artificial intelligence. Furthermore, the processor 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the portable tour guide device 300 based on artificial intelligence to implement the steps of the aforementioned portable tour guide method based on artificial intelligence.

[0057] The portable tour guide device 300 based on artificial intelligence may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 3 The illustrated structure of the portable tour guide device based on artificial intelligence does not constitute a limitation on the portable tour guide device based on artificial intelligence provided by the present invention. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0058] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the portable guide method based on artificial intelligence.

[0059] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0060] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0061] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A portable tour guide method based on artificial intelligence, characterized in that, include: The system receives unique identifiers for attractions or exhibits through portable tour guide terminal devices and sends these unique identifiers to the AI ​​artificial intelligence management server. Collect tourist questions and answers; The dialogue history and the tourist's question text are input into a large language model for intent rewriting to obtain the target intent text, and intent recognition results are generated based on the target intent text. Based on the intent recognition results, a multi-path recall candidate set corresponding to the target intent text and the unique identifier of the attraction or the unique identifier of the exhibit is searched in parallel. The response text is determined based on the multi-channel recall candidate set, and an audio waveform is generated and streamed to the portable tour guide terminal device for playback.

2. The portable tour guide method based on artificial intelligence according to claim 1, characterized in that, Before receiving the unique identifier of a scenic spot or exhibit through a portable tour guide terminal device and sending the unique identifier of the scenic spot or exhibit to the AI ​​artificial intelligence management server, the process also includes: The mini-program obtains the list of configurable parameters for the current service from the AI ​​artificial intelligence management server, and selects the IP tone parameters and language type according to the needs of tourists; Tourist audio recordings are collected and uploaded to a large model server for voiceprint feature extraction and language type detection. A configuration data package containing the IP timbre parameters, language type, and voiceprint feature vector of the settable parameter list is constructed. The mobile phone transmits the configuration data packet to the setting information acquisition module of the portable tour guide terminal device via the NFC near-field communication protocol. The central processing unit of the portable tour guide terminal device sequentially detects the working status of the wireless radio frequency receiving circuit and NFC scanning circuit of the beacon information acquisition module, the working status of the camera module, the network connection status of the communication module, the MIC microphone acquisition sensitivity and speaker output power in the user interaction module, and the battery level of the power module to obtain the device self-test results.

3. The portable tour guide method based on artificial intelligence according to claim 2, characterized in that, The step of receiving a unique attraction identifier or exhibit identifier via a portable tour guide terminal device and sending the unique attraction identifier or exhibit identifier to the AI ​​artificial intelligence management server includes: The portable tour guide terminal device's wireless radio frequency receiving circuit receives the wireless signal emitted by the scenic spot beacon and performs signal demodulation and data unpacking to extract the scenic spot's unique identifier; or the portable tour guide terminal device's NFC scanning circuit transmits a carrier signal to the exhibit's NFC tag to supply power and receives the exhibit's unique identifier through load modulation; or the portable tour guide terminal device's camera reads the exhibit, scenic spot image, or QR code containing a unique identifier to obtain the exhibit or scenic spot's characteristic parameters; the central processing unit obtains the scenic spot's unique identifier or the exhibit's unique identifier. The central processing unit encapsulates the unique identifier of the scenic spot or the unique identifier of the exhibit, the image features of the exhibit or scenic spot, the current GPS positioning coordinates of the device, the unique serial number of the device, and the current timestamp to obtain a location information data packet. The portable tour guide terminal device sends the location information data packet to the AI ​​artificial intelligence management server through the communication module. The AI ​​artificial intelligence management server stores the unique identifier of the attraction or the unique identifier of the exhibit, or the unique identifier of the attraction or exhibit obtained after recognizing the image features of the exhibit or attraction, in the dialogue context state record to obtain the device information that records the current location identifier.

4. The portable tour guide method based on artificial intelligence according to claim 3, characterized in that, The collected tourist question text includes: The headphone in-line control button is connected to the voltage divider acquisition control circuit. When the user presses the headphone in-line control button, the voltage divider acquisition control circuit detects a change in the pin voltage, or the central processing unit detects that the physical button GPIO pin level changes from high level to low level. At this time, the central processing unit sends a stop playback command to the AI ​​artificial intelligence management server and starts the MIC microphone to enter the recording state. The MIC microphone collects tourists' voice signals, and the central processing unit calls the Voice Activity Detection (VAD) algorithm to determine the start and end points of the tourists' voice signals and generate audio data packets. The portable tour guide terminal device transmits the audio data packet to the AI ​​artificial intelligence management server through the communication module. The AI ​​artificial intelligence management server calls the ASR speech recognition model to decode and convert the audio data packet to obtain the tourist question text.

5. The portable tour guide method based on artificial intelligence according to claim 4, characterized in that, The process of inputting the dialogue history and the tourist's question text into a large language model for intent rewriting to obtain the target intent text, and generating intent recognition results based on the target intent text, includes: The AI ​​artificial intelligence management server reads the dialogue history from the dialogue history database and concatenates the unique identifier of the scenic spot or exhibit in the device information, the dialogue history, and the tourist question text into a first prompt word. The first prompt word is input into a large language model for semantic understanding to obtain the target intent text; The target intent text is input into an intent classification model fine-tuned based on the large language model for intent reasoning, and intent recognition results are obtained by classifying the question type as either a scenic area-related question or a general knowledge question.

6. The portable tour guide method based on artificial intelligence according to claim 5, characterized in that, The step of searching in parallel, based on the intent recognition result, for a multi-path recall candidate set corresponding to the target intent text and the unique identifier of the attraction or the unique identifier of the exhibit, includes: The target intent text is converted into a semantic vector, and the text fragment vectors corresponding to the unique identifiers of the attractions or exhibits are read from the vector database to obtain a vector retrieval result set. The maximum boundary correlation (MMR) algorithm is performed on the selected text segment vectors in the vector database to calculate the MMR retrieval result set. The target intent text is segmented into words to obtain query keywords, and the BM25 algorithm is executed in the full-text search database to calculate the fuzzy search result set. The vector retrieval result set, the MMR retrieval result set, and the fuzzy search result set are summarized to obtain a multi-path recall candidate set.

7. The portable tour guide method based on artificial intelligence according to claim 6, characterized in that, The step of determining the response text based on the multi-channel recall candidate set and generating an audio waveform to be streamed to the portable tour guide terminal device for playback includes: Calculate the relevance score between the tourist question text and the candidate text fragments in the multi-path recall candidate set, and select a set of selected content based on the relevance score; The selected content set, the tourist question text, and the dialogue history are concatenated into a second prompt word. The second prompt word is input into the large language model for inference and generation. Based on the IP timbre parameters, the tone and expression style of the corresponding IP character are matched to generate the answer text. The answer text is input into the TTS speech synthesis model for text regularization and phoneme sequence conversion, and reconstructed into an audio waveform. Based on the IP timbre parameters, the corresponding timbre model weight file is loaded to generate personalized speech corresponding to the audio waveform. The personalized voice is streamed to the portable tour guide terminal device via the WebRTC protocol and played through the user interaction module.

8. A portable tour guide system based on artificial intelligence, characterized in that, A portable guide method based on artificial intelligence as described in any one of claims 1-7, comprising: The receiving unit is used to receive the unique identifier of the attraction or the unique identifier of the exhibit through a portable tour guide terminal device, and send the unique identifier of the attraction or the unique identifier of the exhibit to the AI ​​artificial intelligence management server. The data collection unit is used to collect text of tourist questions. The intent recognition unit is used to input the dialogue history and the tourist question text into the large language model for intent rewriting to obtain the target intent text, and generate intent recognition results based on the target intent text; A parallel search unit is used to search in parallel, based on the intent recognition result, the target intent text and the unique identifier of the attraction or the unique identifier of the exhibit, in a multi-way recall candidate set. The playback unit is used to determine the answer text based on the multi-channel recall candidate set and generate an audio waveform to be streamed to the portable tour guide terminal device for playback.

9. A portable tour guide device based on artificial intelligence, characterized in that, The portable tour guide device based on artificial intelligence includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the AI-based portable tour guide device to perform the AI-based portable tour guide method as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the portable guide method based on artificial intelligence as described in any one of claims 1-7.

Images