Exhibition hall guide auxiliary method and device based on large language model, equipment and medium

Abstract

The invention relates to the field of artificial intelligence, in particular to an exhibition hall guide auxiliary method and device based on a large language model, equipment and a medium. The method comprises the steps of collecting and recognizing questioning voices of visitors, converting the questioning voices into texts and then rapidly obtaining answers or prompts, processing the questioning texts through a large language model, and generating accurate and professional answer or prompt texts based on a huge knowledge base and an advanced natural language processing technology so as to give prompts to workers, so that the efficiency of the visitors is improved. Therefore, the staff can more smoothly and effectively solve the problems of visitors, the method is particularly important for temporary exhibition or new employees, the defect that the staff are unfamiliar with specific exhibits or businesses is effectively overcome, the stability and high level of service quality are ensured, and the service quality is improved. And the visiting experience of visitors is improved to a certain extent.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to an exhibition hall navigation auxiliary method, device, equipment and medium based on a large language model. Background Art

[0002] During exhibitions, staff will be arranged in the corresponding exhibition halls (museums, memorial halls, etc.) next to the exhibits and at the information desk to explain or answer questions. They will explain basic knowledge such as the origin of the exhibits to visitors, or answer questions about the exhibits or other questions (such as asking for directions) raised by visitors.

[0003] In order to ensure the visitor's experience, staff are generally trained before they take up their posts so that they have a certain understanding of the exhibits, exhibition hall regulations and infrastructure. However, when there are temporary exhibitions in the exhibition hall or the staff have just completed training, it is difficult for the staff to achieve the best working state when providing explanations or answers to visitors because they are relatively unfamiliar with the temporary exhibits or their business skills are still in the running-in stage. As a result, the fluency and depth of the explanations or answers will be affected to a certain extent, resulting in a poor experience for visitors. Summary of the invention

[0004] In order to improve the experience of visitors, the present application provides an exhibition hall tour assistance method, device, equipment and medium based on a large language model.

[0005] In the first aspect, the present application provides an exhibition hall tour assistance method based on a large language model, which adopts the following technical solutions: An exhibition hall navigation assistance method based on a large language model, comprising: Collect the voice of visitors' questions; Recognizing the question voice to convert the question voice into question text; Based on the question text, determining the question type corresponding to the question text; Inputting the question text and the question type into a large language model to obtain an output text as a prompt text corresponding to the question voice; Based on the prompt text, a prompt is given to the staff corresponding to the visitor.

[0006] By adopting the above technical solution, the voice of visitors' questions can be collected and recognized in real time, and the answers or prompts can be quickly obtained after being converted into text. The question text can be processed by a large language model. Based on a huge knowledge base and advanced natural language processing technology, accurate and professional answers or prompt texts can be generated to give prompts to staff, so that staff can answer visitors' questions more smoothly and confidently. This is especially important for temporary exhibitions or new staff. It effectively makes up for the shortcomings of staff's unfamiliarity with specific exhibits or businesses, ensures the stability and high level of service quality, and improves the visitors' visiting experience to a certain extent.

[0007] In a possible implementation, determining the question type corresponding to the question text based on the question text includes: Extracting question keywords from the question text; Determine the keyword category corresponding to the question keyword; Based on the keyword category, determine the question type corresponding to the question text.

[0008] By adopting the above technical solutions, different types of questions may require different processing methods or resource investment in exhibitions or similar scenarios. By accurately extracting keywords and mapping them to corresponding categories, the core content and intent of the question text can be more effectively identified. By accurately classifying the question types, staff or resources can be deployed more targetedly to ensure that key or complex issues are handled in a timely and effective manner, thereby improving the overall service quality and efficiency.

[0009] In a possible implementation, the keyword categories include exhibit categories, infrastructure categories, path categories, and other categories, and determining the keyword category corresponding to the question keyword includes: Calculating the similarity between the question keyword and each keyword category to obtain a similarity group; Taking the highest similarity in the similarity group as the target similarity; The keyword category corresponding to the target similarity is determined to be the keyword category corresponding to the question keyword.

[0010] By adopting the above technical solution, by quantifying the similarity between keywords and categories, the category to which the question keywords belong can be identified more accurately. The method reduces subjectivity and errors and improves the accuracy and reliability of classification.

[0011] In a possible implementation, the question text and the question type are input into a large language model to obtain an output text as a prompt text corresponding to the question voice, including: Inputting the question text into a convolutional neural network model to obtain a first feature vector; Inputting the first feature vector and the question type into a recurrent neural network model to obtain a second feature vector; Based on the first feature vector and the second feature vector, an output text is obtained to serve as a prompt text corresponding to the question voice.

[0012] By adopting the above technical solution, the question text is feature extracted through a convolutional neural network (CNN). When processing text data, CNN can effectively capture local features, such as word n-gram information, word order relationships, etc., and the question type is used as additional input information, and is sent to the recurrent neural network (RNN) together with the first feature vector extracted by CNN. RNN is good at processing sequence data and can capture long-term dependencies in text. By integrating question type information, RNN can more accurately understand the context and intention of the question, so that it can be more accurate and targeted when generating prompt text. Combining the hybrid model of CNN and RNN, it can fully utilize the advantages of the two networks, retaining the local features of the text and capturing long-term dependencies, improving the quality of the generated prompt text, and making it more natural, fluent and in line with the context.

[0013] In a possible implementation, obtaining output text based on the first feature vector and the second feature vector includes: Merging the first feature vector and the second feature vector to obtain a merged feature vector; Determine the maximum prompt duration corresponding to the question voice; The merged feature vector and the maximum prompt duration are input into a fully connected layer to obtain output text.

[0014] By adopting the above technical solution, by merging the first feature vector (which may contain basic semantic information of the text) with the second feature vector (which may contain additional context, emotion or other auxiliary information), effective information fusion is achieved, and the maximum prompt duration of the question speech is introduced as input, so that the model can take into account the length or complexity of the speech. By adjusting the model parameters to adapt to different prompt durations, the adaptability and robustness of the model to different types of input can be improved.

[0015] In a possible implementation, determining the maximum prompt duration corresponding to the question voice includes: Obtaining the current heart rate of the staff member corresponding to the visitor; Determine a heart rate range corresponding to the current heart rate; Based on the heart rate range corresponding to the current heart rate, a maximum prompt duration corresponding to the question voice is determined.

[0016] By adopting the above technical solution, by monitoring the heart rate of the staff, their current work status, stress level and concentration can be indirectly evaluated. The reminder duration can be adjusted based on this physiological indicator to ensure that the transmission of reminder information is neither too hasty to cause omissions nor too long to affect work efficiency, thereby providing a more personalized service experience.

[0017] In a possible implementation, based on the prompt text, prompting the staff corresponding to the visitor includes any of the following: Based on the prompt text, a prompt word is generated, and the prompt word is sent to a first prompt device of a staff member corresponding to the visitor, so as to prompt the staff member corresponding to the visitor; Based on the prompt text, a prompt voice is generated, and the prompt voice is sent to a second prompt device of the staff member corresponding to the visitor, so as to prompt the staff member corresponding to the visitor.

[0018] By adopting the above technical solution, two different forms of prompts (text and voice) are provided, so that workers can choose the most appropriate receiving method according to their preferences, current working environment or task requirements. For example, in a noisy environment, voice prompts may be more effective; while text prompts are more convenient when you need to quickly record or check information.

[0019] In the second aspect, the present application provides an exhibition hall navigation auxiliary device based on a large language model, which adopts the following technical solution: An exhibition hall navigation auxiliary device based on a large language model, comprising: The collection module is used to collect the voice of visitors’ questions; A conversion module, used for recognizing the question voice to convert the question voice into question text; A determination module, used to determine the question type corresponding to the question text based on the question text; An input module, used for inputting the question text and the question type into a large language model to obtain an output text as a prompt text corresponding to the question voice; The prompt module is used to provide prompts to the staff corresponding to the visitor based on the prompt text.

[0020] In a third aspect, the present application provides an electronic device, which adopts the following technical solution: An electronic device, comprising: at least one processor; Memory; At least one application, wherein at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute the exhibition hall tour assistance method based on a large language model as described in the first aspect above.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium, comprising: storing a computer program that can be loaded by a processor and execute the exhibition hall tour assistance method based on a large language model as described in the first aspect above.

[0022] To summarize, the present application includes the following beneficial technical effects: instant collection and recognition of visitors' question voices, quick acquisition of answers or prompts after conversion into text, and use of a large language model to process question texts, capable of generating accurate and professional answers or prompt texts based on a huge knowledge base and advanced natural language processing technology to give prompts to staff, thereby enabling staff to answer visitors' questions more fluently and confidently, which is especially important for temporary exhibits or new staff, effectively making up for their shortcomings of unfamiliarity with specific exhibits or businesses, ensuring the stability and high level of service quality, and improving visitors' visiting experience to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flowchart of an exhibition hall navigation assistance method based on a large language model provided in an embodiment of the present application; Figure 2 It is a block diagram of an exhibition hall navigation auxiliary device based on a large language model provided in an embodiment of the present application; Figure 3 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following is combined with Figure 1 -Attached Figure 3 This application is described in further detail.

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0027] It should be noted that in the optional embodiments of the present application, the object information and other related data involved, when the embodiments in the present application are applied to specific products or technologies, need to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if the embodiments of the present application involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information needs to obtain the consent of the individual. If sensitive information is involved, the separate consent of the information subject needs to be obtained. The embodiments also need to be implemented with the authorization and consent of the object.

[0028] The present application embodiment provides an exhibition hall navigation assistance method based on a large language model, such as Figure 1 As shown, the method provided in the embodiment of the present application is performed by an electronic device, which can be a server or a terminal device, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present application. The method includes steps S101 to S105, wherein: Step S101: collecting the voice of visitors' questions.

[0029] In order to make up for the staff's lack of familiarity with specific exhibits or businesses and ensure the stability and high level of service quality, the voice of visitors when asking questions can be collected to facilitate subsequent prompts based on the question voice. Specifically, microphones or voice collection devices with high-quality recording functions are arranged in the visiting area to ensure that the visitor's question voice can be clearly captured, and the sound in the exhibition hall is collected in real time. When the sound collected is greater than the decibel threshold, it is recorded as the visitor's question voice; or, when the visitor asks a question, the staff can start the recording device to record his voice through manual triggering (such as pressing a button) or automatic detection (such as using the wake-up word in voice recognition technology) to obtain the visitor's question voice.

[0030] Step S102: Recognize the question voice to convert the question voice into question text.

[0031] After obtaining the voice of the visitor, the question voice can be converted into question text for easy processing. Specifically, the collected question voice is preprocessed (such as denoising, framing, etc.), and the voice signal is converted into text information using speech recognition technology.

[0032] Step S103: Based on the question text, determine the question type corresponding to the question text.

[0033] Specifically, natural language processing technology can be used to analyze the question text, extract features such as keywords, phrases or sentence structures, and input the extracted features into a pre-trained classification model, which can identify and classify different types of questions (such as asking for directions, exhibit information, service requirements, etc.), and determine the specific question type corresponding to the question text based on the output of the classification model. The pre-trained classification model is trained through a large number of keyword samples, phrase samples, sentence structure samples, and question type samples.

[0034] Step S104: input the question text and question type into the large language model to obtain an output text as a prompt text corresponding to the question voice.

[0035] In this embodiment, the large language model is trained in advance with a large number of question text samples, question type samples and prompt text samples to obtain a trained large language model, and the question text and question type are used as input, so that the large language model generates corresponding prompt text based on the input information, and obtains output text as the prompt text corresponding to the question voice.

[0036] Step S105: Prompt the staff corresponding to the visitor based on the prompt text.

[0037] Specifically, after obtaining the prompt text generated by the large language model, the electronic device can transmit it to the staff responsible for receiving the visitor through the headphones worn by the staff. The staff will understand the visitor's questions and needs based on the received prompt text and provide corresponding answers or help accordingly.

[0038] More specifically, based on the location of the received question voice, the number of the staff member corresponding to the location is obtained to obtain the staff member corresponding to the visitor, wherein one staff member corresponds to one location and one number.

[0039] The embodiment of the present application provides an exhibition hall tour assistance method based on a large language model, which collects and recognizes the voice of visitors' questions, converts them into text to quickly obtain answers or prompts, and uses the large language model to process the question text. Based on a huge knowledge base and advanced natural language processing technology, it can generate accurate and professional answer or prompt text to give prompts to staff, so that the staff can answer visitors' questions more fluently and confidently. This is especially important for temporary exhibits or new staff, and effectively makes up for their shortcomings of being unfamiliar with specific exhibits or businesses, ensuring the stability and high level of service quality, and improving the visitors' visiting experience to a certain extent.

[0040] In a possible implementation of the embodiment of the present application, in the above step S103, based on the question text, determining the question type corresponding to the question text may specifically include: Extracting question keywords from the question text; Determine the keyword category corresponding to the question keyword; Based on the keyword category, determine the question type corresponding to the question text.

[0041] Before the exhibition begins, a keyword library containing various keywords and their corresponding categories can be constructed based on the exhibits in the exhibition and the infrastructure of the exhibition hall.

[0042] Specifically, the question text can be segmented by a word segmentation algorithm in natural language processing (NLP) to split it into several independent words or phrases, and possible question keywords can be screened out in the results after word segmentation. More specifically, meaningless words such as interjections and modal particles in the question text are removed by factors such as the part of speech, frequency, and position of each word in the results after word segmentation, and the remaining words except the removed words are compared with the keyword library, and the remaining words that are consistent with the keyword library are used as the screened question keywords.

[0043] Furthermore, by using an exact matching method, the extracted question keywords are matched with the entries in the keyword library to find the most suitable keyword category. Then, according to actual needs, a series of possible question types are defined, such as "route query", "exhibit information", "service demand", etc. Each question type corresponds to a set of keyword categories.

[0044] Furthermore, the question type to which the question text belongs is determined based on the correspondence between the keyword category and the question type. For example, assuming that the question text is "Excuse me, how do I get to the Ancient Egypt exhibition area on the third floor?", after word segmentation, we get "Excuse me", "how", "arrive", "third floor", "Ancient Egypt exhibition area", among which "how to arrive", "third floor", and "Ancient Egypt exhibition area" may be the keywords of the question. Looking for these keywords in the keyword category library, it is found that "how to arrive" belongs to the "navigation / route" category, "third floor" belongs to the "floor" category, and "Ancient Egypt exhibition area" belongs to the "exhibit area" category. Based on these keyword categories, combined with the definition of the question type, it can be determined that this question belongs to the "route query" type, because it asks how to get to a specific location (the Ancient Egypt exhibition area on the third floor).

[0045] A possible implementation of the embodiment of the present application, in the above embodiment, the keyword category may include an exhibit category, an infrastructure category, a path category, and other categories. In the above embodiment, determining the keyword category corresponding to the question keyword may specifically include: Calculate the similarity between the question keyword and each keyword category to obtain a similarity group; The highest similarity in the similarity group is taken as the target similarity; The keyword category corresponding to the target similarity is determined to be the keyword category corresponding to the question keyword.

[0046] Among them, the keyword categories may include: exhibit category: including the name or type of each exhibit in the museum, such as "ancient Egyptian artifacts", "modern artworks", etc.; infrastructure category: including the names of infrastructure in the museum, such as "restroom", "café", "rest area", etc.; path category: including path-related words, such as "arrival", "how to go", "navigation", etc., as well as possible combinations of floors and exhibition area names; other categories: including words that do not directly belong to the above categories, such as "service hours", "ticket prices", etc.

[0047] Specifically, the question pipe detection is preprocessed by removing stop words, stemming and restoring the word form, and then the similarity between the question keywords and the words in each category keyword library is calculated by a similarity algorithm. Among them, the similarity algorithm can be selected from cosine similarity, Jaccard similarity, edit distance (Levenshtein distance), etc. The similarity algorithm described in this embodiment is only used for illustration, and a suitable algorithm can be selected according to actual conditions.

[0048] Further, for each question keyword, traverse the keyword library of all categories, calculate the similarity between the keyword and each word in the library, and take the average value (or other aggregation method) of all similarities under the same category as the similarity between the keyword and the category. Then, combine the similarities calculated for each question keyword and each category into a similarity group, and find the highest similarity corresponding to each question keyword in the similarity group. The maximum value among these highest similarities is the target similarity, and the category corresponding to the target similarity is the keyword category corresponding to the question keyword. Exemplarily, the question keyword is "where are the ancient Egyptian artifacts", and after word segmentation processing, "ancient Egyptian artifacts" and "where are they" are obtained. Calculate similarity: For "ancient Egyptian artifacts", calculate its similarity with "ancient Egyptian artifacts" in "exhibit category" (may be 100%, if exact matching is used), as well as the similarity with words in other categories. For "where is it", calculate its similarity with relevant words in "path category" (such as "how to go", "navigation", etc.), and also calculate similarity with other categories for comparison.

[0049] Determine target similarity: Assume that "ancient Egyptian artifacts" has the highest similarity with "exhibit category", "where" has the highest similarity with "path category", and the maximum value of these two highest similarities comes from the similarity of "exhibit category".

[0050] Determine the keyword category: Therefore, it can be determined that the keyword category corresponding to this question keyword combination is "exhibit category". Although "where" also plays a key role, it indicates more of a query direction (i.e., finding a location) rather than directly pointing to the exhibit itself. However, in actual applications, the two may be combined for comprehensive judgment, for example, marking them as "exhibit category" and "path category" at the same time, and generating more detailed prompt text accordingly.

[0051] In a possible implementation of the embodiment of the present application, in the above step S104, the question text and the question type are input into the large language model to obtain an output text as a prompt text corresponding to the question voice, which may specifically include: Input the question text into the convolutional neural network model to obtain the first feature vector; Inputting the first eigenvector and the question type into the recurrent neural network model to obtain a second eigenvector; Based on the first feature vector and the second feature vector, an output text is obtained to serve as a prompt text corresponding to the question voice.

[0052] Specifically, a trained convolutional neural network (CNN) can be used to extract text features, convert the preprocessed text into a word embedding matrix, and use the word embedding matrix as the input of the CNN to obtain the first feature vector output by the CNN, which contains important feature information of the text. Among them, CNN extracts local features in the text through multiple convolutional layers, pooling layers and activation functions.

[0053] Furthermore, the question type (such as "exhibit information", "path query", etc.) is converted into an encoding form through one-hot encoding. Then, a recurrent neural network (RNN) is used to combine the features, and the first feature vector output by the CNN and the encoding vector of the question type are used as the input of the RNN. At each time step (or the last time step), the RNN outputs a hidden state as the second feature vector. . Among them, the RNN processes the input feature vectors sequentially and uses its internal state to capture the long-term dependencies in the sequence.

[0054] Furthermore, a fully connected layer is used to convert the final output of the RNN into a prompt in text form to obtain the output text, that is, the prompt text corresponding to the question voice. Among them, the cross-entropy loss function can be used as the loss function to train the fully connected layer. The cross-entropy loss function is a loss function commonly used in classification problems, which can measure the difference between the predicted probability distribution of the model and the actual label distribution. The cross-entropy loss function is used in this embodiment only for illustrative purposes. The embodiment of the present invention does not limit the selection of the loss function, and it can be set according to actual conditions. More specifically, this embodiment uses N samples and M labels corresponding to each sample to construct a cross-entropy loss function, wherein, assuming that for the i-th sample in the N samples, its true label is an M-dimensional vector Y_i, and the predicted result of the deep learning network model is an M-dimensional vector P_i, then the cross-entropy loss function can be expressed as: Loss = -(1 / N)*∑{i=1} {N} *∑{j=1} {M} *Y{ij}*log(P{ij}); Among them, Y_{ij} is the true value (0 or 1) of the jth label of the i-th sample, and P_{ij} is the model's predicted probability of the jth label of the i-th sample. The smaller the value of the cross entropy loss, the closer the model's prediction result is to the actual label.

[0055] A possible implementation of the embodiment of the present application, in the above embodiment, obtaining the output text based on the first feature vector and the second feature vector may specifically include: Merge the first eigenvector and the second eigenvector to obtain a merged eigenvector; Determine the maximum prompt duration corresponding to the question voice; The merged feature vector and the maximum prompt duration are input into the fully connected layer to obtain the output text.

[0056] Among them, the maximum prompt duration is pre-set.

[0057] Specifically, the first feature vector (from CNN) and the second feature vector (from RNN) need to be merged into a single feature vector. The merging method can be a simple concatenation, or a weighted sum or other more complex operations, which is not limited in the embodiments of the present application. The merged feature vector will contain comprehensive information from the original question text and question type, providing a basis for subsequent text generation.

[0058] Before the maximum cue duration is input to the fully connected layer, the maximum cue duration can be converted into a vector compatible with the dimension of the merged feature vector, or simply passed to the network as an additional numerical parameter. Specifically, the merged feature vector and the converted maximum cue duration are used as the input of the fully connected layer. If the maximum cue duration is converted into a vector, it can be directly concatenated with the merged feature vector; if the maximum cue duration is a separate value, the value corresponding to the maximum cue duration is used as an additional input to the network.

[0059] Get the final output of the fully connected layer, which is a vector corresponding to the output text, and use greedy search to generate text from the output vector of the fully connected layer. Further, the generated text may need to go through some post-processing steps, such as removing unnecessary punctuation, adjusting capitalization, applying grammar rules, etc., to ensure that the final prompt text is both accurate and easy to understand.

[0060] A possible implementation of the embodiment of the present application, in the above embodiment, determining the maximum prompt duration corresponding to the question voice may specifically include: Get the current heart rate of the staff member corresponding to the visitor; Determine the heart rate range corresponding to the current heart rate; Based on the heart rate range corresponding to the current heart rate, the maximum prompt duration corresponding to the question voice is determined.

[0061] Among them, each staff member in the exhibition hall wears a device that can monitor heart rate in real time, such as a smart watch, a heart rate monitoring belt or a sensor embedded in a uniform. Before the exhibition activity begins, a heart rate test can be performed on each staff member to determine the first heart rate of each visitor when he cannot answer the question at all, the second heart rate when he cannot answer the question partially, and the third heart rate when he can answer the question completely. Specifically, in the process of monitoring the heart rate of the staff, multiple questions (including difficult questions, general difficult questions and easy questions) can be asked to each staff member separately. Specifically, after collecting the question voice, the electronic device obtains the heart rate of the heart rate monitoring device corresponding to the visitor in real time to obtain the current heart rate of the staff member corresponding to the visitor.

[0062] Furthermore, after obtaining the first heart rate, second heart rate and third heart rate of each staff member, the heart rate range higher than the first heart rate is taken as the first heart rate range, the heart rate range between the first heart rate and the second heart rate is taken as the second heart rate range, the second heart rate and the third heart rate are taken as the third heart rate range, and the heart rate range lower than the third heart rate is taken as the fourth heart rate range.

[0063] Since a low heart rate range indicates a relaxed or low-stress state, and a high heart rate range indicates a tense or high-stress state, a low heart rate range may correspond to a shorter prompt duration, because the staff may have a certain degree of confidence in answering the questions asked by the visitor; and a high heart rate range corresponds to a longer prompt duration, because the staff may not have confidence in answering the questions asked by the visitor. Therefore, a prompt duration can be set for each heart rate range, for example, the first heart rate range is 1 minute, the second heart rate range is 40 seconds, the third heart rate range is 20 seconds, and the fourth heart rate range is 10 seconds. After obtaining the current heart rate of the staff member corresponding to the visitor, the current heart rate is compared with the heart rate range corresponding to the staff member to determine the heart rate range to which the current heart rate belongs, and the prompt duration corresponding to the heart rate range is obtained as the maximum prompt duration corresponding to the question voice.

[0064] A possible implementation of the embodiment of the present application is that in the above step S105, based on the prompt text, prompting the staff corresponding to the visitor may include the following methods 1 and 2. Specifically: Method 1: Generate prompt words based on the prompt text, and send the prompt words to the first prompt device of the staff member corresponding to the visitor, so as to prompt the staff member corresponding to the visitor.

[0065] Among them, each staff member corresponds to a first prompt device, and the first prompt device can be a prompt screen or a prompt tablet and other devices, which is not limited in this embodiment of the present application.

[0066] Specifically, the generated prompt text is processed as necessary through natural language processing, such as removing redundant information, extracting keywords or phrases, and the processed prompt words are sent to the first prompt device through wireless communication technology (such as Bluetooth, Wi-Fi, cellular network, etc.).

[0067] Furthermore, when the first prompting device receives the prompt word, the information is immediately displayed on the screen of the device so that the staff can quickly view and respond accordingly.

[0068] Method 2: Generate a prompt voice based on the prompt text, and send the prompt voice to the second prompt device of the staff corresponding to the visitor, so as to prompt the staff corresponding to the visitor.

[0069] Among them, each staff member corresponds to a second prompt device, and the second prompt device can be a device such as a headset or earphone, which is not limited to this in the embodiment of the present application.

[0070] Specifically, the electronic device can use text-to-speech (TTS) technology to convert the generated prompt text into a voice signal, and the electronic device sends the converted prompt voice to the second prompt device through wireless communication technology. When the second prompt device receives the prompt voice, the second prompt device immediately plays the voice information. The staff can listen to the voice prompt through headphones and other devices, and respond accordingly according to the prompt content.

[0071] The above embodiment introduces an exhibition hall navigation assistance method based on a large language model from the perspective of method flow. The following embodiment introduces an exhibition hall navigation assistance device based on a large language model from the perspective of a virtual module or a virtual unit. For details, please refer to the following embodiment.

[0072] See also Figure 2 The exhibition hall navigation auxiliary device 20 based on the large language model may specifically include: a collection module 201, a conversion module 202, a determination module 203, an input module 204 and a prompt module 205, specifically: An exhibition hall navigation auxiliary device 20 based on a large language model may include: The collection module 201 is used to collect the voice of visitors’ questions; The conversion module 202 is used to recognize the question voice to convert the question voice into question text; A determination module 203 is used to determine the question type corresponding to the question text based on the question text; An input module 204 is used to input the question text and question type into the large language model to obtain an output text as a prompt text corresponding to the question voice; The prompt module 205 is used to provide prompts to the staff corresponding to the visitor based on the prompt text.

[0073] In a possible implementation of the embodiment of the present application, when the determination module 203 determines the question type corresponding to the question text based on the question text, it can be specifically used to: Extracting question keywords from the question text; Determine the keyword category corresponding to the question keyword; Based on the keyword category, determine the question type corresponding to the question text.

[0074] In a possible implementation of the embodiment of the present application, the keyword category includes an exhibit category, an infrastructure category, a path category, and other categories. When determining the keyword category corresponding to the question keyword, the determination module 203 may be specifically used to: Calculate the similarity between the question keyword and each keyword category to obtain a similarity group; The highest similarity in the similarity group is taken as the target similarity; The keyword category corresponding to the target similarity is determined to be the keyword category corresponding to the question keyword.

[0075] In a possible implementation of the embodiment of the present application, when the input module 204 inputs the question text and the question type into the large language model to obtain the output text as the prompt text corresponding to the question voice, it can be specifically used to: input the question text into the convolutional neural network model to obtain the first feature vector; Inputting the first eigenvector and the question type into the recurrent neural network model to obtain a second eigenvector; Based on the first feature vector and the second feature vector, an output text is obtained to serve as a prompt text corresponding to the question voice.

[0076] In a possible implementation of the embodiment of the present application, when the input module 204 obtains the output text based on the first feature vector and the second feature vector, it can be specifically used to: Merge the first eigenvector and the second eigenvector to obtain a merged eigenvector; Determine the maximum prompt duration corresponding to the question voice; The merged feature vector and the maximum prompt duration are input into the fully connected layer to obtain the output text.

[0077] In a possible implementation of the embodiment of the present application, when the input module 204 determines the maximum prompt duration corresponding to the question voice, it can be specifically used to: Get the current heart rate of the staff member corresponding to the visitor; Determine the heart rate range corresponding to the current heart rate; Based on the heart rate range corresponding to the current heart rate, the maximum prompt duration corresponding to the question voice is determined.

[0078] In a possible implementation of the embodiment of the present application, when the prompt module 205 prompts the staff corresponding to the visitor based on the prompt text, it can be used in any of the following aspects: Generate a prompt word based on the prompt text, and send the prompt word to a first prompt device of a staff member corresponding to the visitor, so as to prompt the staff member corresponding to the visitor; Based on the prompt text, a prompt voice is generated, and the prompt voice is sent to a second prompt device of a staff member corresponding to the visitor, so as to prompt the staff member corresponding to the visitor.

[0079] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0080] See also Figure 3 , the embodiment of the present application also introduces an electronic device from the perspective of a physical device, such as Figure 3 As shown, Figure 3 The electronic device 30 shown includes: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, such as through a bus 302. Optionally, the electronic device 30 may also include a transceiver 304. It should be noted that in actual applications, the transceiver 304 is not limited to one, and the structure of the electronic device 30 does not constitute a limitation on the embodiments of the present application.

[0081] Processor 301 may be a CPU (Central Processing Unit), a general purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0082] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0083] The memory 303 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0084] The memory 303 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the contents shown in the above method embodiment.

[0085] Among them, electronic devices include but are not limited to: mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc., and can also be servers, etc. Figure 3 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0086] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding content in the aforementioned method embodiment.

[0087] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0088] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for assisting exhibition hall navigation based on a large language model, characterized in that: include: Collect the voice of visitors' questions; Recognizing the question voice to convert the question voice into question text; Based on the question text, determining the question type corresponding to the question text; Inputting the question text and the question type into a large language model to obtain an output text as a prompt text corresponding to the question voice; Based on the prompt text, a prompt is given to the staff corresponding to the visitor.

2. The exhibition hall navigation assistance method based on a large language model according to claim 1 is characterized in that: The determining, based on the question text, the question type corresponding to the question text includes: Extracting question keywords from the question text; Determine the keyword category corresponding to the question keyword; Based on the keyword category, determine the question type corresponding to the question text.

3. The exhibition hall navigation assistance method based on a large language model according to claim 2 is characterized in that: The keyword categories include exhibit categories, infrastructure categories, path categories, and other categories. The determining of the keyword category corresponding to the question keyword includes: Calculating the similarity between the question keyword and each keyword category to obtain a similarity group; Taking the highest similarity in the similarity group as the target similarity; The keyword category corresponding to the target similarity is determined to be the keyword category corresponding to the question keyword.

4. The exhibition hall navigation assistance method based on a large language model according to claim 1 is characterized in that: The step of inputting the question text and the question type into a large language model to obtain an output text as a prompt text corresponding to the question voice includes: Inputting the question text into a convolutional neural network model to obtain a first feature vector; Inputting the first feature vector and the question type into a recurrent neural network model to obtain a second feature vector; Based on the first feature vector and the second feature vector, an output text is obtained to serve as a prompt text corresponding to the question voice.

5. The exhibition hall navigation assistance method based on a large language model according to claim 4 is characterized in that: The step of obtaining output text based on the first feature vector and the second feature vector includes: Merging the first feature vector and the second feature vector to obtain a merged feature vector; Determine the maximum prompt duration corresponding to the question voice; The merged feature vector and the maximum prompt duration are input into a fully connected layer to obtain output text.

6. The exhibition hall navigation assistance method based on a large language model according to claim 5 is characterized in that: The determining of the maximum prompt duration corresponding to the question voice includes: Obtaining the current heart rate of the staff member corresponding to the visitor; Determine a heart rate range corresponding to the current heart rate; Based on the heart rate range corresponding to the current heart rate, a maximum prompt duration corresponding to the question voice is determined.

7. The exhibition hall navigation assistance method based on a large language model according to claim 1 is characterized in that: Providing a prompt to the staff corresponding to the visitor based on the prompt text includes any of the following: Generate a prompt word based on the prompt text, and send the prompt word to a first prompt device of a staff member corresponding to the visitor, so as to prompt the staff member corresponding to the visitor; Based on the prompt text, a prompt voice is generated, and the prompt voice is sent to a second prompt device of the staff member corresponding to the visitor, so as to prompt the staff member corresponding to the visitor.

8. An exhibition hall navigation auxiliary device based on a large language model, characterized in that: include: The collection module is used to collect the voice of visitors’ questions; A conversion module, used for recognizing the question voice to convert the question voice into question text; A determination module, used to determine the question type corresponding to the question text based on the question text; An input module, used for inputting the question text and the question type into a large language model to obtain an output text as a prompt text corresponding to the question voice; The prompt module is used to provide prompts to the staff corresponding to the visitor based on the prompt text.

9. An electronic device, characterized in that: The electronic device includes: at least one processor; Memory; At least one application, wherein the at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute the exhibition hall guide auxiliary method based on a large language model as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed in a computer, the computer is caused to execute the exhibition hall navigation assistance method based on a large language model as described in any one of claims 1 to 7.

Images