A voice interaction method, system, device and storage medium

Abstract

A voice interaction method, system, device, and storage medium are disclosed. The method includes: acquiring historical voice interaction data of a target user, their viewing location, and an audio stream at that location; acquiring a knowledge graph of multiple exhibits within the area where the viewing location is located; determining the target user's voice interaction commands in the audio stream; associating the voice interaction commands with historical voice interaction data based on a preset contextual dialogue model to obtain contextual dialogue data; determining the target user's intent based on the voice interaction commands and contextual dialogue data; determining the explanation content based on the exhibit knowledge graph and the target user's intent, and generating a target display scene based on the explanation content; selecting a target virtual guide from a preset virtual guide library based on the explanation content, and displaying the target display scene; and having the target virtual guide explain the target display scene based on the explanation content. This application can improve the accuracy of voice interaction in VR exhibitions.

Description

Technical Field

[0001] This application relates to the field of voice interaction, specifically to a voice interaction method, system, device, and storage medium. Background Technology

[0002] With the rapid development of Virtual Reality (VR) technology, its application in museums, exhibition halls, and other similar settings is becoming increasingly widespread. Through VR interactive terminals, users can enjoy an immersive viewing experience and engage deeply with the exhibits. In this process, voice interaction, as a natural human-computer interaction method, effectively enhances the user experience and makes it easier for users to access exhibit information.

[0003] In related technologies, voice interaction solutions in VR exhibition scenarios typically employ speech recognition technology to convert user voice input into text commands, and then match and respond based on a pre-set question-and-answer database. The system retrieves the corresponding 3D models and explanatory content of the exhibits based on the user's location information, presenting and explaining them to the user through a virtual guide. This solution enables basic voice interaction functions, providing users with exhibit information and explanation services.

[0004] However, due to the diversity of user questioning methods and expression habits in VR exhibitions, and the fact that the same question intent may be expressed in many different ways, there may be significant deviations when matching user questions with a fixed question-and-answer knowledge base. This makes it difficult for the system to accurately understand the user's true intent, affecting the accuracy of voice interaction in VR exhibitions. Summary of the Invention

[0005] This application provides a voice interaction method, system, device, and storage medium for enhancing the voice interaction experience in VR exhibitions.

[0006] The first aspect of this application provides a voice interaction method applied to a VR interactive terminal. The method includes: acquiring historical voice interaction data of a target user, their viewing location, and an audio stream at that location; acquiring a knowledge graph of multiple exhibits within the area where the viewing location is located; determining the target user's voice interaction commands in the audio stream; associating the voice interaction commands with historical voice interaction data based on a preset contextual dialogue model to obtain contextual dialogue data; determining the target user's intent based on the voice interaction commands and contextual dialogue data; determining the explanation content based on the exhibit knowledge graph and the target user's intent, and generating a target display scene based on the explanation content; selecting a target virtual guide from a preset virtual guide library based on the explanation content, and displaying the target display scene; and having the target virtual guide explain the target display scene based on the explanation content.

[0007] Optionally, the voice interaction instructions of the target user are determined in the audio stream, specifically including: acquiring the target user's voice sample data; extracting the user's voice features from the voice sample data and constructing a voiceprint recognition model of the target user based on the voice features; performing noise reduction processing on the audio stream to obtain target audio data; recognizing the target user's voice segments in the target audio based on the voiceprint recognition model; and determining the target user's voice interaction instructions based on the voice segments.

[0008] Optionally, the target user intent is determined based on the voice interaction command and contextual dialogue data, specifically including: performing semantic analysis on the voice interaction command to determine the target user's first user intent; determining the target user's second user intent based on the first user intent and contextual dialogue data; and determining the target user intent through the first user intent and the second user intent.

[0009] Optionally, the target user intent is determined through the first user intent and the second user intent, specifically including:

[0010] Calculate the semantic similarity between the first user intent and the second user intent; based on historical voice interaction data, determine the first weight coefficient of each first intent feature in the first user intent and the second weight coefficient of each second intent feature in the second user intent; if the semantic similarity is greater than a preset similarity threshold, select multiple target user intent features from multiple first intent features and multiple second intent features according to the first weight coefficient and the second weight coefficient to obtain the target user intent; if the semantic similarity is less than or equal to the preset similarity threshold, perform intent feature fusion based on the first user intent, the second user intent, and historical voice interaction data to obtain the target user intent.

[0011] Optionally, the target user intent is obtained by fusing intent features based on the first user intent, the second user intent, and historical voice interaction data. Specifically, this includes: extracting intent topic words and intent operation words from the first user intent to obtain the first target intent feature; extracting contextual association features from the second user intent to obtain the second target intent feature; extracting the target user's third target intent feature from historical voice interaction data; and fusing the first target intent feature, the second target intent feature, and the third target intent feature to obtain the target user intent.

[0012] Optionally, a target display scene is generated based on the explanatory content, specifically including: extracting descriptive keywords of the target exhibits from the explanatory content, and determining the key display parts of the target exhibits based on the descriptive keywords; generating a camera view sequence of the key display parts, which is used to guide the viewing line of the target users; triggering corresponding animation effects at the key display parts based on the descriptive keywords and the camera view sequence; and combining the camera view sequence and the animation effects to obtain the target display scene.

[0013] Optionally, a voice interaction method further includes: acquiring three-dimensional model data of the target exhibit; performing semantic analysis on the narration content, identifying key narration nodes, and determining the semantic temporal relationship of the key narration nodes; dividing the three-dimensional model data into multiple display units based on the key narration nodes; determining the animation effect triggering sequence of each display unit according to the semantic temporal relationship; mapping the preset narration speed of the target virtual tour guide to the animation effect triggering sequence to generate interactive temporal control instructions; and controlling the display of animation effects in the target display scene based on the interactive temporal control instructions.

[0014] A second aspect of this application provides a voice interaction system, comprising: a first acquisition module for acquiring historical voice interaction data of a target user, the viewing location, and an audio stream at the viewing location; a second acquisition module for acquiring a knowledge association graph of multiple exhibits in the area where the viewing location is located; a first determination module for determining the target user's voice interaction command in the audio stream; an association module for associating the voice interaction command with historical voice interaction data based on a preset contextual dialogue model to obtain contextual dialogue data; a second determination module for determining the target user's intent based on the voice interaction command and the contextual dialogue data; a generation module for determining the explanation content based on the exhibit knowledge association graph and the target user's intent, and generating a target display scene based on the explanation content; a selection module for selecting a target virtual guide from a preset virtual guide library based on the explanation content, and displaying the target display scene; and an explanation module for explaining the target display scene through the target virtual guide based on the explanation content.

[0015] A third aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method described in any of the above.

[0016] A fourth aspect of this application provides a computer-readable storage medium storing instructions that, when executed, perform the method described in any of the preceding descriptions.

[0017] In summary, one or more technical solutions provided in this application have at least the following technical effects or advantages:

[0018] 1. By introducing historical interaction data as a reference sample, the system's understanding of user expressions is expanded; a knowledge graph of exhibits is acquired, establishing logical connections between exhibits and providing knowledge support for understanding user intent; based on this, the current voice interaction command is correlated with historical voice interaction data, and a contextual dialogue model is used to achieve contextual understanding of user expressions, thereby improving the accuracy of intent recognition; furthermore, based on accurately identified user intent, the system generates matching explanation content and display scenarios in conjunction with the exhibit knowledge graph, and selects an appropriate virtual guide for explanation, making the entire interaction process a complete closed loop, ensuring the coherence and accuracy from intent understanding to content display. Therefore, this application improves the accuracy of voice interaction in VR exhibitions through the synergy of multiple aspects such as historical data support, contextual understanding, knowledge association, and scene matching.

[0019] 2. By acquiring user voice samples to construct a voiceprint recognition model and combining it with noise reduction processing to optimize the audio signal, accurate voice command extraction in complex environments is achieved. This voiceprint recognition-based interaction mechanism effectively reduces environmental noise interference, enhances the anti-interference capability and accuracy of speech recognition, and solves the problem of difficulty in distinguishing multiple voices.

[0020] 3. A two-layer intent understanding framework is established by obtaining the first user intent through semantic analysis and combining it with a contextual dialogue model to obtain the second user intent. Credibility is assessed by calculating semantic similarity and weighting it with historical data to ensure stable intent judgment. When the similarity is high, the two intents are fused; when the similarity is low, multi-dimensional features are extracted and fused to enhance contextual coherence and reduce interference from single interactions. This mechanism integrates real-time semantic analysis and historical behavior, improving the accuracy and robustness of intent recognition. Attached Figure Description

[0021] Figure 1 This is a system architecture diagram involving a voice interaction method or a voice interaction system in the embodiments of this application;

[0022] Figure 2 This is a flowchart illustrating a voice interaction method in an embodiment of this application;

[0023] Figure 3 This is another flowchart illustrating a voice interaction method in an embodiment of this application;

[0024] Figure 4 This is a schematic diagram of the structure of a voice interaction system according to an embodiment of this application;

[0025] Figure 5 This is a schematic diagram of the structure of the electronic device in the embodiments of this application.

[0026] Explanation of reference numerals in the attached drawings: 401, First acquisition module; 402, Second acquisition module; 403, First determination module; 404, Association module; 405, Second determination module; 406, Generation module; 407, Selection module; 408, Explanation module; 409, Animation control module; 501, Processor; 502, Communication bus; 503, User interface; 504, Network interface; 505, Memory. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0028] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0029] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0030] Figure 1 An exemplary system architecture 100 is shown, which can be applied to an embodiment of a voice interaction method or a voice interaction system of this application.

[0031] like Figure 1 As shown, the system architecture 100 may include a VR interactive terminal 101, a network 102, and a server cluster 103. The network 102 serves as a medium to provide a communication link between the VR interactive terminal 101 and the server cluster 103. The network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0032] Users can use the VR interactive terminal 101 to interact with the server cluster 103 via the network 102 to receive or send voice commands, receive virtual tour guide explanations, etc. Various interactive applications can be installed on the VR interactive terminal 101, such as VR display applications, voice interaction applications, scene rendering applications, etc.

[0033] VR interactive terminal 101 is a hardware device with VR display capabilities, including but not limited to VR headsets and controllers. It should also be equipped with voice acquisition devices (such as microphone arrays), spatial positioning devices (for determining the user's viewing location), and audio playback devices (for playing virtual tour guide content). These devices work together to provide users with a complete VR voice interaction experience.

[0034] Server cluster 103 includes multiple functional servers, which may include: a speech recognition server, responsible for processing audio streams from terminals and recognizing user voice interaction commands; a dialogue management server, responsible for storing and managing users' historical voice interaction data, processing contextual dialogues, and analyzing user intent; a knowledge graph server, responsible for storing and managing the exhibit knowledge base and processing the relationships between exhibits; a virtual tour guide server, responsible for maintaining the virtual tour guide model library and generating corresponding explanation content based on user intent; and a scene generation server, responsible for generating target display scenes based on user intent and explanation content.

[0035] Server cluster 103 can be implemented as a distributed server cluster consisting of multiple servers, or as multiple functional modules within a single server. The servers or modules exchange data and collaborate via an internal network. Servers can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules (e.g., multiple software programs or software modules used to provide distributed services), or as a single software program or software module. No specific limitations are imposed here.

[0036] It should be noted that, to improve system response speed, the VR interactive terminal can have a local caching function to store frequently used scene data and virtual tour guide models. Simultaneously, the server cluster has a real-time data update mechanism to ensure timely updates of exhibit information and the knowledge graph.

[0037] It should be understood that Figure 1 The number of VR interactive terminals, networks, and server clusters shown is merely illustrative. Depending on implementation needs, any number of VR interactive terminals, networks, and servers can be included. In some embodiments, some functions can be implemented locally, in which case the system architecture may not be entirely dependent on network connectivity.

[0038] This system architecture, through the collaborative work of VR interactive terminals and server clusters, enables intelligent voice guidance based on the association between users' historical interaction data, viewing location, and exhibit knowledge, providing users with an immersive exhibition experience.

[0039] Figure 2 This is a flowchart illustrating a voice interaction method in an embodiment of this application.

[0040] Please see Figure 2 This application provides a voice interaction method, which includes:

[0041] S201. Obtain the target user's historical voice interaction data, exhibition location, and audio stream at the exhibition location;

[0042] In step S201, the system acquires the target user's historical voice interaction data, viewing location, and audio stream at that location. The system records all voice interaction operations performed by the user within the exhibition hall, and this data is stored in the server's user database. The user's historical voice interaction data includes dialogue between the user and the virtual guide, exhibit Q&A records, and information such as interests and preferences. This historical data reflects the target user's visiting habits and areas of focus in different exhibition areas; for example, a user in the bronze ware exhibition area might frequently inquire about the age and craftsmanship of the artifacts. For users using the system for the first time, the server creates a new user profile for data recording.

[0043] VR interactive terminals acquire the user's viewing location in real time through a built-in positioning system. This system can employ indoor positioning beacons, UWB, or other positioning technologies to establish a precise coordinate system within the exhibition space. The viewing location includes not only the target user's specific coordinates and orientation within the exhibition area but also the relative positions of surrounding exhibits. This allows the system to accurately determine the target user's exact location within the exhibition hall, such as "2 meters in front of the bronze tripod display case (exhibit number BC-001) in the southwest corner of the bronze ware exhibition area." This precise location information provides the foundation for the system to understand the range of exhibits the user is currently interested in.

[0044] The VR interactive terminal is equipped with a microphone array responsible for collecting the user's real-time voice and converting the voice signal into audio stream data. The system performs initial noise reduction and signal enhancement on the collected audio stream to ensure the quality of the voice data. The high-quality audio stream enables the system to accurately recognize the user's voice commands, such as specific questions like "What is the casting process of this bronze tripod?" The audio stream acquisition process takes into account the spatial acoustic characteristics of the exhibition hall, adjusting the microphone array's pickup parameters to adapt to the acoustic environment of different exhibition areas.

[0045] The combination of user location information and historical interaction data enables the establishment of user behavior trajectories within the exhibition hall, reflecting user visit paths and dwell patterns. The correlation between audio stream data and location information ensures the system correctly understands the exhibits the user is currently interested in. Historical interaction data provides crucial contextual references for understanding user voice commands. The comprehensive analysis of these three types of data provides a complete data foundation for the system's subsequent voice command recognition, user intent understanding, and personalized explanations.

[0046] S202. Obtain a knowledge graph of multiple exhibits in the area where the exhibition location is located;

[0047] The exhibit knowledge graph is stored in the server's knowledge base, constructing multi-dimensional relationships between exhibits through ontology. The exhibit knowledge graph includes basic attribute information such as the exhibit's name, era, material, and purpose, as well as deeper connections such as spatiotemporal relationships, cultural heritage relationships, and technological evolution relationships between exhibits. For example, in the bronze ware exhibition area, the exhibit knowledge graph can describe the evolution of bronze ware casting techniques in different periods, the development of artifact decorations, and the correlation characteristics between bronze ware and other cultural relics from the same period.

[0048] Based on the user's viewing location, the system retrieves a knowledge graph of the exhibits in that area from the server's knowledge base. When the user is in the bronze ware exhibition area, the system focuses on acquiring a knowledge network related to bronze ware. This network includes not only knowledge nodes of exhibits near the user's current location but also information on other exhibits related to these items. For example, when a user observes a Shang Dynasty bronze ding (a type of ancient Chinese cooking vessel), the system extracts the ding's characteristic information, as well as information on other bronze ware of similar age, decoration, or usage, constructing a knowledge graph centered on that ding. The dynamic acquisition process of the exhibit knowledge graph is closely related to the user's historical interaction data. The system expands the knowledge graph in a targeted manner based on the user's previously expressed interests and preferences. If the user has repeatedly inquired about the casting techniques of bronze ware, the system will strengthen knowledge nodes related to these techniques when constructing the exhibit knowledge graph.

[0049] S203. Determine the target user's voice interaction commands in the audio stream;

[0050] Specifically, the process involves acquiring voice sample data of the target user; extracting the user's voice features from the voice sample data and constructing a voiceprint recognition model for the target user based on the voice features; performing noise reduction on the audio stream to obtain target audio data; identifying the target user's speech segments in the target audio based on the voiceprint recognition model; and determining the target user's voice interaction commands based on the speech segments.

[0051] When a user first registers or initializes the voice interaction function, the system will guide the user to read a series of preset text contents to collect the voice sample data of the target user. These preset text contents not only include basic instruction statements (such as "Turn on the light"), but also more complex descriptive statements (such as "Please introduce the historical background of this exhibit") to ensure that the system can capture the voice characteristics of the user in different contexts. In addition, to enhance the system's compatibility with dialects, the user needs to read the same text content in Mandarin and the local dialect respectively, so as to generate cross-language acoustic feature data. During the voice sample collection process, the system uses the microphone array equipped with the VR interaction terminal or intelligent device to record the voice performance of the user at different volumes, speech rates, and intonations.

[0052] After collecting the voice sample data of the user, the system analyzes the voice sample data to extract the personalized voice characteristics of the user and constructs a voiceprint recognition model for the target user based on the voice characteristics. Specifically, the system first performs voice frame processing on the user's voice sample data and extracts its acoustic feature parameters, including but not limited to the fundamental frequency feature, pitch feature, phoneme feature, formant feature, and intonation feature of the voice. The acoustic feature parameters can comprehensively reflect the user's pronunciation habits and can be used for subsequent identity recognition and voice command parsing. To improve the system's support ability for dialects, the voiceprint recognition model performs dialect feature mapping based on the voice sample data through a preset dialect feature mapping model. The preset dialect feature mapping model is trained through a large-scale dialect voice database. This database contains voice samples of dialect users in different regions in different scenarios, and each voice sample is labeled with the corresponding dialect type, semantic information, and standard Mandarin comparison. The system uses deep learning algorithms to gradually establish the corresponding rules between dialects and standard Mandarin by analyzing the acoustic features, voice patterns, and semantic associations in the dialect voice samples, thereby forming a dialect feature mapping model. For example, the system will match the acoustic features of the Sichuan dialect "这个啥子" with the acoustic features of the Mandarin "这是什么", or map the Cantonese "呢个系咩" to the Mandarin "这个是什么".

[0053] When a user issues a voice command in a real-world application scenario, the system first performs noise reduction processing on the received audio stream to remove environmental noise and extract the target audio data. It's important to note that the system has already performed preliminary noise reduction preprocessing during the audio stream acquisition phase to reduce significant background noise. However, because the actual environment may contain complex noise interference (such as conversations of other visitors in an exhibition hall, background music, footsteps, etc.), the system employs a more advanced adaptive noise reduction algorithm at this stage to further optimize the quality of the target audio data. For example, the system can utilize beamforming technology combined with the directional pickup capabilities of a microphone array to enhance the voice signal from the user's direction while suppressing noise from other directions. Furthermore, the system dynamically adjusts noise reduction parameters based on the user's environment to obtain the optimal voice signal in different scenarios. To further enhance system security and user experience, during the noise reduction process, the system re-verifies whether the user has authorized the use of their voiceprint data to ensure that only the voice of authorized users is processed and to prevent unauthorized voice data from being misidentified or stored.

[0054] After noise reduction, the system needs to identify the target user's speech segments from the target audio data. Since the target audio data may contain speech input from multiple users, the system analyzes the target audio data using the user's voiceprint recognition model to accurately locate the target user's speech segments. Specifically, the system employs a deep learning-based voiceprint comparison algorithm to match the target audio data with the target user's voiceprint recognition model, thus eliminating speech interference from non-target users.

[0055] After successfully extracting the target user's voice segment, the system needs to perform speech recognition and natural language processing to analyze the user's intent and convert it into corresponding voice interaction commands. Specifically, the system first converts the target user's voice content into text format and standardizes the dialect expression based on the previously established dialect voiceprint feature mapping model. For example, if a user asks "What is this?" in Sichuan dialect, the system will convert it into "Please introduce this exhibit" in Mandarin and further match it to the system's standard interaction command set to trigger the corresponding operation. In addition, the system can also combine contextual information and the user's historical interaction records to intelligently complete and optimize voice commands. For example, if a user says "Change it again" while browsing exhibits, the system can understand their intent as "Change the exhibit introduction" based on the current interaction state, rather than misinterpreting it as "Change the music".

[0056] S204. Based on the preset contextual dialogue model, associate the voice interaction commands with historical voice interaction data to obtain contextual dialogue data;

[0057] The system processes user voice interaction commands and historical voice interaction data through a pre-defined contextual dialogue model. This model, trained using deep learning on a large amount of historical dialogue data, comprises three core functional modules: dialogue understanding, context association, and semantic representation. The system first performs semantic parsing on the user's current voice interaction command, extracting key entities, query attributes, and indicator words from the dialogue. For example, when a user asks, "What era is this bronze artifact from?", the system parses out the key entity "bronze artifact" and the query attribute "era," while marking "this artifact" as a pronoun requiring contextual understanding.

[0058] The system retrieves historical voice interaction data within a preset time period (e.g., 30 minutes) to establish semantic connections between the current dialogue and historical dialogues. During this process, the system focuses on identifying and processing pronouns, omitted information, and topic continuation within the dialogue. For example, when a user, after learning about the age of a bronze artifact, asks, "Why could such exquisite artifacts be produced during this period?", the system automatically associates "this period" with the specific era mentioned in the previous dialogue. When a user simply asks, "What do the patterns represent?", the system automatically supplements the previously mentioned exhibit information to construct a complete semantic query. When a user's question extends from the age and craftsmanship of the bronze artifact to its cultural connotations, the system identifies the cultural connotations as an extended discussion within the same theme.

[0059] The system transforms the processed dialogue information into structured contextual dialogue data containing current query intent, historical association information, topic continuation markers, and interest preference features. This data not only records the user's current specific inquiry but also includes key points from related historical dialogues and the user's demonstrated knowledge interests. This provides crucial information for subsequent user intent analysis, exhibit knowledge association graph retrieval, and virtual tour guide explanations. Simultaneously, the system updates the user's dialogue feature database in real time. When it detects frequent use of certain expressions or sustained interest in specific types of knowledge, it records these features to continuously optimize the accuracy of dialogue understanding and content recommendation.

[0060] S205. Determine the target user's intent based on voice interaction commands and contextual dialogue data;

[0061] In step S205, the system determines the target user's true intent through multi-dimensional analysis based on voice interaction commands and contextual dialogue data. This process is mainly achieved through three key steps: semantic analysis, contextual association, and weight calculation. The system first performs independent semantic analysis on the current voice interaction command to obtain a preliminary understanding of the user's intent; then, it performs deep semantic association analysis based on the contextual dialogue data to obtain a user intent understanding that takes into account the historical dialogue background. By calculating the semantic similarity between these two understandings and combining them with user behavior features from historical interaction data, the system can more accurately grasp the user's true needs. When the two understandings have high semantic consistency, the system will fuse the advantageous features of both understandings through weight allocation; when there are significant differences, a feature fusion mechanism will be activated to comprehensively analyze various information to determine the final user intent. This multi-layered intent recognition mechanism ensures that the system can accurately understand the user's true needs in different dialogue scenarios, providing precise intent guidance for subsequent personalized explanation services.

[0062] In one possible implementation, refer to Figure 3 , Figure 3 This is another flowchart illustrating a voice interaction method in an embodiment of this application. Determining the target user's intent based on voice interaction commands and contextual dialogue data may include steps S301 to S304:

[0063] S301. Perform semantic analysis on the voice interaction commands to determine the primary user intent of the target user;

[0064] Upon receiving a voice interaction command from a target user, the system first performs semantic analysis to determine the user's primary intent. Semantic analysis may include lexical parsing, syntactic analysis, semantic modeling, and intent classification. The system parses the core meaning of the user's voice command based on an exhibit knowledge graph and a pre-trained semantic understanding model (such as BERT). For example, when a user asks, "Why is the glaze of this porcelain piece so special?" in front of an ancient porcelain exhibit, the system extracts "porcelain" as the exhibit category and "glaze" as the query topic, mapping them to relevant knowledge nodes in the exhibit knowledge graph to determine the primary user intent as "to query the glaze craftsmanship features of the current exhibit."

[0065] It's important to note that a primary user intent is typically composed of multiple primary intent features, each corresponding to a different aspect of that intent. For example, in the primary user intent of "querying the glaze and craftsmanship features of the current exhibit," the primary intent features might include: exhibit category features (e.g., porcelain); query theme features (e.g., glaze); craftsmanship features (e.g., firing techniques); and user focus features (e.g., uniqueness). These primary intent features collectively constitute the primary user intent, enabling the system to more comprehensively understand user needs.

[0066] S302. Determine the target user's second user intent based on the first user intent and contextual dialogue data;

[0067] After determining the primary user intent, the system further integrates contextual dialogue data to infer the target user's secondary user intent. Contextual dialogue data includes the target user's historical voice interaction data, exhibition location, past query content, and correlation information between exhibits. Based on this data, the system can infer the user's potential needs. For example, if a user continuously queries the glaze characteristics of multiple porcelain exhibits after entering the exhibition hall and subsequently asks, "How did this technique develop?", the system will, based on their past query records, infer that their secondary user intent might be "to inquire about the development history of porcelain glaze techniques." Similarly, the secondary user intent also consists of multiple secondary intent features, such as: exhibit category features (e.g., porcelain); query theme features (e.g., glaze); temporal evolution features (e.g., development history); and correlation features (e.g., relationship with previous exhibits). These secondary intent features allow the system to gain a deeper understanding of the user's query context and obtain a more accurate user intent.

[0068] S303. Calculate the semantic similarity between the first user intent and the second user intent;

[0069] To determine the relevance between the first and second user intentions, the system calculates their semantic similarity. This semantic similarity calculation is based on semantic embedding models (such as BERT and GloVe) or semantic matching algorithms using exhibit knowledge graphs. The system maps the first and second user intentions to a high-dimensional semantic space and calculates their cosine similarity. For example, if the first user intention is "to query the glaze color and craftsmanship features of current porcelain," and the second user intention is "to query the development history of porcelain glaze color and craftsmanship," then because both revolve around the same theme: glaze color and craftsmanship, their semantic similarity scores are high. Furthermore, the system also calculates the feature similarity between the features of the first and second intentions. For example, "glaze color" and "glaze color development" have high theme similarity; "porcelain" and "ceramic craftsmanship" have high category similarity; and "specificity" and "craftsmanship evolution" have low correlation. By calculating the similarity between these features, the system can more accurately determine whether the user's query intent is coherent.

[0070] S304. Based on historical voice interaction data, determine the first weight coefficient of each first intent feature in the first user intent and the second weight coefficient of each second intent feature in the second user intent;

[0071] After determining the first user intent and the second user intent, the system, based on the target user's historical voice interaction data, determines a first weight coefficient for each feature of the first user intent and a second weight coefficient for each feature of the second user intent to measure the user's attention to different intent features. Specifically, the determination of the first and second weight coefficients comprehensively considers multiple influencing factors, including the user's historical query frequency, interaction duration, contextual relevance, interaction pattern, and the knowledge relevance of exhibits, to ensure that the system can dynamically adjust the weight of each intent feature, thereby more accurately determining the target user's intent.

[0072] The system dynamically adjusts the weights of intent features by analyzing the target user's historical interaction behavior from multiple dimensions to optimize the accuracy of voice interaction. First, the system counts the frequency of a user's intent feature in historical queries and assigns a weight based on the number of queries, ensuring that features with high-frequency queries receive higher priority. Second, the system analyzes the time users spend on specific exhibits or content; if a user listens to a particular type of content for a longer period, it indicates higher attention, and the weight of the corresponding intent feature will increase. Furthermore, the system calculates the degree of matching between the user's current query and historical query data, assessing contextual relevance. If the current query is highly consistent with past queries, the weight of that feature is increased to enhance the system's contextual understanding capabilities.

[0073] Furthermore, the system analyzes user interaction patterns, including whether users tend to compare different exhibits or focus more on the details of a single exhibit, and adjusts the weights of intent features based on their behavioral habits. For example, if users primarily focus on the comparative relationships between exhibits, the weight of "relevance features" is increased; conversely, if users focus more on the craftsmanship details of exhibits, the weight of "craftsmanship features" is increased. Finally, the system combines exhibit knowledge association graphs to calculate the knowledge association degree between the exhibit currently queried and the exhibits queried in the past, in order to determine whether users tend to explore a specific category of knowledge, thereby further optimizing the weight settings of intent features.

[0074] For example, if a user mentions "glaze color" 15 times in their historical queries and listens for an average of 12 minutes, the weight of "Query Topic Feature (Glaze Color)" is set to 1.0. Simultaneously, if the user is currently in the celadon exhibition area and their past queries match 90% of their previous queries, the weight of this feature remains at 1.0. On the other hand, if a user only queries "the history of porcelain development" 3 times and listens for less than 5 minutes, the weight of "the time evolution feature (development history)" is set to 0.5. Furthermore, if 70% of the user's interactions in their historical queries involve comparing exhibits, the weight of "relevance feature (exhibit relationship)" is set to 0.8, while the weight of "craftsmanship feature (firing technique)" is set to 0.5. Similarly, if a user previously queried "Song Dynasty celadon glaze color craftsmanship," and the knowledge relevance between the current exhibit and the exhibits in their historical queries is 85%, the weight of "Query Topic Feature (Glaze Color)" is set to 1.0, while "development history," due to its relevance of only 45%, is set to 0.5.

[0075] S305. If the semantic similarity is greater than the preset similarity threshold, then according to the first weight coefficient and the second weight coefficient, select multiple target user intent features from multiple first intent features in the first user intent and multiple second intent features in the second user intent to obtain the target user intent.

[0076] After determining the first weight coefficient of each first intent feature in the first user intent and the second weight coefficient of each second intent feature in the second user intent, if the similarity exceeds a preset similarity threshold, it indicates that the two have a high degree of semantic overlap or correlation. The preset similarity threshold is usually determined through experimental optimization and user behavior analysis, with an initial value generally set between 0.75 and 0.80 to ensure correct matching of similar intents while avoiding mis-fusion of irrelevant intents. The system calculates the semantic similarity between intents based on natural language processing (NLP) methods such as BERT cosine similarity, Word2Vec word vector matching, and TF-IDF keyword matching, and optimizes this by incorporating the similarity distribution of a large amount of user query data. When the semantic similarity exceeds the preset threshold, it indicates that the first user intent and the second user intent have a high degree of semantic overlap or correlation. The system will then select multiple target user intent features from multiple first intent features in the first user intent and multiple second intent features in the second user intent based on the first and second weight coefficients to ultimately determine the target user intent.

[0077] In the process of filtering target user intent features, the system employs strategies such as prioritizing the retention of high-weight features, merging similar features, eliminating low-weight features, and combining user historical interaction patterns with the current query context for optimization. First, if the weight coefficient of a certain intent feature is higher than 0.8, it indicates that the feature is highly important in the user's interaction process, and the system will prioritize retaining this feature as a target user intent feature. Second, if there are highly similar intent features between the first and second user intents, such as "glaze color" and "glaze color development," and their weight coefficients are both higher than 0.6, the system will merge them to form a more comprehensive target user intent feature. Furthermore, if the weight coefficient of a certain intent feature is lower than 0.5 and it does not form a significant correlation with other core features, the system will eliminate it to reduce the interference of redundant information on the target user intent. Meanwhile, the system also optimizes based on the user's historical interaction patterns. For example, if a target user repeatedly compares exhibits in their historical queries, the system will appropriately increase the weight of "relevance features (relationship with previous exhibits)" to enhance the consistency of intent. Conversely, if the user is more concerned with the craftsmanship details of a single exhibit, the system will prioritize retaining relevant features such as "craftsmanship features (firing techniques)." Furthermore, the system will adjust based on the user's current query context to ensure that the selected target user intent features match the needs of the current interaction scenario.

[0078] For example, suppose the first user intent is "to query the glaze craftsmanship characteristics of the current porcelain," which includes first intent features such as "exhibit category (porcelain)," "query topic (glaze)," and "craftsmanship features (firing techniques)." The second user intent is "to query the development history of porcelain glaze craftsmanship," which includes second intent features such as "query topic (glaze development)," "time evolution (development history)," and "relationship features (relationship with previous exhibits)." In the calculated weighting coefficients, if the first weighting coefficient for the first intent feature "query topic (glaze)" is 0.9, and the second weighting coefficient for the second intent feature "query topic (glaze development)" is 0.8, the system may merge these two features into the target user intent feature "glaze and its development." Conversely, if the first weighting coefficient for the first intent feature "craftsmanship features (firing techniques)" is 0.6, and the second weighting coefficient for the second intent feature "time evolution (development history)" is 0.65, the system may choose to fuse both, so that the target user intent includes both the firing process of the current exhibit and the historical evolution of glaze craftsmanship. However, if the weight coefficient of the second intent feature, "association feature (relationship with previous exhibits)," is only 0.5, and the user's historical interaction data shows a low interest in comparing exhibits, the system may not include it in the target user intent. Ultimately, the target user intent generated by the system may be: "Introduce the current glaze techniques of porcelain, and analyze its glaze composition, firing process, and the evolution of glaze techniques in different eras in conjunction with historical development trends."

[0079] S306. If the semantic similarity is less than or equal to the preset similarity threshold, then the intent feature is fused based on the first user intent, the second user intent, and historical voice interaction data to obtain the target user intent.

[0080] Specifically, intent topic words and intent operation words are extracted from the first user intent to obtain the first target intent feature; context association features are extracted from the second user intent to obtain the second target intent feature; the third target intent feature of the target user is extracted from historical voice interaction data; the first target intent feature, the second target intent feature, and the third target intent feature are fused to obtain the target user intent.

[0081] After determining that the semantic similarity between the first user intent and the second user intent is less than or equal to a preset similarity threshold, the system employs intent feature fusion, combining the first user intent, the second user intent, and historical voice interaction data to construct a target user intent that better meets user needs. The core of intent feature fusion lies in extracting key features from information from different sources and comprehensively analyzing user interaction habits to ensure that the final generated target user intent is both accurate and consistent.

[0082] First, the system extracts intent keywords and intent action words from the first user intent to form the first target intent feature. Intent keywords are the core concepts in the user's query content, typically used to describe the object the user is interested in, such as "porcelain," "glaze," and "craftsmanship." Intent action words, on the other hand, indicate the actions the user wishes to perform on that object, such as "query," "compare," and "analyze." For example, if the first user intent is "query the glaze features of current porcelain," the system will extract the intent keywords "porcelain" and "glaze," and the intent action word "query," ultimately forming the first target intent feature {porcelain, glaze, query}.

[0083] Next, the system extracts contextual features from the second user intent to form the second target intent features. Contextual features are used to capture the logical relationship between the current user intent and the previous query, ensuring that the target user intent maintains semantic coherence. For example, if the second user intent is "How did this type of glaze evolve?", the system will extract the contextual features "glaze evolution" and "historical development," and combine them with the content of the first user intent to ensure that the target user intent not only answers the current query but also supplements relevant background information. Therefore, the second target intent features might be {glaze evolution, historical development}.

[0084] Then, the system extracts third-target intent features from the user's historical voice interaction data to further optimize the personalized experience of the target user's intent. Historical voice interaction data reflects the user's expressed interests, query habits, and recurring focus on specific topics during long-term system use. For example, if the system detects that the user has repeatedly queryed "color characteristics of celadon" or "firing techniques of Song Dynasty porcelain" in past interaction records, the system may extract {celadon, Song Dynasty, glaze characteristics} as a third-target intent feature from historical data to ensure that the target user's intent aligns with the user's long-term interests. For instance, if the user is currently querying general "glaze" information, but historical data indicates a greater focus on "the glaze color of celadon," the system can prioritize providing celadon-related knowledge to enhance the personalization of the interaction.

[0085] Finally, the system fuses the first, second, and third target intent features to generate the final target user intent. The intent feature fusion process includes optimization strategies such as prioritizing high-weight features, merging similar features, and filtering out low-weight features. For example, in the above case, the first target intent feature {porcelain, glaze, query}, the second target intent feature {glaze evolution, historical development}, and the third target intent feature {celadon, Song Dynasty, glaze characteristics} might be fused into the target user intent: "Introducing the current glaze characteristics of porcelain, and combining historical development trends, analyzing its glaze composition, firing techniques, and the evolution of glaze techniques in different historical periods, focusing on the development characteristics of celadon in the Song Dynasty." This fusion ensures that the target user intent not only covers the core content of the user's current query but also incorporates historical query preferences and contextual information, making the final generated explanation more accurate, coherent, and tailored to the user's personalized needs.

[0086] S206. Determine the content to be explained based on the knowledge association graph of the exhibits and the intent of the target users, and generate the target display scenario based on the content to be explained;

[0087] Specifically, descriptive keywords of the target exhibits are extracted from the explanatory content, and key display areas of the target exhibits are determined based on the descriptive keywords; a camera view sequence of key display areas is generated, which is used to guide the viewing line of the target users; based on the descriptive keywords and the camera view sequence, corresponding animation effects are triggered at the key display areas; and the camera view sequence and animation effects are combined to obtain the target display scene.

[0088] After determining the target user's intent, the system first determines the explanation content based on the exhibit's knowledge graph and the target user's intent. This ensures that the explanation content not only aligns with the exhibit's core knowledge system but also accurately meets the user's query needs. The exhibit's knowledge graph is a structured knowledge network containing multi-dimensional information such as the exhibit's historical background, materials and craftsmanship, artistic style, and cultural value. By analyzing the target user's intent, the system extracts content matching the intent from the knowledge graph and generates personalized explanation text. For example, if the target user's intent is "to understand the glaze characteristics of Song Dynasty celadon," the system will extract the compositional characteristics, firing techniques, and comparative information with celadon from other eras from the knowledge graph to form complete explanation content.

[0089] After generating the explanatory content, the system extracts descriptive keywords for the target exhibits and determines the key display areas based on these keywords. Descriptive keywords typically include the exhibit's core characteristics, such as "celadon" or "Longquan kiln," while key display areas are the parts of the exhibit that visually embody these characteristics. For example, for a Longquan celadon vase, descriptive keywords might include "glaze color and luster," "carved decoration," and "proportion." The system will accordingly determine the glaze surface, carved decoration area, and overall outline of the vase as key display areas to guide subsequent presentations.

[0090] Next, the system generates a camera view sequence based on key display areas to guide the target user's gaze. This camera view sequence refers to a set of preset camera movement paths and angle adjustments, allowing users to observe the key parts of the exhibit from the best possible angle. For example, when introducing the glaze characteristics of Longquan celadon, the system might first use a frontal view to show the overall shape, then slowly zoom in to the mouth area to highlight the glaze's luster and thickness, and then rotate to the side of the bottle to showcase the details of the carving technique. By dynamically adjusting the camera view, the system enhances the user's immersion, enabling them to understand the content more intuitively.

[0091] After determining the camera view sequence, the system triggers corresponding animation effects on key display areas based on descriptive keywords and the camera view sequence to enhance the interactivity and visualization of the exhibits. Animation effects can include local highlighting, zooming, and more. For example, when introducing the luster variations of glaze, the system can add simulated light reflection to the glaze area of ​​the bottle, allowing users to more clearly see the transparency and color gradations of the glaze. These animation effects help users more intuitively understand the craftsmanship details and historical value of the exhibits.

[0092] Finally, the system combines the camera view sequence and animation effects to generate the final target display scene, providing a complete visual presentation. The target display scene is a comprehensive display solution integrating explanatory content, camera movement trajectory, and dynamic interactive effects, enabling users to have an immersive viewing experience when viewing exhibits. For example, when introducing Longquan celadon, the system will guide users to observe the glaze color and carved details of the bottle according to a preset camera view, triggering dynamic highlighting and magnification animations at key areas, allowing users to gain a deeper understanding of the artistic characteristics and craftsmanship value of celadon.

[0093] S207. Select a target virtual tour guide from the preset virtual tour guide library based on the content of the explanation, and display the target display scene;

[0094] Once the content to be explained is determined, the system will select a target virtual guide from a pre-set virtual guide library based on the content and display the target exhibit scene to provide a more immersive and personalized voice interaction experience. The virtual guide library contains various styles of virtual guides, including professional scholars, historical figures, and cartoonish characters, to meet the preferences and needs of different users. The system will intelligently match the most suitable virtual guide based on the theme of the explanation, the historical background of the exhibits, and the personalized preferences of the target user. For example, if the explanation involves the historical evolution of ancient Chinese porcelain, the system might select an "ancient ceramic craftsman" as the virtual guide, dressed in ancient craftsman's attire, who will explain the porcelain-making process and historical value in a first-person perspective.

[0095] After selecting a target virtual guide, the system will simultaneously display the target exhibit scene to ensure that the virtual guide can provide explanations in a suitable environment. The target exhibit scene is generated based on the content to be explained, including key display areas of the exhibits, corresponding camera view sequences, and dynamic interactive effects.

[0096] S208. The target virtual guide explains the target display scene based on the explanation content.

[0097] After showcasing the target scene, the system will use a virtual guide to explain the scene based on the provided content, offering a more vivid and immersive interactive experience. The virtual guide is not only the executor of the explanation but also the guide for the user in the VR environment. Their character, voice style, and explanation method will be intelligently matched according to the content and user preferences. For example, if the explanation involves "the glaze characteristics of Longquan celadon from the Song Dynasty," the system might select an "ancient ceramic craftsman" as the virtual guide to introduce the firing process of Longquan celadon from a first-person perspective. This guide will combine hands-on demonstrations with interactive Q&A to enhance the user's immersion, such as: "Does the glaze of this Longquan celadon feel particularly warm and lustrous to you? This is because we used a low-temperature reducing atmosphere during the firing process, giving the glaze a unique bluish-green hue."

[0098] During the explanation, the virtual guide not only uses voice to describe the historical background, materials, craftsmanship, and artistic value of the exhibits, but also uses dynamic interactive effects within the target display scene to guide the user's gaze, enabling them to more intuitively understand the details of the exhibits. For example, when explaining the crackle texture of porcelain, the virtual guide might point to a specific area on the bottle and trigger a magnifying animation, allowing the user to clearly observe the fine cracks, while explaining, "These tiny cracks are called 'crackles,' and they are formed during the cooling process of the glaze, which has been considered an aesthetic feature of celadon since ancient times." Furthermore, the virtual guide can also simulate the changes in glaze color under different lighting conditions through gestures, dynamic annotations, or 3D animation demonstrations, allowing users to understand the visual characteristics of the exhibits in a more intuitive way.

[0099] Beyond one-way explanations, the system also allows users to interact with the virtual guide in real time, further enhancing the intelligence and personalized experience of voice interaction. For example, when a user asks, "What are the differences between this celadon piece and Ming dynasty celadon?", the virtual guide can dynamically adjust its explanation, match it with a knowledge graph, and invoke a comparison display function in the target display scene. This allows the user to simultaneously observe the differences in glaze color between Song and Ming dynasty celadon. The virtual guide then adds, "The glaze color of Longquan celadon from the Song dynasty is more uniform and delicate, while the glaze color of celadon from the Ming dynasty tends to be more emerald green. This is mainly related to changes in glaze composition and firing temperature..." This interactive explanation not only increases user engagement but also allows each exhibition experience to be personalized based on the user's interests, thereby enhancing the overall immersiveness and intelligence of the VR voice interaction.

[0100] Optional, Figure 2 One of the voice interaction methods shown also includes the following steps:

[0101] Acquire 3D model data of the target exhibit; perform semantic analysis on the explanation content, identify key explanation nodes, and determine the semantic temporal relationship of the key explanation nodes; based on the key explanation nodes, divide the 3D model data into multiple display units; determine the animation effect triggering sequence of each display unit according to the semantic temporal relationship; map the preset explanation speed of the target virtual tour guide with the animation effect triggering sequence to generate interactive temporal control instructions; based on the interactive temporal control instructions, control the display of animation effects in the target display scene.

[0102] During voice interaction, to enhance the immersive and interactive experience of the exhibit explanations, the system first acquires the 3D model data of the target exhibit to ensure that it can be presented in a high-precision visual format within the VR environment. The 3D model data includes the exhibit's geometric structure, material texture, and lighting information, allowing users to observe the exhibit's details from multiple angles. For example, for a Longquan celadon vase, the 3D model will accurately reproduce its body curves, glaze luster, and carved decorations, laying the foundation for subsequent dynamic interaction.

[0103] After acquiring the 3D model data, the system performs semantic analysis on the explanatory content, identifies key explanatory nodes, and determines the semantic temporal relationship between these nodes. Key explanatory nodes are the core information points in the explanatory content, such as the material, manufacturing process, and historical background of the exhibits. The semantic temporal relationship is used to determine the logical order of these information points during the explanation. For example, when introducing Longquan celadon, the system might first explain its historical origins, then its glaze characteristics, and finally analyze the carving technique, ensuring through semantic analysis that the explanation order aligns with the user's understanding habits.

[0104] Based on key explanatory nodes, the system divides the 3D model data into multiple display units, allowing each unit to independently present the exhibit related to specific explanatory content. For example, when introducing the glaze characteristics of porcelain, the system might use the glaze surface of the bottle as one display unit, while when explaining the carving technique, it might allocate a separate display unit for the decorative area of ​​the bottle. This division method ensures that the different features of the exhibits are presented to the user in the best way as the explanation progresses.

[0105] Subsequently, based on semantic temporal relationships, the system determines the triggering sequence of animation effects for each display unit, enabling the dynamic display of key features of the exhibits during the explanation process. For example, when explaining the glaze's luster, the system might trigger light and shadow effects to simulate the reflective properties of the glaze surface; when introducing the carving technique, the system might trigger a local magnification animation, allowing users to observe detailed textures. Through reasonable timing arrangements, the system ensures that the animation effects are synchronized with the explanation content, making the user's viewing experience more natural and smooth.

[0106] To ensure the coordination between audio narration and animation, the system maps the preset speaking speed of the target virtual tour guide to the triggering sequence of animation effects, generating interactive timing control commands. Different virtual tour guides may have different speaking speeds and tones; for example, a scholar-type tour guide may speak slowly and with depth, while a cartoon-type tour guide may speak quickly and with humor. The system adjusts the animation trigger interval according to the selected tour guide's speaking speed, synchronizing the narration with the visual effects. For example, when the virtual tour guide narrates "the changes in glaze luster under different lighting conditions," the animation effect will simultaneously adjust the lighting angle to enhance the continuity of the visual presentation.

[0107] Finally, based on interactive timing control commands, the system controls the display of animation effects in the target exhibition scene, ensuring that users can intuitively understand the characteristics of the exhibits through dynamic visual effects while listening to explanations. For example, when the virtual guide explains "how the transparent glaze of celadon affects its color," the system adjusts the rendering effect of the porcelain in real time, causing changes in the transparency of its glaze, allowing users to intuitively perceive the layering of the glaze color. Through this process, the system achieves precise synchronization between voice explanations and dynamic displays, providing users with a more immersive and interactive exhibition experience in the VR environment.

[0108] Please see Figure 4 This is a schematic diagram of the structure of a voice interaction system provided in an embodiment of this application. The voice interaction system 400 specifically includes: a first acquisition module 401, used to acquire the target user's historical voice interaction data, viewing location, and audio stream at the viewing location; a second acquisition module 402, used to acquire a knowledge association graph of multiple exhibits in the area where the viewing location is located; a first determination module 403, used to determine the target user's voice interaction command in the audio stream; an association module 404, used to associate the voice interaction command with historical voice interaction data based on a preset contextual dialogue model to obtain contextual dialogue data; a second determination module 405, used to determine the target user's intent based on the voice interaction command and contextual dialogue data; a generation module 406, used to determine the explanation content according to the exhibit knowledge association graph and the target user's intent, and generate a target display scene based on the explanation content; a selection module 407, used to select a target virtual guide from a preset virtual guide library based on the explanation content, and display the target display scene; and an explanation module 408, used to explain the target display scene through the target virtual guide based on the explanation content.

[0109] Optionally, the first determining module 403 is specifically used for: acquiring the target user's voice sample data; extracting the user's voice features from the voice sample data and constructing a voiceprint recognition model for the target user based on the voice features; performing noise reduction processing on the audio stream to obtain target audio data; recognizing the target user's speech segments in the target audio based on the voiceprint recognition model; and determining the target user's voice interaction commands based on the speech segments.

[0110] Optionally, the second determining module 405 is specifically used for: performing semantic analysis on the voice interaction command to determine the first user intent of the target user; determining the second user intent of the target user based on the first user intent and contextual dialogue data; and determining the target user intent through the first user intent and the second user intent.

[0111] Optionally, the second determining module 405 is further specifically used for: calculating the semantic similarity between the first user intent and the second user intent; determining the first weight coefficient of each first intent feature in the first user intent and the second weight coefficient of each second intent feature in the second user intent based on historical voice interaction data; if the semantic similarity is greater than a preset similarity threshold, then selecting multiple target user intent features from multiple first intent features and multiple second intent features according to the first weight coefficient and the second weight coefficient to obtain the target user intent; if the semantic similarity is less than or equal to the preset similarity threshold, then performing intent feature fusion based on the first user intent, the second user intent, and historical voice interaction data to obtain the target user intent.

[0112] Optionally, the second determining module 405 is further specifically used to: extract intent topic words and intent operation words from the first user intent to obtain a first target intent feature; extract context association features from the second user intent to obtain a second target intent feature; extract the third target intent feature of the target user from historical voice interaction data; and fuse the first target intent feature, the second target intent feature, and the third target intent feature to obtain the target user intent.

[0113] Optionally, the generation module 406 is specifically used for: extracting descriptive keywords of the target exhibit from the explanatory content, and determining the key display parts of the target exhibit based on the descriptive keywords; generating a camera view sequence of the key display parts, which is used to guide the viewing line of the target users; triggering corresponding animation effects at the key display parts based on the descriptive keywords and the camera view sequence; and combining the camera view sequence and the animation effects to obtain the target display scene.

[0114] Optionally, the system also includes an animation control module 409, specifically used for: acquiring the 3D model data of the target exhibit; performing semantic analysis on the explanation content, identifying key explanation nodes, and determining the semantic temporal relationship of the key explanation nodes; dividing the 3D model data into multiple display units based on the key explanation nodes; determining the animation effect triggering sequence of each display unit according to the semantic temporal relationship; mapping the preset explanation speed of the target virtual tour guide to the animation effect triggering sequence to generate interactive temporal control instructions; and controlling the display of animation effects in the target display scene based on the interactive temporal control instructions.

[0115] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0116] This embodiment also discloses an electronic device, as shown in the reference. Figure 5 The electronic device may include: at least one processor 501, at least one communication bus 502, a user interface 503, a network interface 504, and at least one memory 505. The communication bus 502 is used to enable communication between these components. The user interface 503 may include a display screen or a camera; optionally, the user interface 503 may also include a standard wired interface or a wireless interface. The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0117] The processor 501 may include one or more processing cores. The processor 501 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 505, and by calling data stored in memory 505. Optionally, the processor 501 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 501 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 501 and may be implemented as a separate chip.

[0118] The memory 505 may include random access memory (RAM) or read-only memory. Optionally, the memory 505 may include a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 505 may also be at least one storage device located remotely from the aforementioned processor 501. Figure 5 As shown, the memory 505, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for a voice interaction method.

[0119] exist Figure 5 In the electronic device shown, the user interface 503 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 501 can be used to call an application program that stores a voice interaction method in the memory 505. When executed by one or more processors 501, the electronic device executes one or more methods as described in the above embodiments.

[0120] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not described in detail in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0121] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 505 and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory 505 includes various media capable of storing program code, such as a USB flash drive, external hard drive, magnetic disk, or optical disk.

Claims

1. A voice interaction method, characterized in that, The method, applied to a VR interactive terminal, includes: Acquire the target user's historical voice interaction data, exhibition location, and audio stream at the exhibition location; Obtain an exhibit knowledge association graph among multiple exhibits in the area where the exhibition location is located; Determine the target user's voice interaction commands from the audio stream; Based on a preset contextual dialogue model, the voice interaction commands are associated with the historical voice interaction data to obtain contextual dialogue data. The target user's intent is determined based on the voice interaction commands and the contextual dialogue data. The content to be explained is determined based on the knowledge association graph of the exhibits and the intent of the target user, and a target display scenario is generated based on the content to be explained. Based on the narration content, a target virtual tour guide is selected from a preset virtual tour guide library, and the target display scene is presented. The target virtual guide explains the target display scene based on the explanation content; Determining the target user's intent based on the voice interaction commands and the contextual dialogue data specifically includes: Semantic analysis is performed on the voice interaction commands to determine the target user's first user intent; Determine the target user's second user intent based on the first user intent and the contextual dialogue data; The target user intent is determined by the first user intent and the second user intent; Determining the target user intent through the first user intent and the second user intent specifically includes: Calculate the semantic similarity between the first user intent and the second user intent; Based on the historical voice interaction data, a first weight coefficient for each first intent feature in the first user intent and a second weight coefficient for each second intent feature in the second user intent are determined. If the semantic similarity is greater than a preset similarity threshold, then based on the first weight coefficient and the second weight coefficient, multiple target user intent features are selected from multiple first intent features and multiple second intent features to obtain the target user intent; If the semantic similarity is less than or equal to the preset similarity threshold, then the intent feature is fused based on the first user intent, the second user intent, and the historical voice interaction data to obtain the target user intent; The step of fusing intent features based on the first user intent, the second user intent, and the historical voice interaction data to obtain the target user intent specifically includes: From the first user intent, intent topic words and intent operation words are extracted to obtain the first target intent features; Extract contextual features from the second user intent to obtain the second target intent features; Extract the third target intent feature of the target user from the historical voice interaction data; The first target intent feature, the second target intent feature, and the third target intent feature are fused to obtain the target user intent. The generation of the target display scene based on the explained content specifically includes: Extract descriptive keywords of the target exhibits from the explanation content, and determine the key display parts of the target exhibits based on the descriptive keywords; Generate a sequence of camera viewpoints for the key display areas, which is used to guide the viewing line of the target user. Based on the descriptive keywords and the camera view sequence, corresponding animation effects are triggered at the key display areas; The target display scene is obtained by combining the camera view sequence and the animation effect.

2. The method according to claim 1, characterized in that, Determining the target user's voice interaction command in the audio stream specifically includes: Obtain the voice sample data of the target user; Extract the user's voice features from the sound sample data, and construct a voiceprint recognition model for the target user based on the voice features; The audio stream is subjected to noise reduction processing to obtain the target audio data; Based on the voiceprint recognition model, the voice segments of the target user are identified in the target audio; Based on the audio segment, determine the target user's voice interaction command.

3. The method according to claim 1, characterized in that, The method further includes: Obtain the 3D model data of the target exhibit; Semantic analysis is performed on the content being explained to identify key explanation nodes and determine the semantic temporal relationship of the key explanation nodes. Based on the key explanation nodes, the 3D model data is divided into multiple display units; Based on the semantic temporal relationship, determine the animation effect triggering sequence for each display unit; The preset speaking speed of the target virtual tour guide is mapped to the timing of the animation effect triggering to generate interactive timing control instructions; Based on the interactive timing control instructions, the display of animation effects in the target display scene is controlled.

4. A voice interaction system, characterized in that, include: The first acquisition module is used to acquire the target user's historical voice interaction data, exhibition location, and audio stream at the exhibition location; The second acquisition module is used to acquire an exhibit knowledge association graph among multiple exhibits in the area where the viewing location is located. The first determining module is used to determine the voice interaction command of the target user in the audio stream; The association module is used to associate the voice interaction command with the historical voice interaction data based on a preset context dialogue model to obtain context dialogue data. The second determining module is used to determine the target user's intent based on the voice interaction command and the contextual dialogue data. The generation module is used to determine the explanation content based on the exhibit knowledge association graph and the target user intent, and to generate a target display scene based on the explanation content; The selection module is used to select a target virtual tour guide from a preset virtual tour guide library based on the narration content, and to display the target display scene; The explanation module is used to explain the target display scene based on the explanation content by the target virtual tour guide; The second determining module is specifically used to: perform semantic analysis on the voice interaction command to determine the first user intent of the target user; Determine the target user's second user intent based on the first user intent and the contextual dialogue data; The target user intent is determined by the first user intent and the second user intent; The second determining module is further specifically used for: calculating the semantic similarity between the first user intent and the second user intent; and determining, based on the historical voice interaction data, a first weight coefficient for each first intent feature in the first user intent and a second weight coefficient for each second intent feature in the second user intent; If the semantic similarity is greater than a preset similarity threshold, then based on the first weight coefficient and the second weight coefficient, multiple target user intent features are selected from multiple first intent features and multiple second intent features to obtain the target user intent; If the semantic similarity is less than or equal to the preset similarity threshold, then the intent feature is fused based on the first user intent, the second user intent, and the historical voice interaction data to obtain the target user intent; The second determining module is further specifically used to: extract intent topic words and intent operation words from the first user intent to obtain the first target intent feature; Extract contextual features from the second user intent to obtain the second target intent features; Extract the third target intent feature of the target user from the historical voice interaction data; fuse the first target intent feature, the second target intent feature, and the third target intent feature to obtain the target user intent; The generation module is specifically used for: extracting descriptive keywords of the target exhibit from the explanatory content, and determining the key display parts of the target exhibit based on the descriptive keywords; generating a camera view sequence of the key display parts, the camera view sequence being used to guide the viewing line of the target user; triggering corresponding animation effects at the key display parts based on the descriptive keywords and the camera view sequence; and combining the camera view sequence and the animation effects to obtain the target display scene.

5. An electronic device, characterized in that, include: One or more processors and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1-3.

6. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-3.

Images