Voice interaction method, system and device and storage medium

Abstract

A voice interaction method, system and device and a storage medium, the method comprising: acquiring historical voice interaction data of a target user, an exhibition position and an audio stream at the exhibition position; obtaining an exhibit knowledge association graph among a plurality of exhibit in the area where the exhibition viewing position is located; determining a voice interaction instruction of the target user in the audio stream; based on a preset context dialogue model, associating the voice interaction instruction with historical voice interaction data to obtain context dialogue data; determining a target user intention of the target user based on the voice interaction instruction and the context dialogue data; determining explanation content according to the exhibit knowledge association graph and the target user intention, and generating a target display scene based on the explanation content; selecting a target virtual tour guide from a preset virtual tour guide library based on the explanation content, and displaying a target display scene; and explaining the target display scene based on the explanation content through the target virtual tour guide. According to the invention, the accuracy of voice interaction in VR exhibition can be improved.

Description

Technical Field

[0001] This application relates to the field of voice interaction, and particularly to a voice interaction method, system, device, and storage medium. Background Art

[0002] With the rapid development of Virtual Reality (VR) technology, the application of VR technology in scenarios such as museums and exhibition halls is becoming increasingly widespread. Through a VR interaction terminal, users can obtain an immersive exhibition experience and achieve in-depth interaction with exhibits. In this process, voice interaction, as a natural human-computer interaction method, can effectively improve the user experience and enable users to obtain exhibit information more conveniently.

[0003] In related technologies, the voice interaction solution in a VR exhibition scenario usually uses speech recognition technology to convert the user's voice input into a text instruction, and performs matching and response based on a preset Q&A library. The system will retrieve the 3D model and explanation content of the corresponding exhibit according to the user's location information, and display and explain it to the user through a virtual guide. This solution can achieve basic voice interaction functions and provide exhibit information and explanation services for users.

[0004] However, due to the diversity of users' questioning methods and expression habits in a VR exhibition, and the same question intention may have multiple different expression forms, there may be a large deviation when matching the user's question through a fixed Q&A knowledge base. This results in the system being difficult to accurately understand the user's true intention and affects the accuracy of voice interaction in a VR exhibition. Summary of the Invention

[0005] This application provides a voice interaction method, system, device, and storage medium for enhancing the experience of voice interaction in a VR exhibition.

[0006] In the first aspect of this application, a voice interaction method is provided, which is applied to a VR interaction terminal. The method includes: obtaining the historical voice interaction data, exhibition viewing location, and audio stream at the exhibition viewing location of a target user; obtaining an exhibit knowledge association graph among multiple exhibits in the area where the exhibition viewing location is located; determining a voice interaction instruction of the target user in the audio stream; associating the voice interaction instruction with the historical voice interaction data based on a preset context dialogue model to obtain context dialogue data; determining the target user intention of the target user based on the voice interaction instruction and the context dialogue data; determining an explanation content according to the exhibit knowledge association graph and the target user intention, and generating a target display scene based on the explanation content; selecting a target virtual guide from a preset virtual guide library based on the explanation content, and displaying the target display scene; and explaining the target display scene based on the explanation content through the target virtual guide.

[0007] Optionally, determining a voice interaction instruction of a target user in an audio stream specifically includes: obtaining voice sample data of the target user; extracting the voice features of the user from the voice sample data and constructing a voiceprint recognition model of the target user based on the voice features; performing noise reduction processing on the audio stream to obtain target audio data; recognizing the voice segment of the target user in the target audio based on the voiceprint recognition model; and determining the voice interaction instruction of the target user according to the voice segment.

[0008] Optionally, determining the target user intention of the target user based on the voice interaction instruction and the context dialogue data specifically includes: performing semantic analysis on the voice interaction instruction to determine the first user intention of the target user; determining the second user intention of the target user based on the first user intention and the context dialogue data; and determining the target user intention through the first user intention and the second user intention. <*

[0009] Optionally, determining the target user intention through the first user intention and the second user intention specifically includes: calculating the semantic similarity between the first user intention and the second user intention; determining the first weight coefficient of each first intention feature in the first user intention and the second weight coefficient of each second intention feature in the second user intention based on the historical voice interaction data; if the semantic similarity is greater than a preset similarity threshold, then selecting multiple target user intention features from multiple first intention features and multiple second intention features according to the first weight coefficient and the second weight coefficient to obtain the target user intention; if the semantic similarity is less than or equal to the preset similarity threshold, then performing intention feature fusion based on the first user intention, the second user intention and the historical voice interaction data to obtain the target user intention.

[0010] Optionally, performing intention feature fusion based on the first user intention, the second user intention and the historical voice interaction data to obtain the target user intention specifically includes: extracting an intention topic word and an intention operation word from the first user intention to obtain a first target intention feature; extracting a context association feature from the second user intention to obtain a second target intention feature; extracting a third target intention feature of the target user from the historical voice interaction data; and performing intention feature fusion on the first target intention feature, the second target intention feature and the third target intention feature to obtain the target user intention.

[0011] Optionally, generating a target display scenario based on the explanation content specifically includes: extracting description keywords of a target exhibit from the explanation content and determining the key display parts of the target exhibit based on the description keywords; generating a camera view sequence of the key display parts, and the camera view sequence is used to guide the viewing line of the target user; triggering corresponding animation effects at the key display parts based on the description keywords and the camera view sequence; and combining the camera view sequence and the animation effects to obtain the target display scenario.

[0012] Optionally, a voice interaction method further includes: obtaining three-dimensional model data of a target exhibit; performing semantic analysis on the explanation content to identify key explanation nodes and determine the semantic temporal relationship of the key explanation nodes; dividing the three-dimensional model data into multiple display units based on the key explanation nodes; determining the trigger timing of the animation effect for each display unit according to the semantic temporal relationship; mapping the preset explanation speed of the target virtual tour guide to the trigger timing of the animation effect to generate an interactive timing control instruction; and controlling the display of the animation effect in the target display scene based on the interactive timing control instruction.

[0013] In a second aspect of the present application, a voice interaction system is provided, including: a first acquisition module for acquiring historical voice interaction data of a target user, the viewing position, and the audio stream at the viewing position; a second acquisition module for acquiring an exhibit knowledge association graph among multiple exhibits in the area where the viewing position is located; a first determination module for determining a voice interaction instruction of the target user in the audio stream; an association module for associating the voice interaction instruction with the historical voice interaction data based on a preset context dialogue model to obtain context dialogue data; a second determination module for determining the target user intention of the target user based on the voice interaction instruction and the context dialogue data; a generation module for determining the explanation content according to the exhibit knowledge association graph and the target user intention and generating a target display scene based on the explanation content; a selection module for selecting a target virtual tour guide from a preset virtual tour guide library based on the explanation content and displaying the target display scene; and an explanation module for explaining the target display scene based on the explanation content through the target virtual tour guide.

[0014] In a third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface, and a network interface. The memory is used for storing instructions, and both the user interface and the network interface are used for communicating with other devices. The processor is used for executing the instructions stored in the memory to enable the electronic device to execute the method described in any one of the above.

[0015] In a fourth aspect of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions, and when the instructions are executed, the method described in any one of the above is executed.

[0016] In summary, one or more technical solutions provided by the present application have at least the following technical effects or advantages: 1. By introducing historical interaction data as reference samples, the understanding basis of the system for users' expression methods is expanded; by obtaining the knowledge association graph of exhibits and establishing the logical connections between exhibits, knowledge support is provided for understanding users' intentions; on this basis, the current voice interaction instruction is associated and analyzed with historical voice interaction data, and the context understanding of users' expressions is realized by using the context dialogue model, thereby improving the accuracy of intention recognition; further, based on the accurately recognized user intentions, the system generates matching explanation content and display scenarios in combination with the knowledge association graph of exhibits, and selects a suitable virtual tour guide for explanation, so that the entire interaction process forms a complete closed loop, ensuring the coherence and accuracy from intention understanding to content display. Thus, through the coordination of multiple links such as historical data support, context understanding, knowledge association, and scenario matching, this application improves the accuracy of voice interaction in VR exhibitions.

[0017] 2. By obtaining users' voice samples to construct a voiceprint recognition model and optimizing the audio signal in combination with noise reduction processing, accurate voice command extraction in complex environments is realized. This interaction mechanism based on voiceprint recognition effectively reduces environmental noise interference, enhances the anti-interference ability and accuracy of voice recognition, and solves the problem of difficult differentiation of multiple voices.

[0018] 3. By performing semantic analysis to obtain the first user intention and combining it with the context dialogue model to obtain the second user intention, a two-layer intention understanding framework is established. By calculating the semantic similarity and combining historical data for weighted evaluation of credibility, stable intention judgment is ensured. When the similarity is high, the two intentions are fused; when it is low, multi-dimensional features are extracted for feature fusion to enhance context coherence and reduce single-interaction interference. This mechanism combines instant semantic analysis and historical behavior, improving the accuracy and robustness of intention recognition. Brief Description of the Drawings

[0019] Figure 1 is the system architecture diagram involved in a voice interaction method or a voice interaction system in an embodiment of this application; Figure 2 is the flowchart of a voice interaction method in an embodiment of this application; Figure 3 is another flowchart of a voice interaction method in an embodiment of this application; Figure 4 is the structural diagram of a voice interaction system in an embodiment of this application; Figure 5 is the structural diagram of an electronic device in an embodiment of this application.

[0020] Description of the drawing reference numerals: 401, the first acquisition module; 402, the second acquisition module; 403, the first determination module; 404, the association module; 405, the second determination module; 406, the generation module; 407, the selection module; 408, the explanation module; 409, the animation control module; 501, the processor; 502, the communication bus; 503, the user interface; 504, the network interface; 505, the memory. Detailed implementation manners

[0021] In order to enable those skilled in the art to better understand the technical solutions in this specification, the following will clearly and completely describe the technical solutions in the embodiments of this specification with reference to the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments.

[0022] In the description of the embodiments of this application, words such as "for example" or "for illustration" are used to give examples, illustrations or explanations. Any embodiment or design solution described as "for example" or "for illustration" in the embodiments of this application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, the use of words such as "for example" or "for illustration" is intended to present relevant concepts in a specific manner.

[0023] In the description of the embodiments of this application, the meaning of the term "a plurality" refers to two or more. For example, a plurality of systems refers to two or more systems, and a plurality of screen terminals refers to two or more screen terminals. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the technical features indicated. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. The terms "include", "comprise", "have" and their variants all mean "include but not limited to", unless otherwise specifically emphasized in other ways.

[0024] Figure 1 An exemplary system architecture 100 showing an embodiment of a voice interaction method or a voice interaction system to which this application can be applied is shown.

[0025] As Figure 1 shown, the system architecture 100 may include a VR interaction terminal 101, a network 102, and a server cluster 103. The network 102 is used to provide a medium for a communication link between the VR interaction terminal 101 and the server cluster 103. The network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0026] Users can use the VR interaction terminal 101 to interact with the server cluster 103 through the network 102 to receive or send voice commands, receive virtual tour explanations, etc. Various interaction applications can be installed on the VR interaction terminal 101, such as VR display applications, voice interaction applications, scene rendering applications, etc.

[0027] The VR interaction terminal 101 is a hardware device with VR display functions, including but not limited to VR headsets, controllers, etc. It should also be equipped with a voice collection device (such as a microphone array), a spatial positioning device (for obtaining the user's exhibition viewing position), and an audio playback device (for playing virtual tour explanation content). These devices work together to provide users with a complete VR voice interaction experience.

[0028] The server cluster 103 includes multiple functional servers, which can include: a voice recognition server, responsible for processing the audio stream from the terminal and identifying the user's voice interaction commands; a dialogue management server, responsible for storing and managing the user's historical voice interaction data, processing context conversations, and analyzing the user's intentions; a knowledge graph server, for storing and managing the exhibit knowledge base and processing the association relationships between exhibits; a virtual tour server, for maintaining the virtual tour model library and generating corresponding explanation content according to the user's intentions; a scene generation server, responsible for generating the target display scene according to the user's intentions and explanation content.

[0029] The server cluster 103 can be implemented as a distributed server cluster composed of multiple servers, or as multiple functional modules in a single server. Data exchange and collaborative work are carried out between each server or module through an internal network. The server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers or as a single server. When the server is software, it can be implemented as multiple software or software modules (such as multiple software or software modules used to provide distributed services), or as a single software or software module. No specific limitations are made here.

[0030] It should be noted that to improve the system response speed, the VR interaction terminal can have a local cache function to store common scene data and virtual tour models. At the same time, the server cluster has a real-time data update mechanism to ensure the timely update of exhibit information and knowledge graphs.

[0031] It should be understood that Figure 1 the numbers of VR interaction terminals, networks, and server clusters in are only illustrative. According to implementation requirements, there can be any number of VR interaction terminals, networks, and servers. In some embodiments, some functions can be implemented locally, and in this case, the system architecture may not be completely dependent on network connections.

[0032] Through the collaborative work of the VR interaction terminal and the server cluster, this system architecture realizes the intelligent voice guide function based on the user's historical interaction data, exhibition viewing location, and the association of exhibit knowledge, providing users with an immersive exhibition experience.

[0033] Figure 2 It is a schematic flow diagram of a voice interaction method in an embodiment of this application.

[0034] Please refer to Figure 2 , a voice interaction method in an embodiment of this application, the method includes: S201. Obtain the historical voice interaction data, exhibition viewing location, and audio stream at the exhibition viewing location of the target user; In step S201, the system obtains the historical voice interaction data, exhibition viewing location, and audio stream of the target user. The system records all voice interaction operations of the user in the exhibition hall, and these operation data are stored in the user database of the server. The historical voice interaction data of the user includes information such as the conversation content with the virtual guide, exhibit Q&A records, and interest preferences. The historical voice interaction data can reflect the visiting habits and focus of attention of the target user in different exhibition areas. For example, the user may frequently ask about the age and technological characteristics of the artifacts in the bronze ware exhibition area. For users who use the system for the first time, the server will create a new user profile for data recording.

[0035] The VR interaction terminal obtains the user's exhibition viewing location in real time through the built-in positioning system. The positioning system can use indoor positioning beacons, UWB, or other positioning technologies to establish an accurate position coordinate system in the exhibition hall space. The exhibition viewing location includes not only the specific coordinates and orientation of the target user in the exhibition area, but also the relative position relationship of the surrounding exhibits. This enables the system to accurately know the specific location of the target user in the exhibition hall, such as "2 meters in front of the display cabinet of bronze tripod BC-001 at the southwest corner of the bronze ware exhibition area". The accurate acquisition of location information provides a basis for the system to understand the range of exhibits currently concerned by the user.

[0036] The microphone array equipped on the VR interaction terminal is responsible for collecting the user's real-time voice and converting the voice signal into audio stream data. The system performs the first preprocessing on the collected audio stream, such as noise reduction and signal enhancement, to ensure the quality of the voice data. High-quality audio streams enable the system to accurately recognize the user's voice commands, such as specific questions asked by the user like "What is the casting process of this bronze tripod". The audio stream collection process takes into account the spatial acoustic characteristics of the exhibition hall and adjusts the pickup parameters of the microphone array to adapt to the acoustic environment of different exhibition areas.

[0037] The combination of user location information and historical interaction data can establish the user's behavior trajectory in the exhibition hall, reflecting the user's visiting path and staying patterns. The association between audio stream data and location information ensures that the system correctly understands the exhibits that the user is currently interested in. Historical interaction data provides important context references for understanding the user's voice commands. The comprehensive analysis of these three types of data provides a complete data basis for the system to subsequently perform voice command recognition, user intention understanding, and personalized explanations.

[0038] S202. Obtain the knowledge association graph between multiple exhibits in the area where the viewing location is located; The knowledge association graph of exhibits is stored in the knowledge base of the server. The knowledge association graph of exhibits constructs multi-dimensional association relationships between exhibits through an ontological approach. The knowledge association graph of exhibits contains basic attribute information of exhibits, such as the name, age, material, and use of the exhibits. At the same time, it also contains deep-level associations such as spatio-temporal relationships, cultural inheritance relationships, and technological evolution relationships between exhibits. For example, in the bronze ware exhibition area, the knowledge association graph of exhibits can describe the evolution of the casting technology of bronze wares in different periods, the development context of the patterns on the utensils, and the associated characteristics of bronze wares and other cultural relics of the same period.

[0039] Based on the user's viewing location, the system obtains the knowledge association graph of the exhibits in this area from the knowledge base of the server. When the user is in the bronze ware exhibition area, the system will focus on obtaining the knowledge network related to bronze wares. This network includes not only the knowledge nodes of the exhibits near the user's current location, but also the information of other exhibits associated with these exhibits. For example, when the user is observing a certain Shang Dynasty bronze tripod, the system will extract the characteristic information of the bronze tripod, as well as the information of other bronze wares with similar ages, similar patterns, or related usage scenarios, and construct a knowledge association network centered on this bronze tripod. The dynamic acquisition process of the knowledge association graph of exhibits is closely related to the user's historical interaction data. The system will expand the knowledge graph in a targeted manner according to the user's previously shown interest preferences. If the user has repeatedly asked about the casting technology of bronze wares before, the system will strengthen the knowledge nodes related to technology when constructing the knowledge association graph of exhibits.

[0040] S203. Determine the voice interaction command of the target user in the audio stream; Specifically, obtain the voice sample data of the target user; extract the voice characteristics of the user from the voice sample data, and construct a voiceprint recognition model of the target user based on the voice characteristics; perform noise reduction processing on the audio stream to obtain the target audio data; based on the voiceprint recognition model, identify the voice segment of the target user in the target audio; according to the voice segment, determine the voice interaction command of the target user.

[0041] When a user registers for the first time or initializes the voice interaction function, the system will guide the user to read aloud a series of preset text contents to collect voice sample data of the target user. These preset text contents include not only basic command sentences (such as "turn on the lights"), but also more complex descriptive sentences (such as "please introduce the historical background of this exhibit") to ensure that the system can capture the user's voice characteristics in different contexts. In addition, in order to enhance the system's compatibility with dialects, users need to read aloud the same text content in Mandarin and local dialect respectively to generate cross-language acoustic feature data. During the sound sample collection process, the system uses the microphone array equipped with the VR interactive terminal or smart device to record the user's voice performance at different volumes, speaking speeds, and tones.

[0042] After collecting user voice sample data, the system analyzes it to extract the user's personalized voice features and constructs a voiceprint recognition model for the target user based on these features. Specifically, the system first performs voice frame processing on the user's voice sample data and extracts its acoustic feature parameters, including but not limited to fundamental frequency, pitch, phoneme, formant, and intonation. These acoustic feature parameters comprehensively reflect the user's pronunciation habits and can be used for subsequent identification and voice command parsing. To improve the system's support for dialects, the voiceprint recognition model uses a preset dialect feature mapping model based on the voice sample data. This preset dialect feature mapping model is trained using a large-scale dialect speech database. This database contains voice samples from speakers of various dialects in different scenarios. Each voice sample is annotated with the corresponding dialect type, semantic information, and standard Mandarin equivalents. Using a deep learning algorithm, the system analyzes the acoustic features, speech patterns, and semantic associations in the dialect speech samples to gradually establish a correspondence between the dialect and standard Mandarin, thereby forming the dialect feature mapping model. For example, the system will match the acoustic features of the Sichuan dialect "this what" with the acoustic features of the Mandarin "this what", or map the Cantonese "this what" to the Mandarin "this what".

[0043] When the user issues a voice command in an actual application scenario, the system will first perform noise reduction processing on the received audio stream to remove ambient noise and extract the target audio data. It should be noted that during the audio stream collection stage, the system has already performed a preliminary noise reduction preprocessing to weaken the obvious background noise. However, due to the fact that the actual environment may contain complex noise interferences (such as the conversations of other visitors in the exhibition hall, background music, footsteps, etc.), the system will adopt a more advanced adaptive noise reduction algorithm at this stage to further optimize the quality of the target audio data. For example, the system can utilize beamforming technology combined with the directional sound pickup ability of the microphone array to enhance the speech signal from the user's direction while suppressing the noise from other directions. In addition, the system will also dynamically adjust the noise reduction parameters according to the environment where the user is located to obtain the optimal speech signal in different scenarios. To further enhance the security and user experience of the system, during the noise reduction process, the system will verify again whether the user has authorized the use of their voiceprint data to ensure that only the speech of authorized users is processed and to prevent unauthorized speech data from being misrecognized or stored.

[0044] After completing the noise reduction processing, the system needs to identify the speech segment of the target user from the target audio data. Since the target audio data may contain the speech inputs of multiple users, the system will analyze the target audio data in combination with the user's voiceprint recognition model and accurately locate the speech segment of the target user. Specifically, the system adopts a deep learning-based voiceprint comparison algorithm to match the target audio data with the voiceprint recognition model of the target user to eliminate the speech interference of non-target users.

[0045] After successfully extracting the speech segment of the target user, the system needs to perform speech recognition and natural language processing on it to analyze the user's intention and convert it into the corresponding voice interaction command. Specifically, the system first converts the speech content of the target user into text format and standardizes the dialect expression based on the previously established dialect voiceprint feature mapping model. For example, if the user asks "What's this thing" in Sichuan dialect, the system will convert it into Mandarin "Please introduce this exhibit" and further match it to the system's standard interaction command set to trigger the corresponding operation. In addition, the system can also combine the context information and the user's historical interaction records to intelligently complete and optimize the voice command. For example, if the user says "Change another one" while browsing the exhibit, the system can combine the current interaction state to understand its intention as "Change the exhibit introduction" instead of misinterpreting it as "Change the music".

[0046] S204. Based on the preset context dialogue model, associate the voice interaction command with the historical voice interaction data to obtain the context dialogue data; The system processes the user's voice interaction instructions and historical voice interaction data through a preset context dialogue model. The preset context dialogue model is obtained through deep learning training on a large amount of historical dialogue interaction data. This model includes three core functional modules: dialogue understanding, context association, and semantic representation. The system first performs semantic parsing on the user's current voice interaction instructions, extracting key entities, query attributes, and indicating words in the dialogue. For example, when the user asks "What era is this bronze ware from?", the system parses out the key entity "bronze ware" and the query attribute "era", and at the same time marks "this" as a demonstrative pronoun that needs to be understood in combination with the context.

[0047] The system will retrieve the historical voice interaction data within the preset time period of the user (such as within 30 minutes) to establish the semantic association between the current dialogue and the historical dialogue. In this process, the system focuses on identifying and processing demonstrative pronouns, omitted information, and topic continuation in the dialogue. For example, when the user continues to ask "Why could such exquisite artifacts be made during this period?" after learning about the era of a certain bronze ware, the system will automatically associate "this period" with the specific era mentioned in the previous round of dialogue; when the user simply asks "What does the decoration represent?", the system will automatically supplement the exhibit information mentioned earlier to construct a complete query semantics. When the user's questions extend from the era and craftsmanship of the bronze ware to its cultural connotations, the system identifies the cultural connotations as an extended discussion under the same theme.

[0048] The system converts the processed dialogue information into structured context dialogue data containing the current query intention, historical association information, topic continuation markers, and interest preference characteristics. These data not only record the user's current specific query points but also include the key points of the historical dialogue related to the current dialogue and the user's demonstrated knowledge interest tendencies. This provides an important basis for subsequent user intention analysis, retrieval of the exhibit knowledge association graph, and virtual tour guide explanations. At the same time, the system will update the user's dialogue feature library in real-time. When it is found that the user frequently uses certain specific expressions or shows a continuous interest in a certain type of knowledge, these features will be recorded for continuously optimizing the accuracy of dialogue understanding and content recommendation.

[0049] S205. Determine the target user intention of the target user based on the voice interaction instruction and the context dialogue data; In step S205, the system determines the true intention of the target user through multi-dimensional analysis based on the voice interaction instruction and the context dialogue data. This process is mainly achieved through three key links: semantic analysis, context association, and weight calculation. First, the system conducts an independent semantic analysis of the current voice interaction instruction to obtain a preliminary understanding of the user's intention. Subsequently, it combines the context dialogue data for in-depth semantic association analysis to obtain an understanding of the user's intention considering the historical dialogue background. By calculating the semantic similarity of these two understanding results and combining the user behavior characteristics in the historical interaction data, the system can more accurately grasp the true needs of the user. When the two understanding results have high semantic consistency, the system will fuse the advantageous features of the two understandings through weight allocation. When there are significant differences, a feature fusion mechanism will be activated to comprehensively analyze various information to determine the final user intention. This multi-level intention recognition mechanism ensures that the system can accurately understand the true needs of the user in different dialogue scenarios and provides accurate intention guidance for subsequent personalized explanation services.

[0050] In one possible implementation, referring to Figure 3 , Figure 3 is another flowchart of a voice interaction method in an embodiment of the present application. Determining the target user intention of the target user based on the voice interaction instruction and the context dialogue data may include steps S301 to S304: S301. Conduct a semantic analysis of the voice interaction instruction to determine the first user intention of the target user; After receiving the voice interaction instruction of the target user, the system first conducts a semantic analysis of the instruction to determine the first user intention of the target user. The semantic analysis may include processing steps such as lexical analysis, syntactic analysis, semantic modeling, and intention classification. The system parses the core meaning of the user's voice instruction based on the exhibit knowledge association graph and the pre-trained semantic understanding model (such as BERT). For example, when the user asks "Why is the glaze color of this porcelain so special?" in front of an ancient porcelain exhibit, the system will extract "porcelain" as the exhibit category, "glaze color" as the query topic, and map it to the relevant knowledge nodes in the exhibit knowledge association graph to determine that the first user intention is "query the glaze color process characteristics of the current exhibit".

[0051] It should be noted that the first user intention usually consists of multiple first intention features, and each first intention feature corresponds to a different aspect of this intention. For example, in the first user intention of "query the glaze color process characteristics of the current exhibit", its first intention features may include: exhibit category feature (such as porcelain); query topic feature (such as glaze color); process feature (such as firing technique); user concern feature (such as specialness). These first intention features together constitute the first user intention, enabling the system to more comprehensively understand the user's needs.

[0052] S302. Determine the second user intention of the target user based on the first user intention and the context dialogue data; After determining the first user intention, the system further combines the context dialogue data to infer the second user intention of the target user. The context dialogue data includes the target user's historical voice interaction data, viewing location, past query content, and the association information between exhibits. Based on these data, the system can infer the potential needs of the user. For example, if the user has continuously queried the glaze color characteristics of multiple porcelain exhibits after entering the exhibition hall and then asks "How did this technique develop?", the system will combine its past query records and infer that the second user intention may be "Query the development history of the porcelain glaze color process". Similarly, the second user intention is also composed of multiple second intention features, such as: exhibit category feature (such as porcelain); query theme feature (such as glaze color); time evolution feature (such as development history); relevance feature (such as the relationship with the previous exhibit). These second intention features enable the system to understand the user's query background more deeply and obtain a more accurate user intention.

[0053] S303. Calculate the semantic similarity between the first user intention and the second user intention; To judge the relevance between the first user intention and the second user intention, the system calculates the semantic similarity between them. The semantic similarity calculation is based on a semantic embedding model (such as BERT, GloVe) or a semantic matching algorithm of the exhibit knowledge association graph. The system maps the first user intention and the second user intention into a high-dimensional semantic space and calculates their cosine similarity. For example, if the first user intention is "Query the glaze color process characteristics of the current porcelain" and the second user intention is "Query the development history of the porcelain glaze color process", then since both revolve around the same theme: glaze color process, their semantic similarity score is relatively high. In addition, the system also calculates the feature similarity between the first intention features and the second intention features. For example: the theme similarity between "glaze color" and "glaze color development" is relatively high; the category similarity between "porcelain" and "ceramic process" is relatively high; the relevance between "particularity" and "process evolution" is relatively low. By calculating the similarity between these features, the system can more accurately judge whether the user's query intention is coherent.

[0054] S304. Based on the historical voice interaction data, determine the first weight coefficient of each first intention feature in the first user intention and the second weight coefficient of each second intention feature in the second user intention; After determining the first user intention and the second user intention, the system determines the first weight coefficient of each first intention feature in the first user intention and the second weight coefficient of each second intention feature in the second user intention based on the historical voice interaction data of the target user, so as to measure the user's attention to different intention features. Specifically, the determination of the first weight coefficient and the second weight coefficient comprehensively considers multiple influencing factors, including the user's historical query frequency, interaction duration, context relevance, interaction mode, and the relevance degree of exhibit knowledge, etc., to ensure that the system can dynamically adjust the weights of each intention feature, so as to more accurately determine the target user intention.

[0055] The system dynamically adjusts the weights of intention features by analyzing the historical interaction behaviors of the target user from multiple dimensions to optimize the accuracy of voice interaction. First, the system counts the frequency of a certain intention feature involved in the user's historical queries and sets the weight according to the number of queries to ensure that the features with high-frequency queries get higher weight priorities. Second, the system analyzes the user's staying time on a specific exhibit or content. Assuming that the user has a longer listening duration for a certain type of content, it indicates that their attention is higher, and the weight of the corresponding intention feature will also increase. In addition, the system calculates the matching degree between the user's current query content and the historical query data to evaluate the context relevance. If the current query is highly consistent with the past queries, the weight of this feature will be increased to enhance the system's context understanding ability.

[0056] Furthermore, the system analyzes the user's interaction mode, including whether the user tends to compare different exhibits or is more concerned about the details of a single exhibit, and adjusts the weights of intention features based on their behavior habits. For example, if the user mainly focuses on the comparison relationship between exhibits, the weight of the "correlation feature" will be increased; on the contrary, if the user is more concerned about the technical details of the exhibit, the weight of the "technical feature" will be increased. Finally, the system combines the exhibit knowledge association graph to calculate the knowledge association degree between the user's current query exhibit and the historical query exhibits to determine whether the user tends to explore a specific category of knowledge, thereby further optimizing the weight setting of intention features.

[0057] For example, if the user mentions "glaze color" 15 times in historical queries and the average listening time reaches 12 minutes, the weight of "query topic feature (glaze color)" is set to 1.0. At the same time, if the user is currently in the celadon exhibition area and the matching degree with their past queries reaches 90%, the weight of this feature still remains 1.0. On the other hand, if the user only queries "the development history of porcelain" 3 times and the listening time is less than 5 minutes, the weight of "temporal evolution feature (development history)" is set to 0.5. In addition, if 70% of the interactions in the user's historical queries involve exhibit comparison, the weight of "relevance feature (exhibit relationship)" is set to 0.8, while the weight of "craft feature (firing technique)" is set to 0.5. Similarly, if the user has queried "the glaze color technique of celadon in the Song Dynasty" and the knowledge correlation degree between the current exhibit and the historical query exhibit is 85%, the weight of "query topic feature (glaze color)" is set to 1.0, while "development history" is set to 0.5 due to the correlation degree of only 45%.

[0058] S305. If the semantic similarity is greater than the preset similarity threshold, then based on the first weight coefficient and the second weight coefficient, select multiple target user intention features from multiple first intention features in the first user intention and multiple second intention features in the second user intention to obtain the target user intention; After determining the first weight coefficient of each first intention feature in the first user intention and the second weight coefficient of each second intention feature in the second user intention, if the similarity exceeds the preset similarity threshold, it indicates that there is a high overlap or correlation between the two in terms of semantics. Among them, the preset similarity threshold is usually determined through experimental tuning and user behavior analysis, and the initial value is generally set between 0.75 - 0.80 to ensure that similar intentions can be correctly matched while avoiding the misintegration of irrelevant intentions. The system calculates the semantic similarity between intentions based on natural language processing (NLP) methods such as BERT cosine similarity, Word2Vec word vector matching, and TF-IDF keyword matching, and optimizes it in combination with the similarity distribution of a large amount of user query data. When the semantic similarity exceeds the preset threshold, it indicates that the first user intention and the second user intention have a high overlap or correlation in terms of semantics, and the system will select multiple target user intention features from multiple first intention features in the first user intention and multiple second intention features in the second user intention based on the first weight coefficient and the second weight coefficient to finally determine the target user intention.

[0059] During the screening process of target user intention features, the system adopts strategies such as preferentially retaining high-weight features, merging similar features, screening out low-weight features, combining the user's historical interaction patterns, and the current query context for optimization. First, if the weight coefficient of a certain intention feature is higher than 0.8, it indicates that this feature has high importance in the user's interaction process, and the system will preferentially retain this feature as the target user intention feature. Second, if there are highly similar intention features between the first user intention and the second user intention, such as "glaze color" and "development of glaze color", and their weight coefficients are both higher than 0.6, the system will fuse them to form a more comprehensive target user intention feature. In addition, if the weight coefficient of a certain intention feature is lower than 0.5 and does not form a significant association with other core features, the system will screen it out to reduce the interference of redundant information on the target user intention. At the same time, the system will also optimize in combination with the user's historical interaction patterns. For example, if the target user has conducted multiple comparison queries between exhibits in historical queries, the system will appropriately increase the weight of the "correlation feature (relationship with the previous exhibit)" to enhance the coherence of the intention; conversely, if the user pays more attention to the technical details of a single exhibit, the system will preferentially retain relevant features such as the "technical feature (firing technique)". In addition, the system will also make adjustments in combination with the user's current query context to ensure that the screened target user intention features meet the requirements of the current interaction scenario.

[0060] For example, assume that the first intention in the first user intention is "query the glaze color process characteristics of the current porcelain". The first intention includes first intention features such as "exhibit category (porcelain)", "query topic (glaze color)", and "process characteristics (firing technique)". The second user intention is "query the development history of the porcelain glaze color process", which includes second intention features such as "query topic (glaze color development)", "temporal evolution (development history)", and "associated feature (relationship with the previous exhibit)". Among the calculated weight coefficients, if the first weight coefficient of the first intention feature "query topic (glaze color)" is 0.9, and the second weight coefficient of the second intention feature "query topic (glaze color development)" is 0.8, the system may merge these two features into the target user intention feature "glaze color and its development". At the same time, if the first weight coefficient of the first intention feature "process characteristics (firing technique)" is 0.6, and the second weight coefficient of the second intention feature "temporal evolution (development history)" is 0.65, the system may choose to fuse the two, so that the target user intention includes both the firing process of the current exhibit and the historical evolution of the glaze color process. However, if the weight coefficient of the second intention feature "associated feature (relationship with the previous exhibit)" is only 0.5, and the user's historical interaction data shows that their interest in comparing exhibits is low, the system may not include it in the target user intention. Finally, the target user intention generated by the system may be: "Introduce the glaze color process of the current porcelain, and combine the historical development trend to analyze its glaze composition, firing process, and the evolution of glaze techniques in different eras." S306. If the semantic similarity is less than or equal to the preset similarity threshold, then perform intention feature fusion based on the first user intention, the second user intention, and the historical voice interaction data to obtain the target user intention.

[0061] Specifically, extract the intention topic words and intention operation words from the first user intention to obtain the first target intention feature; extract the context association features from the second user intention to obtain the second target intention feature; extract the third target intention feature of the target user from the historical voice interaction data; perform intention feature fusion on the first target intention feature, the second target intention feature, and the third target intention feature to obtain the target user intention.

[0062] After determining that the semantic similarity between the first user intention and the second user intention is less than or equal to the preset similarity threshold, the system adopts the method of intention feature fusion, combines the first user intention, the second user intention, and the historical voice interaction data to construct a target user intention that better meets the user's needs. The core of intention feature fusion is to extract key features from information from different sources and comprehensively analyze the user's interaction habits to ensure that the finally generated target user intention is both accurate and coherent.

[0063] First, the system extracts the intent topic words and intent operation words from the first user intent to form the first target intent feature. The intent topic words are the core concepts in the user's query content, usually used to describe the objects that the user is concerned about, such as "porcelain", "glaze color", "craft technology", etc., while the intent operation words are used to represent the operations that the user hopes to perform on the object, such as "query", "compare", "analyze", etc. For example, if the first user intent is "query the glaze color characteristics of current porcelain", the system will extract the intent topic words: "porcelain", "glaze color", and the intent operation word: "query", and finally form the first target intent feature {porcelain, glaze color, query}.

[0064] Next, the system extracts the context association features from the second user intent to form the second target intent feature. The context association features are used to capture the logical relationship between the current user intent and the previous query, ensuring that the target user intent can maintain semantic coherence. For example, if the second user intent is "how did this type of glaze color evolve?", the system will extract the context association features: "glaze color evolution", "historical development", and combine the content of the first user intent to ensure that the target user intent can not only answer the current query but also supplement relevant background information. Therefore, the second target intent feature may be {glaze color evolution, historical development}.

[0065] Then, the system extracts the third target intent feature from the user's historical voice interaction data to further optimize the personalized experience of the target user intent. The historical voice interaction data reflects the user's interest preferences, query habits, and specific topics that the user repeatedly focuses on during the long-term use of the system. For example, if the system detects that the user has repeatedly queried "the color characteristics of celadon" or "the firing process of Song Dynasty porcelain" in the past interaction records, the system may extract {celadon, Song Dynasty, glaze color characteristics} from the historical data as the third target intent feature to ensure that the target user intent can conform to the user's long-term interests. For example, when the user's current query is general "glaze color" information, and the historical data shows that the user is more concerned about "the glaze color of celadon", the system can give priority to providing knowledge related to celadon to improve the personalization of the interaction.

[0066] Finally, the system fuses the first target intent feature, the second target intent feature, and the third target intent feature to generate the final target user intent. The process of intent feature fusion includes optimization strategies such as preferentially retaining high-weight features, merging similar features, and screening out low-weight features. For example, in the above case, the first target intent feature {porcelain, glaze color, query}, the second target intent feature {evolution of glaze color, historical development}, and the third target intent feature {celadon, Song Dynasty, glaze color characteristics} may be fused into the target user intent: "Introduce the glaze color characteristics of current porcelain, and combine with the historical development trend to analyze its glaze composition, firing process, and the evolution of glaze techniques in different historical periods, with a focus on introducing the development characteristics of celadon in the Song Dynasty." This fusion ensures that the target user intent not only covers the core content of the user's current query but also combines historical query preferences and context information, making the finally generated explanation content more accurate, coherent, and in line with the user's personalized needs.

[0067] S206. Determine the explanation content according to the exhibit knowledge association graph and the target user intent, and generate a target display scenario based on the explanation content; Specifically, extract the description keywords of the target exhibit from the explanation content, and determine the key display parts of the target exhibit based on the description keywords; generate a camera view sequence of the key display parts, and the camera view sequence is used to guide the viewing line of the target user; trigger corresponding animation effects at the key display parts based on the description keywords and the camera view sequence; combine the camera view sequence and the animation effects to obtain the target display scenario.

[0068] After determining the target user intent, the system first determines the explanation content based on the exhibit knowledge association graph and the target user intent to ensure that the explanation content not only conforms to the core knowledge system of the exhibit but also accurately meets the user's query needs. The exhibit knowledge association graph is a structured knowledge network that contains multi-dimensional information such as the historical background, material technology, artistic style, and cultural value of the exhibit. The system analyzes the target user intent and extracts the content matching the intent from the knowledge graph to generate personalized explanation text. For example, if the target user intent is "to understand the glaze color characteristics of celadon in the Song Dynasty", the system will extract the composition characteristics, firing process of celadon in the Song Dynasty, and comparison information with celadon in other eras from the knowledge graph to form a complete explanation content.

[0069] After generating the explanatory content, the system extracts the descriptive keywords of the target exhibit from the explanatory content and determines the key display parts of the exhibit based on these keywords. Descriptive keywords usually include the core features of the exhibit, such as "celadon" and "Longquan Kiln". The key display parts are the parts of the exhibit that can visually reflect these features. For example, for a Longquan Kiln celadon vase, the descriptive keywords may include "glaze luster", "engraving technique", "shape proportion", etc. The system will accordingly determine the glaze surface, the engraved decoration area, and the overall outline of the vase as the key display parts for subsequent display guidance.

[0070] Next, the system generates a camera view sequence based on the key display parts to guide the target user's viewing line of sight. The camera view sequence refers to a set of preset camera movement paths and angle adjustment schemes, enabling the user to observe the key parts of the exhibit from the best perspective. For example, when introducing the glaze color characteristics of Longquan Kiln celadon, the system may first use a front view to show the overall shape, then slowly zoom in to the bottle mouth area to highlight the luster and thickness of the glaze, and then rotate to the side of the bottle to show the details of the engraving technique. By dynamically adjusting the camera view, the system can enhance the user's immersion and enable them to more intuitively understand the explanatory content.

[0071] After determining the camera view sequence, the system triggers corresponding animation effects at the key display parts based on the descriptive keywords and the camera view sequence to enhance the interactivity and visualization of the display. The types of animation effects can include local highlighting, zooming in and out, etc. For example, when introducing the change in glaze luster, the system can add light reflection simulation in the glaze area of the bottle body, enabling the user to more clearly see the transparency and color layers of the glaze layer. These animation effects can help the user more intuitively understand the technical details and historical value of the exhibit.

[0072] Finally, the system combines the camera view sequence and the animation effects to generate the final target display scene to provide a complete visual presentation. The target display scene is a comprehensive display solution integrating the explanatory content, the camera movement trajectory, and the dynamic interaction effects, enabling the user to obtain an immersive viewing experience when observing the exhibit. For example, when introducing Longquan Kiln celadon, the system will gradually guide the user to observe the glaze color and engraving details of the bottle body according to the preset camera view, and trigger dynamic highlighting and zooming animations at the key parts, enabling the user to more deeply understand the artistic features and technical value of the celadon.

[0073] S207. Select the target virtual tour guide from the preset virtual tour guide library based on the explanatory content and display the target display scene; After determining the content of the explanation, the system will select a target virtual tour guide from the preset virtual tour guide library based on the content of the explanation and display the target display scene to provide a more immersive and personalized voice interaction experience. The virtual tour guide library has preset virtual tour guide images of various styles, including professional scholars, historical figures, cartoons, etc., to meet the preferences and needs of different users. The system will intelligently match the most suitable virtual tour guide based on the theme of the explanation content, the historical background of the exhibits, and the personalized preferences of the target user. For example, if the explanation content involves the historical evolution of ancient Chinese porcelain, the system may select an "ancient ceramic craftsman" as a virtual tour guide, who will wear ancient craftsman clothing and explain the production process and historical value of porcelain in the first person.

[0074] After selecting the target virtual tour guide, the system will simultaneously display the target exhibition scene to ensure that the virtual tour guide can provide the explanation in the appropriate environment. The target exhibition scene is generated based on the explanation content, including the key display parts of the exhibits, the corresponding camera angle sequence, and dynamic interactive effects.

[0075] S208: Explain the target display scene based on the explanation content through the target virtual tour guide.

[0076] After displaying the target display scene, the system will use the target virtual tour guide to explain the target display scene based on the explanation content to provide a more vivid and immersive interactive experience. The virtual tour guide is not only the executor of the explanation, but also the guide of the user in the VR environment. Its character image, voice style and explanation method will be intelligently matched according to the explanation content and user preferences. For example, if the explanation content involves "the glaze color characteristics of Longquan kiln celadon in the Song Dynasty", the system may select an "ancient ceramic craftsman" as a virtual tour guide to introduce the firing process of Longquan kiln celadon to the user from a first-person perspective, and combine the actual operation process with interactive Q&A to enhance the user's immersion, such as: "Does the glaze color of this Longquan kiln celadon make you feel particularly warm? This is because we used a low-temperature reducing atmosphere during the firing process, which makes the glaze surface present a unique blue-green color."

[0077] During the explanation process, the virtual tour guide will not only use voice to describe the historical background, material technology, and artistic value of the exhibits, but also combine the dynamic interaction effects in the target display scene to guide the user's viewing line of sight, enabling them to more intuitively understand the details of the exhibits. For example, when explaining the crackled texture of porcelain, the virtual tour guide may point to a specific area of the bottle body and trigger a magnifying animation, allowing the user to clearly observe the fine cracks of the crackle, while explaining: "These tiny cracks are called 'crackle'. They are formed during the cooling process of the glaze layer and have been regarded as an aesthetic feature of celadon since ancient times." In addition, the virtual tour guide can also simulate the color change effect of the glaze under different lights through gestures, dynamic annotations, or 3D animation demonstrations, enabling the user to understand the visual characteristics of the exhibits in a more intuitive way.

[0078] In addition to one-way explanations, the system also allows users to interact with the virtual tour guide in real time, further enhancing the intelligence and personalized experience of voice interaction. For example, when the user asks, "What's the difference between this celadon and the celadon of the Ming Dynasty?", the virtual tour guide can dynamically adjust the explanation content, match it with the knowledge association graph, and call the comparison display function in the target display scene, enabling the user to simultaneously observe the glaze color differences between the Song Dynasty and Ming Dynasty celadons. At the same time, the virtual tour guide supplements the explanation: "The glaze color of the celadon of the Longquan Kiln in the Song Dynasty is more uniform and delicate, while the glaze color of the celadon in the Ming Dynasty tends to be emerald green. This is mainly related to the changes in the glaze composition and firing temperature..." This interactive explanation not only improves the user's sense of participation but also enables each viewing experience to be personalized according to the user's interests, thereby enhancing the immersion and intelligence level of the overall VR voice interaction.

[0079] Optionally, Figure 2 A voice interaction method shown also includes the following steps: Obtain the three-dimensional model data of the target exhibit; perform semantic analysis on the explanation content, identify the key explanation nodes, and determine the semantic temporal relationship of the key explanation nodes; divide the three-dimensional model data into multiple display units based on the key explanation nodes; determine the triggering timing of the animation effects for each display unit according to the semantic temporal relationship; map the preset explanation speed of the target virtual tour guide to the triggering timing of the animation effects to generate an interactive timing control instruction; control the display of the animation effects in the target display scene based on the interactive timing control instruction.

[0080] Among them, during the voice interaction process, in order to enhance the immersion and interactivity of the exhibit explanation, the system first obtains the 3D model data of the target exhibit to ensure that the exhibit can be presented in a high-precision visualization form in the VR environment. The 3D model data includes the geometric structure, material texture, and lighting information of the exhibit, enabling users to observe the details of the exhibit from multiple angles. For example, for a Longquan Kiln celadon vase, the 3D model will accurately reproduce features such as the curve of the vase body, the luster of the glaze color, and the carved decoration, laying a foundation for subsequent dynamic interactions.

[0081] After obtaining the 3D model data, the system performs semantic analysis on the explanation content, identifies key explanation nodes, and determines the semantic temporal relationship of the key explanation nodes. Key explanation nodes are the core information points in the explanation content, such as the material, production process, and historical background of the exhibit, while the semantic temporal relationship is used to determine the logical order of these information points during the explanation process. For example, when introducing Longquan Kiln celadon, the system may first explain its historical origin, then introduce the characteristics of its glaze color, and finally analyze the carved decoration process, and ensure that the explanation order conforms to the user's understanding habits through semantic analysis.

[0082] Based on the key explanation nodes, the system divides the 3D model data into multiple display units, enabling each unit to independently present the part of the exhibit related to specific explanation content. For example, when introducing the glaze color characteristics of porcelain, the system may take the glaze surface of the vase body as a display unit, and when explaining the carved decoration process, it separately divides the decorative area of the vase body into another display unit. This division method ensures that different characteristics of the exhibit can be gradually presented to users in the best way as the explanation content progresses.

[0083] Subsequently, the system determines the animation effect trigger timing for each display unit according to the semantic temporal relationship, so as to dynamically display the key features of the exhibit during the explanation process. For example, when explaining the luster of the glaze color, the system may trigger a light and shadow change effect to simulate the reflection characteristics of the glaze surface; when introducing the carved decoration process, the system may trigger a local magnification animation, enabling users to observe the detailed texture. Through reasonable timing arrangement, the system ensures that the animation effect can be synchronized with the explanation content, making the user's exhibition experience more natural and smooth.

[0084] To ensure the coordination between the voice explanation and the animation display, the system maps the preset explanation speed of the target virtual tour guide to the animation effect trigger timing to generate an interactive timing control instruction. Different virtual tour guides may have different speaking speeds and intonations. For example, a scholar-type tour guide may have a slower speaking speed and be more profound, while a cartoon-type tour guide may have a faster speaking speed and be more interesting. The system will adjust the animation trigger interval according to the speaking speed of the selected tour guide, making the explanation content synchronized with the visual effect. For example, when the virtual tour guide describes "the changes in the glaze color under different lights", the animation effect will simultaneously adjust the lighting angle to enhance the coherence of the visual presentation.

[0085] Finally, based on the interactive timing control instruction, the system controls the display of the animation effect in the target display scene, ensuring that while listening to the explanation, the user can intuitively understand the characteristics of the exhibits through the dynamic visual effect. For example, when the virtual tour guide explains "how the transparent glaze layer of celadon affects the color", the system will adjust the rendering effect of the porcelain in real time, changing the transparency of its glaze surface, allowing the user to intuitively feel the layering of the glaze color. Through this process, the system realizes the precise synchronization of voice explanation and dynamic display, enabling the user to obtain a more immersive and interactive exhibition experience in the VR environment.

[0086] Please refer to Figure 4 FIG. 4, which is a schematic structural diagram of a voice interaction system provided by an embodiment of the present application. A voice interaction system 400 specifically includes: a first acquisition module 401, configured to acquire the historical voice interaction data, viewing position, and audio stream at the viewing position of the target user; a second acquisition module 402, configured to acquire the exhibit knowledge association graph between multiple exhibits in the area where the viewing position is located; a first determination module 403, configured to determine the voice interaction instruction of the target user in the audio stream; an association module 404, configured to associate the voice interaction instruction with the historical voice interaction data based on a preset context dialogue model to obtain context dialogue data; a second determination module 405, configured to determine the target user intention of the target user based on the voice interaction instruction and the context dialogue data; a generation module 406, configured to determine the explanation content according to the exhibit knowledge association graph and the target user intention, and generate a target display scene based on the explanation content; a selection module 407, configured to select a target virtual tour guide from a preset virtual tour guide library based on the explanation content and display the target display scene; an explanation module 408, configured to explain the target display scene through the target virtual tour guide based on the explanation content.

[0087] Optionally, the first determination module 403 is specifically configured to: acquire the voice sample data of the target user; extract the voice features of the user from the voice sample data, and construct a voiceprint recognition model of the target user based on the voice features; perform noise reduction processing on the audio stream to obtain target audio data; identify the voice segment of the target user in the target audio based on the voiceprint recognition model; and determine the voice interaction instruction of the target user according to the voice segment.

[0088] Optionally, the second determination module 405 is specifically configured to: perform semantic analysis on the voice interaction instruction to determine the first user intention of the target user; determine the second user intention of the target user based on the first user intention and the context dialogue data; and determine the target user intention through the first user intention and the second user intention.

[0089] Optionally, the second determination module 405 is further specifically configured to: calculate the semantic similarity between the first user intention and the second user intention; determine, based on historical voice interaction data, a first weight coefficient of each first intention feature in the first user intention and a second weight coefficient of each second intention feature in the second user intention; if the semantic similarity is greater than a preset similarity threshold, select multiple target user intention features from the multiple first intention features and the multiple second intention features according to the first weight coefficient and the second weight coefficient to obtain a target user intention; if the semantic similarity is less than or equal to the preset similarity threshold, perform intention feature fusion based on the first user intention, the second user intention, and the historical voice interaction data to obtain a target user intention.

[0090] Optionally, the second determination module 405 is further specifically configured to: extract an intention topic word and an intention operation word from the first user intention to obtain a first target intention feature; extract a context association feature from the second user intention to obtain a second target intention feature; extract a third target intention feature of the target user from the historical voice interaction data; perform intention feature fusion on the first target intention feature, the second target intention feature, and the third target intention feature to obtain a target user intention.

[0091] Optionally, the generation module 406 is specifically configured to: extract a description keyword of the target exhibit from the explanation content, and determine a key display part of the target exhibit based on the description keyword; generate a camera view sequence of the key display part, where the camera view sequence is used to guide the viewing line of the target user; trigger a corresponding animation effect at the key display part based on the description keyword and the camera view sequence; and combine the camera view sequence and the animation effect to obtain a target display scene.

[0092] Optionally, the system further includes an animation control module 409, which is specifically configured to: obtain three-dimensional model data of the target exhibit; perform semantic analysis on the explanation content, identify key explanation nodes, and determine the semantic temporal relationship of the key explanation nodes; divide the three-dimensional model data into multiple display units based on the key explanation nodes; determine the animation effect trigger timing of each display unit according to the semantic temporal relationship; map the preset explanation speech rate of the target virtual guide to the animation effect trigger timing to generate an interactive timing control instruction; and control the display of the animation effect in the target display scene based on the interactive timing control instruction.

[0093] It should be noted that: when the device provided in the above embodiment implements its functions, only the above division of each functional module is used for illustration. In actual application, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process can be found in the method embodiment, which will not be repeated here.

[0094] This embodiment also discloses an electronic device. Referring to Figure 5 , the electronic device may include: at least one processor 501, at least one communication bus 502, a user interface 503, a network interface 504, and at least one memory 505. Among them, the communication bus 502 is used to realize the connection and communication between these components. Among them, the user interface 503 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 503 may further include a standard wired interface and a wireless interface. Among them, the network interface 504 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0095] Among them, the processor 501 may include one or more processing cores. The processor 501 connects various parts within the entire server through various interfaces and lines, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 505, and by calling the data stored in the memory 505. Optionally, the processor 501 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 501 may integrate one or a combination of several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, the user interface, and application programs, etc.; the GPU is responsible for the rendering and drawing of the content to be displayed on the display screen; the modem is used to process wireless communication. It can be understood that the above modem may not be integrated into the processor 501 and may be implemented separately by a single chip.

[0096] Among them, the memory 505 may include a Random Access Memory (RAM), or may include a Read-Only Memory. Optionally, the memory 505 includes a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 505 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store data involved in the above-mentioned method embodiments. Optionally, the memory 505 may also be at least one storage device located far from the aforementioned processor 501. As Figure 5 shown, the memory 505, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for a voice interaction method.

[0097] In Figure 5 the electronic device shown, the user interface 503 is mainly used to provide an input interface for the user to obtain user input data; while the processor 501 can be used to call the application program for a voice interaction method stored in the memory 505. When executed by one or more processors 501, the electronic device executes the method of one or more of the above embodiments.

[0098] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. In addition, in each embodiment of the present application, the functional units may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0099] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 505 and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned memory 505 includes various media that can store program codes, such as USB flash drives, mobile hard disks, magnetic disks, or optical discs.

Claims

1. A voice interaction method, characterized in that, Applied to a VR interaction terminal, the method includes: Obtain the historical voice interaction data, exhibition viewing location, and audio stream at the exhibition viewing location of the target user; Obtain the exhibition knowledge association graph among multiple exhibits in the area where the exhibition viewing location is located; Determine the voice interaction instruction of the target user in the audio stream; Based on a preset context dialogue model, associate the voice interaction instruction with the historical voice interaction data to obtain context dialogue data; Determine the target user intention of the target user based on the voice interaction instruction and the context dialogue data; Determine the explanation content according to the exhibition knowledge association graph and the target user intention, and generate a target display scene based on the explanation content; Select a target virtual tour guide from a preset virtual tour guide library based on the explanation content, and display the target display scene; Have the target virtual tour guide explain the target display scene based on the explanation content.

2. The method according to claim 1, wherein The determining the voice interaction instruction of the target user in the audio stream specifically includes: Obtain the voice sample data of the target user; Extract the voice features of the user from the voice sample data, and construct a voiceprint recognition model of the target user based on the voice features; Perform noise reduction processing on the audio stream to obtain target audio data; Based on the voiceprint recognition model, identify the voice segment of the target user in the target audio; Determine the voice interaction instruction of the target user according to the voice segment.

3. The method according to claim 1, characterized in that, The determining the target user intention of the target user based on the voice interaction instruction and the context dialogue data specifically includes: Perform semantic analysis on the voice interaction instruction to determine the first user intention of the target user; Determine the second user intention of the target user based on the first user intention and the context dialogue data; Determine the target user intention through the first user intention and the second user intention.

4. The method according to claim 3, characterized in that, The determining the target user intention through the first user intention and the second user intention specifically includes: Calculate the semantic similarity between the first user intention and the second user intention; Based on the historical voice interaction data, determine the first weight coefficient of each first intention feature in the first user intention and the second weight coefficient of each second intention feature in the second user intention; If the semantic similarity is greater than a preset similarity threshold, then according to the first weight coefficient and the second weight coefficient, select multiple target user intention features from multiple first intention features and multiple second intention features to obtain the target user intention; If the semantic similarity is less than or equal to the preset similarity threshold, then perform intention feature fusion based on the first user intention, the second user intention, and the historical voice interaction data to obtain the target user intention.

5. The method according to claim 4, characterized in that, The performing intention feature fusion based on the first user intention, the second user intention, and the historical voice interaction data to obtain the target user intention specifically includes: Extract an intention theme word and an intention operation word from the first user intention to obtain a first target intention feature; Extract context - related features from the second user intention to obtain second target intention features; Extract third target intention features of the target user from the historical voice interaction data; Fuse the first target intention features, the second target intention features, and the third target intention features to obtain the target user intention.

6. The method according to claim 1, wherein The generating the target display scene based on the explanation content specifically includes: Extract description keywords of the target exhibit from the explanation content, and determine the key display parts of the target exhibit based on the description keywords; Generate a camera view sequence for the key display parts, and the camera view sequence is used to guide the viewing line of the target user; Based on the description keywords and the camera view sequence, trigger corresponding animation effects at the key display parts; Combine the camera view sequence and the animation effects to obtain the target display scene.

7. The method according to claim 1, wherein The method further includes: Obtain three - dimensional model data of the target exhibit; Perform semantic analysis on the explanation content, identify key explanation nodes, and determine the semantic temporal relationship of the key explanation nodes; Based on the key explanation nodes, divide the three - dimensional model data into multiple display units; According to the semantic temporal relationship, determine the animation effect trigger timing of each display unit; Map the preset explanation speech rate of the target virtual tour guide to the animation effect trigger timing to generate an interactive timing control instruction; Based on the interactive timing control instruction, control the display of the animation effects in the target display scene.

8. A voice interaction system, characterized in that, It includes: A first acquisition module, configured to acquire historical voice interaction data of a target user, the viewing position, and the audio stream at the viewing position; A second acquisition module, configured to acquire an exhibit knowledge association graph among multiple exhibits in the area where the viewing position is located; A first determination module, configured to determine the voice interaction instruction of the target user in the audio stream; An association module, configured to associate the voice interaction instruction with the historical voice interaction data based on a preset context dialogue model to obtain context dialogue data; A second determination module, configured to determine the target user intention of the target user based on the voice interaction instruction and the context dialogue data; A generation module, configured to determine the explanation content according to the exhibit knowledge association graph and the target user intention, and generate a target display scene based on the explanation content; A selection module, configured to select a target virtual tour guide from a preset virtual tour guide library based on the explanation content, and display the target display scene; An explanation module, configured to explain the target display scene through the target virtual tour guide based on the explanation content.

9. An electronic device, characterized in that, It includes: One or more processors and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, and the computer program code includes computer instructions. The one or more processors call the computer instructions to enable the electronic device to execute the method according to any one of claims 1 - 7.

10. A computer-readable storage medium, comprising instructions, characterized in that, When the instruction runs on the electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 7.

Images