Voice interaction method, device and equipment

Abstract

The invention discloses a voice interaction method, device and equipment, and the method comprises the steps: recognizing the voice of a current interaction request, and obtaining the voice of each user; converting the voice of each user into a text; performing semantic analysis on the text corresponding to the voice of each user, extracting keywords, and performing topic analysis on the text corresponding to the voice of each user to obtain topics corresponding to the text; and calling a large language model, generating a personalized reply based on the text, the keyword, the information stored in the two-dimensional storage structure and the information stored in the global structure of the user in the current round of interaction, and converting the generated personalized reply into voice to be output. According to the method, parallel management and response to multiple topics are realized, the problems of semantic loss and context misjudgment are effectively solved, and the practicability and user experience of intelligent equipment in a complex social scene are enhanced.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a voice interaction method, apparatus, and device. Background Technology

[0002] With the continuous advancement of artificial intelligence technology, human-computer interaction has also made great strides, and various voice interaction systems have emerged.

[0003] Existing voice interaction methods have significant shortcomings when dealing with scenarios involving multiple participants simultaneously and cross-topic discussions. These shortcomings are mainly reflected in the following aspects: Weak topic tracking capability: It can only handle linear topic flows (topics of a single user). When faced with concurrent topics (when a single user mentions multiple topics or when multiple users mention one or more topics), it lacks a thread-based management structure, which can easily lead to context loss or cross-misunderstanding.

[0004] The storage and memory structure is simple: existing methods have a relatively flat organizational structure for dialogue history (e.g., simply recording the user's historical dialogues in chronological order).

[0005] Lack of dynamic mapping mechanism: It is impossible to dynamically determine and bind users to the topics they participate in, making it difficult to achieve personalized, multi-turn and continuous semantic understanding and output in multi-person, multi-topic scenarios.

[0006] To address the problems existing in the aforementioned interaction solutions, this application proposes a voice interaction method. This method enables multi-topic thread management and context tracking, improving the ability to understand and respond to complex social contexts in voice interaction. Summary of the Invention

[0007] To address the aforementioned problems, this invention proposes a voice interaction method, device, equipment, and program product, aiming to improve the understanding and response capabilities of smart devices in complex social contexts. The aforementioned smart devices can be digital human terminals, mobile phones, smartwatches, smart home devices, etc.

[0008] The first aspect provides a voice interaction method, including: The voice of each user is recognized in this round of interaction requests; Convert each user's voice into text; Semantic parsing is performed on the text corresponding to each user's voice to extract keywords, and topic analysis is performed on the text corresponding to each user's voice to obtain the topic corresponding to the text; The large language model is invoked to generate personalized responses based on user text, keywords, information stored in the two-dimensional storage structure, and information stored in the global structure in the current round of interaction. The two-dimensional storage structure pre-stores the text content of a single user under a topic in the historical interaction, and the global structure pre-stores the text content of multiple users discussing the same topic. The two-dimensional storage structure and the global structure have a mapping relationship. The generated personalized replies are converted into speech and output.

[0009] In one implementation, before recognizing the speech of the current interaction request, the method further includes: Determine if multiple users are speaking simultaneously; if so, prompt each user to speak one by one.

[0010] In one implementation, the two-dimensional storage structure pre-stores the text content of a single user under a topic from historical interactions, including: Retrieve the text corresponding to the user's voice in historical interactions; Perform topic analysis on the text corresponding to the user's voice to obtain the topic corresponding to the text, and mark the correspondence between the user and the topic; Based on the correspondence between users and topics, the association between the user table and the user topic table is set to obtain a two-dimensional storage structure; Set a content field in the user topic table to store all text content of a user under a topic.

[0011] In one implementation, the user table further includes a user profile field for storing user profiles, including age, preferences, and historical behavior. After semantically parsing the text corresponding to each user's voice and extracting keywords, the method further includes: Update the extracted keywords to the user profile field.

[0012] In one embodiment, the method further includes: Set a "Recent Interaction Rounds" field for the user topic table to record the occurrence of user topics in recent interactions; The activity level of user topics is tracked based on the most recent interaction round field.

[0013] In one implementation, after performing semantic parsing on the text corresponding to each user's voice to extract keywords, and performing topic analysis on the text corresponding to each user's voice to obtain the topic corresponding to the text, the method further includes: Based on the topic analysis, the text corresponding to the topic is used to determine whether multiple users are discussing the same topic in this round of interaction. If so, the same topic discussed by multiple users is taken as a common topic and stored using a global topic structure.

[0014] In one embodiment, the method further includes: Determine if the storage structure for public topics exists. If it does, reuse the existing storage structure for public topics. If it does not exist, create a new storage structure for public topics.

[0015] In one embodiment, the method further includes: Public topics are frozen based on the number of times they have not been interacted with and the time of their most recent interaction.

[0016] In one embodiment, the method further includes: If a frozen topic is mentioned again in a round of interaction, the frozen topic will be unfrozen.

[0017] In one implementation, a large language model is invoked to generate a personalized response based on the user's text, keywords, content stored in the two-dimensional storage structure, and content stored in the global structure during the current interaction. This response includes: If the current interaction involves a user's individual topic, a personalized response will be generated for the individual user based on the user's text, keywords, user profile stored in the user table, and text content under the individual topic stored in the two-dimensional storage structure. If the current round of interaction involves a public topic, a personalized response will be generated for each user based on the user's text, keywords, user profiles stored in the user table, and the corresponding text content of the public topic stored in the global structure.

[0018] In one embodiment, the method further includes: storing the text content of multiple users discussing the same topic using a global topic table, recording the generation time of tokens in the user topic table and the global topic table, determining whether the generation time of the token exceeds a preset time, and freezing the corresponding token if it does. The tokens in the user topic table are keywords and semantic tags extracted from the content of a single user's speech, and the tokens in the global topic table are keywords and semantic tags extracted from the public discussion content of all users on the corresponding public topic.

[0019] A second aspect of the present invention provides a voice interaction device, comprising: The speech recognition module is used to recognize the speech of the current round of interaction requests and obtain the speech of each user; The speech conversion module is used to convert each user's speech into text; The semantic and topic analysis module is used to perform semantic parsing on the text corresponding to each user's voice, extract keywords, perform topic analysis on the text corresponding to each user's voice, obtain the topic corresponding to the text, and mark the correspondence between users and topics. The response generation module is used to generate personalized responses based on the user's text, keywords, information stored in the two-dimensional storage structure, and information stored in the global structure in the current round of interaction. The two-dimensional storage structure pre-stores the text content of a single user under a topic in the historical interaction, and the global structure pre-stores the text content of multiple users discussing the same topic. The two-dimensional storage structure and the global structure have a mapping relationship. The voice output module is used to convert the generated personalized replies into voice output.

[0020] A third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the voice interaction method described in the first aspect.

[0021] A fourth aspect of the present invention provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the voice interaction method described in the first aspect.

[0022] The fifth aspect of the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the voice interaction method described in the first aspect.

[0023] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows: The present invention proposes a voice interaction method that recognizes the voice of the current interaction request to obtain the voice of each user, and can accurately distinguish the user's identity in a multi-person environment.

[0024] Simultaneously, each user's voice is converted into text; semantic parsing is performed on the text corresponding to each user's voice to extract keywords; topic analysis is performed on the text corresponding to each user's voice to obtain the topic corresponding to the text; a two-dimensional storage structure is used to pre-store the text content of a single user under a topic in historical interactions, and a global structure is used to pre-store the text content of multiple users discussing the same topic, realizing parallel management and response to multiple topics.

[0025] When calling large models, user-topic content in the two-dimensional storage structure and public topics in the global structure are used as context tracking, which effectively solves the problems of semantic loss and context misjudgment, and enhances the practicality and user experience of smart devices in complex social scenarios.

[0026] Furthermore, a recent interaction round field is set for the user topic table in the dimension storage structure, and the activity of user topics is tracked based on the recent interaction round field.

[0027] Furthermore, topics discussed by multiple users are designated as public topics. Public topics are frozen based on the number of uninterrupted rounds and the time of the most recent interaction. When a public topic is mentioned again, it is unfrozen.

[0028] Furthermore, a forgetting mechanism is set up for tokens in the user topic table and the global title table. Frozen tokens are not used as token inputs into the large model and do not participate in topic replies, thereby reducing the input of tokens. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart of a voice interaction method in one embodiment of the present invention; Figure 2 This is a flowchart illustrating a method for reusing a public topic storage structure in one embodiment of the present invention; Figure 3 This is a structural diagram of the voice interaction device in an embodiment of the present invention. Detailed Implementation

[0031] The following will further explain the concept, specific structure and technical effects of the present invention in conjunction with the accompanying drawings and embodiments, so as to fully understand the purpose, features and effects of the present invention.

[0032] This application provides a voice interaction method that can be used for interaction between humans and smart devices.

[0033] First, let me explain the technical terms used in this invention: ASR stands for Automatic Speech Recognition.

[0034] Token: Keywords or semantic units extracted from text (such as "weather" or "restaurant"), used for topic categorization and user profile updates.

[0035] Two-dimensional storage structure: A database table with "User ID" and "Topic ID" as dual indexes stores the corresponding data content.

[0036] Topic Freeze: Set the read-only attribute for topics that have not been interacted with for a long time, stop updating but retain history, and save storage and computing resources.

[0037] This embodiment provides a voice interaction method; please refer to [link / reference]. Figure 1 ,include: S100: Recognize the voice of the current round of interaction requests to obtain the voice of each user.

[0038] Specifically, the voice signal in the current round of interaction request can first be collected by a voice acquisition device (such as a multi-microphone array). The voice in the current round of interaction request includes the voice of all users in the current round of interaction request. Then, the collected voice signal is recognized to obtain the voice of each user in the current round of interaction request.

[0039] Voiceprint recognition can distinguish the voices of different users. The recognition process includes: Extract voiceprint features from speech and compare them with voiceprint information in a pre-built user database; If the comparison fails, a user ID is automatically created and its voiceprint characteristics are stored. If the comparison is successful, the user's voiceprint will be segmented into individual user voice based on the user's voiceprint characteristics.

[0040] The pre-built user database includes a two-dimensional storage structure and other user information (such as age, gender, voiceprint, etc.).

[0041] Since a single round of interaction requests may involve multiple users speaking simultaneously, the method further includes the following steps before performing speech recognition on the current round of interaction requests: Determine if multiple users are speaking simultaneously; if so, prompt each user to speak one by one.

[0042] Specifically, if multiple users are speaking at the same time, in order to avoid confusion, users will be prompted to speak one by one in sequence.

[0043] S110: Converts each user's speech into text.

[0044] Specifically, speech recognition is performed on the split single-user speech to convert the speech signal into text content, such as "User 1: How is the weather today?".

[0045] S120: Perform semantic analysis on the text corresponding to each user's voice, extract keywords, and perform topic analysis on the text corresponding to each user's voice to obtain the topic corresponding to the text.

[0046] In practice, semantic parsing can be performed using a large language model API to generate keywords (tokens). Then, a topic analysis API (such as a text classification model based on the LDA algorithm) can be called to identify the topic corresponding to the text and mark the correspondence between them, for example, "User 1 - Topic A".

[0047] Other implementations can use a rule engine based on keyword matching for topic analysis, such as "mentioning 'temperature' is determined to be a 'weather' topic," which is suitable for lightweight scenarios, but its accuracy is lower than that of the API model.

[0048] S130: Call the large language model and generate a personalized response based on the user's text, keywords, information stored in the two-dimensional storage structure, and information stored in the global structure in this round of interaction. The two-dimensional storage structure pre-stores the text content of a single user under a topic in the history of interaction, and the global structure pre-stores the text content of multiple users discussing the same topic.

[0049] Specifically, the two-dimensional storage structure is user-oriented, storing each user's text content under a topic. The global structure is topic-oriented; if multiple users discuss the same topic, then that topic is a public topic, and the global structure is used to store that public topic.

[0050] The text content of a single user under a topic, as well as the text content of multiple users discussing the same topic, constitute the historical chat history. Keywords in the historical chat history (such as "weather" and "bring an umbrella") are extracted as tokens (recorded in the token_list field in the user topic table below) and used to update user profiles (such as adding a "follow travel suggestions" tag), directly affecting the personalized response in subsequent conversations.

[0051] The user topic table includes multiple fields used to store user information, such as user identity, user's chat history, and keywords from the chat history.

[0052] In one implementation, the user table further includes a user profile field for storing user profiles, including age, preferences, and historical behavior; after semantically parsing the text corresponding to each user's voice and extracting keywords, the method further includes: Update the extracted keywords to the user profile field.

[0053] Specifically, user profiles can be updated using keywords (Tokens) in the text corresponding to a user's voice (e.g., if user 1 frequently mentions "weather", then a "follow the weather" tag will be added to their profile).

[0054] The two-dimensional storage structure pre-stores the text content of a single user under a topic in historical interactions, including: Retrieve the text corresponding to the user's voice in historical interactions; Perform topic analysis on the text corresponding to the user's voice to obtain the topic corresponding to the text, and mark the correspondence between the user and the topic; Based on the correspondence between users and topics, the association between the user table and the user topic table is set to obtain a two-dimensional storage structure; Set a content field in the user topic table to store all text content of a user under a topic.

[0055] Specifically, this implementation creates a two-dimensional storage structure (e.g., "User 1 - Topic A" and "User 2 - Topic B") for each "user-topic" combination, storing all the text content of that user under that topic.

[0056] The relationship between the user table and the user topic table can be achieved by setting a mapping between fields in the user table and fields in the user topic table.

[0057] To enable tracking of user topics, the following are also included: Set a "Recent Interaction Rounds" field for the user topic table to record the occurrence of user topics in recent interactions; The activity level of user topics is tracked based on the number of non-interaction rounds.

[0058] Specifically, the larger the value of the "most recent interaction round" field, the more active the topic is.

[0059] For example, one form of a two-dimensional storage structure is as follows: (1) Core data table design The design uses two core tables: a User Profile table and a User-Topic Content table. The relationship between the two tables is achieved through a dual index (User ID + Topic ID). The core fields of the two tables are shown in the table below:

[0060] (2) Example of storage process Let's take a discussion between user A and user B about the topic of "weather" as an example: Initialize the storage structure: If the topic "weather" has not been created, the method of this invention will first generate a unique topic_id (such as weather_20241207) and insert two records into the user_topic table: For example... INSERT INTO user_topic (user_id, topic_id, content, token_list) VALUES ('user_A_id', 'weather_20241207', 'Today's weather is sunny', ['weather', 'sunny']), ('user_B_id', 'weather_20241207', 'Recommendation to bring an umbrella', ['Weather', 'Bring an umbrella']).

[0061] Synchronously update user profiles: Add {"Follow Weather": 1} to User A's portrait; Add {"Follow travel advice": 1} to User B's portrait.

[0062] To store topics discussed by multiple users, this implementation method, after performing semantic parsing on the text corresponding to each user's voice to extract keywords, and performing topic analysis on the text corresponding to each user's voice to obtain the topic corresponding to the text, also includes: Based on the topics identified through topic analysis, it's determined whether multiple users discussed the same topic in this round of interaction. If so, the shared topic is treated as a common topic and stored using a global topic structure. Specifically, if each user discusses their own issue (i.e., the topic is user-independent, such as user 1 discussing "weather" and user 2 discussing "food"), their respective "user-topic" two-dimensional structure is maintained. If multiple users discuss the same topic (e.g., both user 1 and user 2 mention "weather"), this can be determined based on the topic analysis results, and the topic is then considered a common topic.

[0063] In practice, if each user discusses their own topic, the aforementioned two-dimensional storage structure is used to store the topic corresponding to each user.

[0064] In the specific implementation process, the storage of public topics adopts a global structure, and the storage design of the public topic structure is as follows: (1) Design objectives Stores global discussion content for public topics (such as "public topics - weather"), supporting contextual association across users and sessions.

[0065] (2) Data table design A separate global topic table is designed, which is linked to the user topic table via topic_id. The fields of the global topic table are as follows:

[0066] (3) Example of storage process Public topic creation When at least one user first discusses "weather", the method of this invention automatically generates a public topic global_weather and merges all user comments into the public_content field: For example, { "content": [ {"user": "user_A", "text": "The weather is sunny today"}, {"user": "user_B", "text": "It is recommended to bring an umbrella"}, {"user": "user_C", "text": "Weather station issues heavy rain warning"} ], "contributor_list": ["user_A", "user_B", "user_C"] } Merging multi-user content When multiple users discuss the same topic, the system merges the content in chronological order and tags the speaking users (i.e., tags the content corresponding to each user's post): For example, { "content": [ {"user": "user_A", "text": "The weather is sunny today"}, {"user": "user_B", "text": "It is recommended to bring an umbrella"} ], "token_list": ["weather", "sunny", "bring an umbrella"], "last_interaction_round": 2 } Cross-user reuse and updates If user D subsequently inquires about the "weather" separately, the method according to the present invention will first extract the latest public information (such as rainstorm warnings) from global_weather (the global topic table of weather) and merge the response with user D's private topic weather_20241207.

[0067] Furthermore, in order to manage the storage structure of public topics, the method of this implementation also includes: determining whether a storage structure for public topics exists; if it exists, the existing storage structure for public topics is directly reused; if it does not exist, a new storage structure for public topics is created.

[0068] Please see Figure 2 This is a schematic diagram illustrating the reuse method of the storage structure for public topics.

[0069] In the specific implementation process, if the storage structure of public topics (global topic table) already exists, the existing structure is directly reused. Before reusing the structure of public topics, it is determined whether the public topic is frozen. If it is frozen, as long as a user discusses the public topic, the storage structure of the public topic is activated and reused. If it is not frozen, it is reused directly. The specific reuse process includes: determining whether the user ID in the current round of interaction exists in the global topic table. If it exists, only the corresponding fields of the global topic are updated. For example, the public_content (TEXT) field of the global topic table is updated, which is the public discussion content of the new user on the public topic; the last_refresh_time field is updated, which changes the time of the most recent interaction of the public topic to the time of this interaction; and the non_interaction_round field is updated, which sets the number of non-interaction rounds to 0. If it does not exist, in addition to the above fields, the contributor_list (JSON) field is also updated, which updates the ID of the new user to the list of participating users.

[0070] If the storage structure for public topics does not exist, a new storage structure for public topics will be created, as detailed in the aforementioned public topic creation section.

[0071] To freeze public topics, the method also includes: Public topics are frozen based on the number of times they have not been interacted with and the time of their most recent interaction.

[0072] There are two ways to freeze public topics.

[0073] The first method is to determine whether to freeze based on the number of uninterrupted rounds of public topics.

[0074] In the specific implementation process, for the global topic table, a "Non-interaction rounds" field is set to record how many interactions the topic has not appeared. If the topic is not mentioned in the current round of interaction, its value is incremented by 1. If the topic is mentioned in the current round of interaction (it is a topic discussed in the current round of interaction), its value is set to "0".

[0075] Whether to freeze a public topic is determined by the value of the "Non-interaction rounds" field. Specifically, if the value of the "Non-interaction rounds" field for a public topic exceeds a preset threshold N, the "Non-interaction rounds" field is set to read-only, thereby freezing the corresponding topic. Once a topic is frozen, its storage structure is read-only and cannot be updated.

[0076] The second method is to determine whether to freeze the public topic based on the time of the most recent interaction.

[0077] In the specific implementation process, for the global topic table, a "time of the most recent interaction" field is set to record the time of the most recent interaction for that topic.

[0078] The system calculates the inactivity time of a public topic based on the difference between the time of the most recent interaction and the current time. It then checks if this inactivity time exceeds a preset time; if it does, the topic is frozen. For example, if the preset time is set to 30 minutes, the topic is frozen if it hasn't been discussed (interacted) for more than 30 minutes since the last interaction. In other words, the topic is frozen if there are no updates for 30 minutes.

[0079] Once a topic is frozen, its storage structure is read-only and cannot be updated; the `freeze_status` field is set to `true`. Frozen public topics become historical references and do not participate in subsequent reply generation. Furthermore, semantic tags accumulated in frozen topics (such as user A's "following the weather" tag) may be reused in subsequent new topics as a static supplement to user profiles.

[0080] Furthermore, the method also includes: If a frozen topic is mentioned again in a round of interaction, the frozen topic will be unfrozen.

[0081] Specifically, in contrast to the freezing mechanism, if a public topic is mentioned again in an interaction, the corresponding public topic will be unfrozen. At this time, the value of the "Non-interaction round" field will be reset to 0, and the value of the "Time of the most recent interaction" field will be set to the time of the current round of interaction.

[0082] Additionally, setting the `freeze_status` field to `false` will cause public topics in the global topic table to be merged with current interactions as historical reference content after unfreezing.

[0083] The key points of the technical implementation of this invention include: (1) Dual-index association Quickly locate user-private topics using a unique index of user_id + topic_id; By associating global_topic_id with user_topic.topic_id, it supports the linkage between public and private topics.

[0084] (2) Dynamic merging strategy Time sequence merging: Sort by speech timestamp to ensure contextual coherence; User identification labeling: Label the user who made the comment in the content (e.g., "User A: Today's weather...") to avoid confusion.

[0085] (3) Implementation of the freezing mechanism A scheduled task scans the global_topic table and freezes topic structures where last_refresh_time > Time (preset time) and non_interaction_round > threshold (threshold N). (4) Extensibility Design Database sharding and table partitioning: Sharding by user_id or topic_id hash, supporting horizontal scaling; Hot and cold separation: Frequently accessed active topics are stored in high-speed databases (such as Redis), while infrequent topics are archived to cold storage.

[0086] S130 can be implemented in the following way: If the current interaction involves a user's individual topic, a personalized response will be generated for the individual user based on the user's text, keywords, and information stored in the two-dimensional storage structure. If the current interaction involves a public topic, a personalized response will be generated for each user based on the user's text, keywords, information stored in the two-dimensional storage structure, and information stored in the global structure.

[0087] Specifically, if the topic involved in this round of interaction is a user-specific topic, then the text content of the specific topic stored in the two-dimensional storage structure (all the messages and corresponding tokens under the specific topic) will be used as historical chat records and keywords as context for tracking and generating a reply.

[0088] If the current interaction involves a public topic, the context (public topic) of the topic is tracked based on the information stored in the two-dimensional storage structure (user profile) and the text content stored in the global structure (public discussion content of all users on the public topic and the corresponding token), and a reply is generated.

[0089] Specifically, the tokens corresponding to all posts under a single topic can be recorded by setting a token_list field in the corresponding user topic table, while the tokens corresponding to public discussions on public topics can be recorded by setting a token_list field in the corresponding global topic table.

[0090] To further improve interaction efficiency and reduce token input, this invention includes a forgetting mechanism.

[0091] Record the generation time of tokens in the user topic table and the global topic table, and determine whether the generation time of a token exceeds a preset time. If it does, freeze the corresponding token.

[0092] Specifically, if the historical chat tokens in the global topic table and the user topic table exceed a preset time, the corresponding tokens will be frozen (forgotten tokens). That is, frozen tokens will not be used as tokens input into the large model and will not participate in topic replies.

[0093] To further expand the content of the two-dimensional storage structure and the global structure, in one implementation, the method further includes: synchronously storing the response content output by the large model to the corresponding two-dimensional structure or global structure (if it is a public topic).

[0094] S140: Convert the generated personalized response into speech for output.

[0095] Specifically, personalized replies are converted into speech through text-to-speech (TTS) and output sequentially according to user priority (such as speaking order) (e.g., "User 1, the weather is sunny today; User 2, I recommend restaurant XX").

[0096] The key innovations of this invention include: (1) "User-Topic" Two-Dimensional Storage Structure: Create an independent storage unit for each topic of each user to avoid content confusion and support topic reuse across rounds.

[0097] (2) Automatic management of topic lifecycle: Freeze global topics based on the parameters of "number of non-interactions" or "time of the most recent interaction", and unfreeze them when the topic is mentioned again.

[0098] (3) Set up a forgetting mechanism for the tokens in the user topic table and the global title table, record the generation time of the tokens in the user topic table and the global topic table, and determine whether the generation time of the token exceeds the preset time. If it exceeds the preset time, the corresponding token will be frozen. The frozen token will not be used as a token input into the large model and will not participate in topic replies.

[0099] The voice interaction method provided by this invention recognizes the voice of the current interaction request, enabling accurate differentiation of user identities in a multi-person environment; it achieves parallel management and response to multiple topics through a two-dimensional storage structure and a global structure; when calling a large model, it effectively solves the problems of semantic loss and context misjudgment by using the user-topic content in the two-dimensional storage structure and the public topics in the global structure as context tracking, thereby enhancing the practicality and user experience of smart devices in complex social scenarios.

[0100] The voice interaction device provided in the embodiments of this application is described below. The voice interaction device described below can be referred to in correspondence with the voice interaction method described above.

[0101] See Figure 3 , Figure 3 This is a schematic diagram of the structure of a voice interaction device disclosed in an embodiment of this application.

[0102] like Figure 3 As shown, the voice interaction device includes: The speech recognition module 101 is used to recognize the speech of the current round of interaction requests to obtain the speech of each user; The speech conversion module 102 is used to convert each user's speech into text; The semantic and topic analysis module 103 is used to perform semantic parsing on the text corresponding to each user's voice, extract keywords, perform topic analysis on the text corresponding to each user's voice, obtain the topic corresponding to the text, and mark the correspondence between users and topics. The response generation module 104 is used to generate personalized responses based on the user's text, keywords, information stored in the two-dimensional storage structure, and information stored in the global structure in the current round of interaction. The two-dimensional storage structure pre-stores the text content of a single user under a topic in the historical interaction, and the global structure pre-stores the text content of multiple users discussing the same topic. The voice output module 105 is used to convert the generated personalized reply into voice for output.

[0103] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the aforementioned voice interaction method.

[0104] This application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned voice interaction method.

[0105] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned voice interaction method.

[0106] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of this invention and computer equipment including the computer program running the corresponding computer program, should also be within the protection scope of this invention.

[0107] The electronic device provided by the present invention is described below. The electronic device described below and the voice interaction method described above can be referred to in correspondence.

[0108] The electronic device may include a processor, a communications interface, memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor can call logical instructions in the memory to execute voice interaction methods, mainly including the software processing part mentioned above.

[0109] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0110] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0111] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0112] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various modifications and variations to the embodiments of the invention without departing from the spirit and scope of the invention. Thus, if these modifications and variations of the embodiments of the invention fall within the scope of the claims of the invention and their equivalents, the invention also intends to include these modifications and variations.

Claims

1. A voice interaction method, characterized in that, include: The voice of each user is recognized in this round of interaction requests; Convert each user's voice into text; Semantic parsing is performed on the text corresponding to each user's voice to extract keywords, and topic analysis is performed on the text corresponding to each user's voice to obtain the topic corresponding to the text; The large language model is invoked to generate personalized responses based on the user's text, keywords, information stored in the two-dimensional storage structure, and information stored in the global structure in the current round of interaction. The two-dimensional storage structure pre-stores the text content of a single user under a topic in the historical interaction, and the global structure pre-stores the text content of multiple users discussing the same topic. The two-dimensional storage structure and the global structure have a mapping relationship. The generated personalized replies are converted into speech and output.

2. The voice interaction method as described in claim 1, characterized in that, Before recognizing the speech of the current interaction request, the method further includes: Determine if multiple users are speaking simultaneously; if so, prompt each user to speak one by one.

3. The voice interaction method as described in claim 1, characterized in that, The two-dimensional storage structure pre-stores the text content of a single user under a topic from historical interactions, including: Retrieve the text corresponding to the user's voice in historical interactions; Perform topic analysis on the text corresponding to the user's voice to obtain the topic corresponding to the text, and mark the correspondence between the user and the topic; Based on the correspondence between users and topics, the association relationships between the user table and the user topic table, and between the user table and the user topic table are set to obtain a two-dimensional storage structure; Set a content field in the user topic table to store all text content of a user under a topic.

4. The voice interaction method as described in claim 1, characterized in that, The method further includes: Set a "Recent Interaction Rounds" field for the user topic table to record the occurrence of user topics in recent interactions; The activity level of user topics is tracked based on the most recent interaction round field.

5. The voice interaction method as described in claim 1, characterized in that, After performing semantic parsing on the text corresponding to each user's voice to extract keywords, and performing topic analysis on the text corresponding to each user's voice to obtain the topic of the text, the method further includes: Based on the topic analysis, the text corresponding to the topic is used to determine whether multiple users are discussing the same topic in this round of interaction. If so, the same topic discussed by multiple users is taken as a common topic and stored using a global topic structure.

6. The voice interaction method as described in claim 1, characterized in that, The method also includes freezing public topics based on the number of uninterrupted rounds and the time of the most recent interaction.

7. A voice interaction device, characterized in that, include: The speech recognition module is used to recognize the speech in this round of interaction requests and obtain the speech of each user. The speech conversion module is used to convert each user's speech into text; The semantic and topic analysis module is used to perform semantic parsing on the text corresponding to each user's voice, extract keywords, perform topic analysis on the text corresponding to each user's voice, obtain the topic corresponding to the text, and mark the correspondence between users and topics. The response generation module is used to generate personalized responses based on the user's text, keywords, information stored in the two-dimensional storage structure, and information stored in the global structure in the current round of interaction. The two-dimensional storage structure pre-stores the text content of a single user under a topic in the historical interaction, and the global structure pre-stores the text content of multiple users discussing the same topic. The two-dimensional storage structure and the global structure have a mapping relationship. The voice output module is used to convert the generated personalized replies into voice output.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the voice interaction method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the voice interaction method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by the processor, it implements the voice interaction method as described in any one of claims 1 to 6.

Images