Voice interaction method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of smart televisions, in particular to a voice interaction method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining an emotion feature vector through obtaining the interaction voice of a user, inputting the emotion feature vector into a preset neural network model for classification, obtaining the current emotion state of the user, and determining the historical preference of the user. And determining recommended content of the television equipment according to the historical preference and the current emotional state, determining a response strategy according to the current emotional state, and outputting the recommended content according to the response strategy. The current emotional state of the user is obtained by extracting and classifying emotional feature vectors in interactive voice of the user, historical preferences of the user are determined according to historical behavior data and the interactive voice, recommended content is determined according to the historical preferences and the current emotional state, and the recommended content is output in combination with the current emotional state. Therefore, emotion recognition is comprehensively performed on the interactive voice of the user at the television terminal so as to accurately feed back the interactive voice.

Description

Technical Field

[0001] The present invention relates to the field of smart televisions, and in particular to a voice interaction method, device, computer equipment, and storage medium. Background Art

[0002] With the widespread adoption of smart TVs, users can now use voice interaction with their devices to perform basic control commands (such as changing channels and adjusting volume) or simple Q&A. However, these systems often lack a deep understanding of users' emotional states and personalized needs, resulting in mechanical and rigid interactions and a struggle to provide truly intelligent services.

[0003] Currently, most TV devices rely on affective computing to identify users' emotional states during user interaction. For example, by analyzing the spectral characteristics of speech signals (such as Mel-Frequency Cepstral Coefficients (MFCCs)), a preliminary assessment of the user's emotional state can be made. However, this analysis is relatively one-sided and fails to fully reflect complex emotional states, resulting in a lack of personalized content recommendations.

[0004] Therefore, how to comprehensively perform emotion recognition on the user's interactive voice on the TV terminal in order to accurately provide feedback on the interactive voice has become an urgent problem to be solved. Summary of the Invention

[0005] The embodiments of the present invention provide a voice interaction method, apparatus, computer equipment and storage medium to solve the problem of how to comprehensively perform emotion recognition on the user's interactive voice on a television terminal to accurately provide feedback on the interactive voice.

[0006] In a first aspect, an embodiment of the present invention provides a voice interaction method, which is applied to a television terminal and includes: Acquire the user's interactive speech, extract the spectrum features and speech time sequence features of the interactive speech, and fuse the spectrum features and the speech time sequence features to obtain an emotion feature vector; Inputting the emotional feature vector into a preset neural network model for classification to obtain the user's current emotional state; Acquiring historical behavior data of the user, determining the user's historical preferences based on the historical behavior data and the interactive voice, and determining recommended content for the television device based on the historical preferences and the current emotional state; A response strategy is determined according to the current emotional state, and the recommended content is output according to the response strategy.

[0007] In a second aspect, an embodiment of the present invention provides a voice interaction device, which is applied to a television terminal and includes: A feature fusion module is used to obtain the user's interactive speech, extract the spectrum features and speech time sequence features of the interactive speech, and fuse the spectrum features and the speech time sequence features to obtain an emotion feature vector; A classification module is used to input the emotional feature vector into a preset neural network model for classification to obtain the user's current emotional state; a data analysis module, configured to obtain historical behavior data of the user, determine the user's historical preferences based on the historical behavior data and the interactive voice, and determine recommended content for the television device based on the historical preferences and the current emotional state; The response module is used to determine a response strategy according to the current emotional state, and output the recommended content according to the response strategy.

[0008] In a third aspect, an embodiment of the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned voice interaction method when executing the computer program.

[0009] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned voice interaction method is implemented.

[0010] Compared with the prior art, the present invention has the following beneficial effects: by obtaining the user's interactive voice, extracting the spectral features and voice timing features of the interactive voice, fusing the spectral features and voice timing features to obtain an emotional feature vector, inputting the emotional feature vector into a preset neural network model for classification, obtaining the user's current emotional state, obtaining the user's historical behavior data, determining the user's historical preferences based on the historical behavior data and interactive voice, determining the recommended content of the television device based on the historical preferences and current emotional state, determining the response strategy based on the current emotional state, and outputting the recommended content based on the response strategy. By extracting and classifying the emotional feature vector in the user's interactive voice, the user's current emotional state is obtained, the user's historical preferences are determined based on the historical behavior data and interactive voice, the recommended content is determined based on the historical preferences and current emotional state, and the recommended content is output in combination with the current emotional state. In this way, the user's interactive voice is comprehensively emotionally recognized at the television terminal to accurately provide feedback on the interactive voice. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0012] Figure 1 This is a schematic diagram of an application environment of a voice interaction method provided in Example 1 of the present invention; Figure 2 This is a flow chart of a voice interaction method provided in the second embodiment of the present invention; Figure 3 This is a flow chart of a voice interaction method provided in the third embodiment of the present invention; Figure 4 This is a flow chart of a voice interaction method provided in the fourth embodiment of the present invention; Figure 5 This is a flow chart of a voice interaction method provided in Embodiment 5 of the present invention; Figure 6 This is a flow chart of a voice interaction device provided by Example 6 of the present invention; Figure 7 This is a structural diagram of a computer device provided in Example 7 of the present invention. DETAILED DESCRIPTION

[0013] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0014] like Figure 1 As shown, it is a schematic diagram of the application environment of a voice interaction method provided in the first embodiment of the present invention, wherein the voice interaction method is applied to a television terminal, wherein the client and the server are connected for communication, the server is connected for communication with the television terminal, and the user can provide conditions, requirements and operation instructions for voice interaction by operating the client, and the server is used to generate control instructions for the television terminal according to the relevant content sent by the client, so that the user can remotely control the television terminal. The client includes but is not limited to various computer devices such as personal computers, laptops, smart phones, tablet computers and portable wearable devices. The computer device corresponding to the server can be implemented with an independent server or a server cluster composed of multiple servers.

[0015] like Figure 2FIG. 1 is a flow chart of a voice interaction method provided in Embodiment 2 of the present invention. The voice interaction method is applied to a television terminal and may include the following steps: Step S201 : Acquire the user's interactive speech, extract the spectrum features and speech time sequence features of the interactive speech, fuse the spectrum features and the speech time sequence features, and obtain an emotion feature vector.

[0016] TV terminals are equipped with audio capture devices such as microphones. When a user interacts with the TV through voice, such as saying things like "I want to watch a suspense film" or "Play music," the microphone captures the surrounding sound signals, converts them into electrical signals, and then digitizes them to produce digital voice data that can be processed by computers. Voice signals are inherently complex signals that vary over time. Spectral analysis converts voice signals from the time domain to the frequency domain and, using mathematical methods such as Fourier transforms, decomposes them into a combination of frequency components. Different speech content and pronunciation methods exhibit distinct distribution patterns in the frequency spectrum. For example, in sentiment analysis, angry speech may have more high-frequency components and greater energy, while calm speech may have a relatively flat spectrum. Common spectral features include spectral envelopes and cepstral coefficients (such as Mel-frequency cepstral coefficients (MFCCs)).

[0017] Spectral features and speech timing features describe the characteristics of speech signals from different perspectives. Using either feature alone cannot fully and accurately convey the emotional information contained in user speech. Fusion can leverage the strengths of both and more comprehensively and meticulously characterize the emotional characteristics of speech. Directly concatenate the spectral feature vector and the speech timing feature vector to form a longer vector. For example, a spectral feature vector with 10 dimensions and a speech timing feature vector with 5 dimensions can be concatenated to produce a 15-dimensional vector. Machine learning or deep learning methods, such as neural networks, can be used to achieve this. By training a specialized fusion model, the model learns how to effectively fuse the two features to produce a more representative emotional feature vector.

[0018] Optionally, the spectral features include pitch features and energy features, and the speech temporal features include speech rate features, speech pause frequency features, and intonation change features; The step S201 of fusing the spectral features and the speech time sequence features to obtain an emotion feature vector may include the following steps: The pitch feature, the energy feature, the speech rate feature, the speech pause frequency feature, and the intonation change feature are fused to form a multidimensional feature vector, which is the emotion feature vector.

[0019] Pitch essentially reflects the highness of a sound, which is determined by the frequency of the sound-producing organ's vibration. In speech signals, pitch is related to the frequency of vocal cord vibration. Pitch varies significantly with different emotional states. For example, when excited or agitated, people tend to unconsciously raise their pitch, making their voices shrill. Conversely, when sad or depressed, their pitch typically drops, making their voices sound deeper. Therefore, pitch characteristics are an important clue to determining the emotion of speech.

[0020] The energy characteristic of speech reflects the strength or loudness of a sound, and is related to the force with which it is uttered. Generally speaking, when a speaker is expressing strong emotions, such as anger or excitement, their speech energy is higher, resulting in a louder and more forceful voice. Meanwhile, when a speaker is calm or tired, their speech energy is lower, resulting in a softer voice. By analyzing the energy characteristics of speech, we can capture these emotional differences.

[0021] Speech rate refers to the speed of speech. Changes in speech rate can reflect the speaker's emotions and mental state. When someone is nervous, anxious, or excited, their speech rate tends to increase; when they are relaxed, leisurely, or sad, their speech rate may slow down. For example, when someone is excitedly recounting something, they may speak quickly and hurriedly.

[0022] Pause frequency refers to the frequency of pauses during speech. In normal communication, people pause appropriately to convey their thoughts, but emotional fluctuations can affect pause frequency. For example, when someone is hesitant, nervous, or thinking, pause frequency may increase. On the other hand, when expressing themselves fluently and confidently, pause frequency is relatively low.

[0023] Inflection reflects the rhythmic changes in pitch, strength, duration, and other aspects of speech. Different intonations can convey different emotions and tones. A rising intonation often expresses questioning or surprise, while a falling intonation can indicate a statement or affirmation. Fluctuations in intonation can also reflect the speaker's emotional attitude, from enthusiasm to indifference. For example, saying "Wow!" with a rising and varied intonation often conveys surprise and excitement.

[0024] The pitch features, energy features, speaking speed features, voice pause frequency features and intonation change features are integrated to form a multi-dimensional feature vector (i.e., emotion feature vector) in order to comprehensively utilize these different aspects of feature information to more comprehensively and accurately portray the emotions contained in the speech.

[0025] Optionally, before obtaining the user's interactive voice, the method further includes: Acquire the original voice of the user received by the microphone of the television device, analyze the original voice, and determine the voice starting point and the voice ending point; According to the speech starting point and the speech ending point, a valid speech segment is determined, and noise reduction is performed on the valid speech segment to obtain interactive speech.

[0026] The TV's microphone is responsible for collecting ambient sound, including the user's voice. The original voice signal is a continuous analog signal that contains the user's voice information but is also mixed with various environmental noises, such as background noise and the sounds of electrical equipment.

[0027] The energy of a speech signal is significantly higher when there is speech than when there is silence or noise. By setting an energy threshold, when the energy of the speech signal exceeds this threshold, it is considered that a speech segment has begun. When the energy falls below the threshold for a period of time, the speech is considered to have ended.

[0028] The zero-crossing rate refers to the number of times a speech signal crosses zero level per unit time. The zero-crossing rate of a speech signal is typically more regular than that of noise and exhibits significant differences between speech and non-speech segments. Combining the zero-crossing rate with energy features can more accurately determine the onset and end of speech.

[0029] By monitoring and analyzing the original speech signal in real time, the system uses the aforementioned features to determine the moment when speech begins, known as the speech onset, and the moment when speech ends and silence returns, known as the speech end. Determining these two points separates the original speech signal into a portion containing speech information and a portion containing non-speech information.

[0030] Once the speech start and end points are determined, the portion between these two points can be extracted from the original speech signal to create a valid speech segment. This valid segment contains only the user's voice information, removing the preceding and following silence and noise, making subsequent processing more precise and effective.

[0031] Step S202: input the emotional feature vector into a preset neural network model for classification to obtain the user's current emotional state.

[0032] The preset neural network model is a complex network structure composed of multiple neuron layers, and can use multi-layer perceptron (MLP), convolutional neural network (CNN), recurrent neural network (RNN) and its variants (such as LSTM and GRU). These network structures can automatically learn complex patterns and features in input data.

[0033] Before formal application, the model requires training using a large amount of labeled data. This training data includes emotional feature vectors for different emotional states and their corresponding emotion labels (such as happiness, sadness, anger, surprise, fear, and calmness). Using backpropagation algorithms and optimizers (such as stochastic gradient descent and Adam), the model's weights and parameters are continuously adjusted to ensure that the model can accurately predict the corresponding emotion label based on the input emotional feature vector.

[0034] Before inputting the sentiment feature vector into the neural network model, some preprocessing operations, such as normalization, are required. Normalization scales the feature vector values ​​to a specific range, helping to improve model training efficiency and stability and preventing excessive impact of certain features due to excessively large numerical ranges. The preprocessed sentiment feature vector is then passed as input to the input layer of the neural network model. After receiving this data, the input layer sequentially passes it to the hidden layers for feature transformation and nonlinear processing, according to the model's structure and connection method.

[0035] The hidden layer is the core of the neural network model and consists of multiple neurons. Each neuron performs a weighted summation of the input data and applies a nonlinear transformation using an activation function (such as ReLU, Sigmoid, or Tanh). By stacking multiple hidden layers, the model can learn the feature representations in the sentiment feature vector.

[0036] As data is continuously passed through and processed in the hidden layers, the model gradually extracts key features relevant to emotion classification. Ultimately, the data is passed to the output layer, which typically has a number of neurons equal to the number of emotion label categories. Each neuron corresponds to an emotion category, and its output value represents the probability that the input emotion feature vector belongs to that emotion category. Based on the output values ​​of the output layer neurons, the model selects the category with the highest probability as the final classification result, representing the user's current emotional state. For example, if the output value of the neuron representing "happy" in the output layer is the highest, the model will determine that the user's current emotional state is happy.

[0037] Step S203 , obtaining the historical behavior data of the user, determining the historical preferences of the user based on the historical behavior data and the interactive voice, and determining the recommended content of the television device based on the historical preferences and the current emotional state.

[0038] Among them, the TV terminal will continuously record various operating behaviors during the user's use. These data cover a wealth of information, can reflect the user's past interests and tendencies, and record the types of programs the user has watched, such as movies, TV series, variety shows, sports events, etc. It will also record the specific content watched, such as the name of a certain movie, the number of a certain variety show, etc., as well as the viewing time, such as whether it is watched during the day or at night, and the length of each viewing.

[0039] Keywords that users search for on TV, such as "science fiction movies" and "latest football matches," can directly reflect the content that users may be interested in at the moment. Programs and channels that users have favorited indicate that these are contents that users are particularly interested in or would like to view again. Associate interactive voice with historical behavior data. For example, if a user says "I want to watch suspense-type programs" in interactive voice, combined with historical behavior data showing that the user frequently watches suspense movies and suspense TV series, it can be determined that the user has a clear preference for suspense-type content. User preferences can be further clarified by analyzing patterns in historical behavior data. For example, if it is found that a user always watches sports events on weekend nights, it can be inferred that the user has a preference for watching sports programs on weekend nights.

[0040] The app filters content tailored to the user's current emotional state. For example, if the user is feeling happy, it recommends lighthearted and cheerful programs like comedy movies and funny variety shows. For those feeling sad, it recommends heartwarming and healing content like heartwarming documentaries and inspirational movies.

[0041] After filtering out content that matches the user's emotional state, further integration is performed based on the user's historical preferences. For example, if the user has a history of preferring suspenseful content and is currently feeling calm, the recommended content might be suspense dramas or documentaries with a less intense, slower pace. To meet the diverse needs of users, content recommendations can employ diverse strategies. In addition to recommending content that the user is likely to be most interested in, it can also include related but slightly different content to expand the user's viewing options. For example, in addition to recommending classic suspense films, it can also recommend emerging suspense web series.

[0042] Step S204: determining a response strategy based on the current emotional state, and outputting the recommended content based on the response strategy.

[0043] Different strategies correspond to different emotions. When a user is in a happy mood, the response strategy can be lively, cheerful, and enthusiastic. A more relaxed and playful tone, using positive vocabulary and expressions, can create a more pleasant atmosphere. For example, when recommending content, you can say, "Wow! I found a great show for you! I guarantee it will make you laugh even harder!" If the user is feeling sad, your response should be gentle, comforting, and empathetic. Use softer language to provide emotional support. For example, "Don't be too sad. I recommend some heartwarming and healing content to help you feel better." If you're dealing with an angry user, stay calm and sincere in your response to avoid further angering them. First, express understanding, then offer a solution. For example, "We understand you're a little angry right now. We'd like to recommend some great shows to help you cool down." When the user is calm, the response can be concise, objective, and neutral, with recommendations provided in a normal conversational tone. For example, "I recommend the following programs. Please check them out to see if they are interesting." The tone and intonation of the voice broadcast should be adjusted according to the determined response strategy. If the response strategy is cheerful, the voice pitch can be appropriately raised, the speed can be slightly faster, and it can be more energetic; if the response strategy is comforting, the tone should be low, soft, and the speed can be slowed down.

[0044] In an embodiment of the present application, by obtaining the user's interactive voice, extracting the spectral features and voice timing features of the interactive voice, fusing the spectral features and voice timing features to obtain an emotional feature vector, inputting the emotional feature vector into a preset neural network model for classification, obtaining the user's current emotional state, obtaining the user's historical behavior data, determining the user's historical preferences based on the historical behavior data and interactive voice, determining the recommended content of the television device based on the historical preferences and current emotional state, determining the response strategy based on the current emotional state, and outputting the recommended content based on the response strategy. By extracting and classifying the emotional feature vector in the user's interactive voice, the user's current emotional state is obtained, determining the user's historical preferences based on the historical behavior data and interactive voice, determining the recommended content based on the historical preferences and current emotional state, and outputting the recommended content based on the response strategy. In this way, the user's interactive voice is comprehensively emotionally recognized at the television terminal to accurately provide feedback on the interactive voice.

[0045] like Figure 3 FIG. 2 is a flow chart of a voice interaction method according to a third embodiment of the present invention. In step S203, determining the user's historical preferences based on the historical behavior data and the interactive voice may include the following steps: Step S301: extracting historical viewing records, rating data, and favorite data from the historical behavior data.

[0046] Step S302: Analyze the speech semantics of the interactive speech, and determine the user's historical preferences based on the speech semantics, the historical viewing records, the rating data, and the favorite data.

[0047] Among them, historical viewing records are detailed information about various programs that users have watched on their TV devices in the past. They record the time, title, and genre of the programs they watched. By analyzing historical viewing records, we can understand the types of programs that users frequently watch, such as movies, TV series, variety shows, and sports events. We can also understand the user's viewing schedule patterns, such as whether they often watch specific types of programs on weekend nights. This data is typically automatically recorded by the TV device's operating system or application and stored in a local database or cloud server.

[0048] Rating data is the user's evaluation score for the programs they watched, reflecting their preference for the programs. A high score indicates that the user enjoys the program, while a low score indicates dissatisfaction. By analyzing rating data, we can clearly identify user preferences for different program genres and specific programs. For example, if a user gives high scores to most science fiction movies, we can infer that the user has a high preference for science fiction movies.

[0049] Favorite data records users' active additions of certain programs to their favorites. Favorited programs are typically programs that users are particularly interested in and want to rewatch. Analyzing favorite data can directly reveal a user's core interests. For example, if a user has favorited multiple suspense dramas, this indicates a strong preference for them.

[0050] Natural language processing (NLP) technology analyzes interactive speech, converts it into text, and then understands the semantics of the text. This includes identifying keywords, sentence structure, and semantic relationships. For example, if a user says, "I like watching comedy movies," semantic analysis can extract key information such as "like" and "comedy movies."

[0051] In addition to analyzing the semantics of individual sentences, the context of the interactive speech needs to be considered. Sometimes, a user's expression can be ambiguous or require contextual understanding. For example, if a user mentions "I've been very stressed lately" and then says "I'm looking for something relaxing," the context suggests that the user might be looking for something relaxing and relaxing, such as a comedy.

[0052] In this embodiment, by utilizing the user's historical behavior data and interactive voice information, the user's historical preferences are accurately determined, thereby providing the user with recommended content that is more in line with his or her interests.

[0053] like Figure 4 FIG. 2 is a flow chart of a voice interaction method according to a fourth embodiment of the present invention. The response strategy includes a response tone, a response speed, and a response vocabulary. Step S204 of determining the response strategy based on the current emotional state and outputting the recommended content based on the response strategy may include the following steps: Step S401: determining the tone style of the response tone according to the current emotional state.

[0054] Step S402: determining the response speed according to the current emotional state.

[0055] Step S403: determining the vocabulary type of the response vocabulary according to the current emotional state.

[0056] Step S404: Generate a response voice based on the tone style, the speed, and the vocabulary type, and output the recommended content based on the response voice.

[0057] Among them, the tone style can intuitively convey attitudes and emotions. Different emotional states correspond to different tone styles. For example, positive emotions (such as happiness and excitement) should adopt a cheerful, enthusiastic, and lively tone style, while negative emotions (such as sadness and anger) require a gentle, comforting, and sincere tone style.

[0058] Response speed affects the rhythm of communication and the user experience. Response speed varies depending on the user's mood. When the user is in a positive mood, the response speed can be increased appropriately to demonstrate enthusiasm and positivity, matching the user's excitement and making the communication feel smoother and more efficient.

[0059] In situations of negative emotion, respond slowly, giving users ample time to process the information and demonstrating patience and consideration. Avoid responding too quickly, making users feel rushed or ignored. For example, if a user is angry, offer recommendations slowly and clearly, saying, "Please calm down. We've prepared some suitable programs for you."

[0060] When the user is calm, the response speed should be moderate to maintain a normal communication rhythm, and the user will not feel overly anxious or slow.

[0061] Speech synthesis technology is used to generate a voice response that meets the requirements, taking into account the previously determined tone, tempo, and vocabulary. During the generation process, parameters such as pitch, timbre, and rhythm are adjusted to achieve the desired tone and tempo. The generated voice response is played through the TV's speakers, and information related to the recommended content, such as the program title, poster, and brief description, can be displayed on the screen. This allows for a more intuitive presentation of recommended content, enhancing the user experience.

[0062] In this embodiment, appropriate response voice is generated and recommended content is output based on the user's current emotional state, thereby achieving a more personalized and humanized interactive service.

[0063] like Figure 5FIG. 2 is a flow chart of a voice interaction method according to a fifth embodiment of the present invention. After outputting the recommended content in step S204, the method may further include the following steps: Step S501: obtaining the user's feedback behavior on the recommended content, and analyzing the feedback behavior to obtain a feedback result.

[0064] Step S502: determining an optimization strategy for the recommended content according to the feedback result.

[0065] Optionally, the feedback behavior includes voice feedback and behavioral feedback; Determine whether the voice feedback matches a preset voice keyword, and if so, determine a voice reply result; Determine whether the behavior feedback conforms to the preset operation behavior, and if so, determine the behavior response result; An optimization strategy for the response voice and the recommended content is determined based on the voice response result and the behavioral response result.

[0066] The system then determines whether the voice feedback matches pre-set voice keywords. These keywords can be set based on different feedback types, such as positive feedback keywords ("like," "good," "great," etc.) and negative feedback keywords ("dislike," "not interested," "terrible," etc.). If the pre-set voice keywords are matched, the corresponding voice response result can be determined. For example, if the user says "like," the voice response result is positive feedback; if they say "dislike," it is negative feedback.

[0067] Determine whether the behavioral feedback matches the pre-set action. Pre-set actions correspond to different feedback types. For example, clicking to play a recommended program may indicate user interest, which is positive feedback, while skipping a recommended content may indicate user disinterest, which is negative feedback. When the behavioral feedback matches the pre-set action, the behavioral response result is determined.

[0068] If both the voice response and the behavioral response show positive feedback, it indicates that the recommended content meets the user's needs and interests. At this point, we can increase our recommendations for similar content, further exploring similar programs that the user might be interested in, and broaden our recommendations. For example, if a user gives positive feedback on a sci-fi movie recommendation, we can recommend more excellent sci-fi movies of the same genre.

[0069] If the feedback is negative, the recommendation strategy needs to be adjusted. You can reduce recommendations for content of the current type and analyze the reasons for the user's dislike, such as program type, theme, and style. Based on the analysis, adjust feature weights or change the recommendation algorithm to provide content that better suits the user's interests. For example, if users give poor feedback on a comedy show, you might reduce the proportion of comedy recommendations or try recommending different styles of comedy.

[0070] Feedback can also be used to optimize the response voice. If a user is interested in the recommended content but finds the tone, speed, or wording of the response voice unsuitable, adjustments can be made based on this feedback. For example, if a user says the response voice is too fast, the subsequent response voice will be slowed down appropriately. If the user feels the wording is not vivid enough, more engaging words can be replaced.

[0071] In this embodiment, by continuously collecting user feedback and optimizing the recommended content and response voice according to the feedback results, the recommendation strategy is continuously improved to provide users with more personalized and high-quality services.

[0072] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0073] like Figure 6 The figure shows a schematic diagram of a voice interaction device provided in the sixth embodiment of the present invention. The voice interaction device is applied to a television terminal and corresponds one-to-one with the voice interaction method in the above embodiment. The voice interaction device includes a feature fusion module 61, a classification module 62, a data analysis module 63, and a response module 64. The functional modules are described in detail as follows: A feature fusion module 61 is configured to obtain the user's interactive speech, extract the spectrum features and speech time sequence features of the interactive speech, and fuse the spectrum features and speech time sequence features to obtain an emotion feature vector; A classification module 62 is configured to input the emotion feature vector into a preset neural network model for classification to obtain the user's current emotional state; a data analysis module 63 configured to obtain historical behavior data of the user, determine the user's historical preferences based on the historical behavior data and the interactive voice, and determine recommended content for the television device based on the historical preferences and the current emotional state; The response module 64 is configured to determine a response strategy based on the current emotional state, and output the recommended content based on the response strategy.

[0074] Optionally, the feature fusion module 61 includes: The emotion feature unit is used to fuse the pitch feature, the energy feature, the speech rate feature, the voice pause frequency feature and the intonation change feature to form a multidimensional feature vector, which is the emotion feature vector.

[0075] Optionally, the data analysis module 63 includes: A data extraction unit, configured to extract historical viewing records, rating data, and favorite data from the historical behavior data; A historical preference determination unit is used to analyze the speech semantics of the interactive speech and determine the user's historical preferences based on the speech semantics, the historical viewing records, the rating data, and the favorite data.

[0076] Optionally, the response module 64 includes: a tone style determining unit, configured to determine the tone style of the response tone according to the current emotional state; a response speed determining unit, configured to determine a response speed according to the current emotional state; a vocabulary type determination unit, configured to determine the vocabulary type of the response vocabulary according to the current emotional state; The content recommendation unit is used to generate a response voice according to the tone style, the speed and the vocabulary type, and output the recommended content according to the response voice.

[0077] Optionally, the response module 64 includes: A behavior analysis unit, configured to obtain user feedback on the recommended content and analyze the feedback to obtain a feedback result; The optimization strategy determining unit is used to determine an optimization strategy for the recommended content according to the feedback result.

[0078] Optionally, the response module 64 further includes: A voice reply unit, configured to determine whether the voice feedback matches a preset voice keyword, and if so, determine a voice reply result; A behavior response unit is used to determine whether the behavior feedback conforms to a preset operation behavior, and if so, determine a behavior response result; An optimization strategy determination unit is used to determine an optimization strategy for the response voice and the recommended content according to the voice reply result and the behavior reply result.

[0079] Optionally, the voice interaction device further includes: A speech analysis module is used to obtain the original speech of the user received by the microphone of the television device before obtaining the interactive speech of the user, analyze the original speech, and determine the speech starting point and speech ending point; The interactive speech acquisition module is used to determine a valid speech segment according to the speech starting point and the speech ending point, and perform noise reduction on the valid speech segment to obtain interactive speech.

[0080] For the specific definition of the voice interaction device, please refer to the definition of the voice interaction method above, which will not be repeated here. The above-mentioned voice interaction device and each module in the voice interaction device can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0081] like Figure 7 The figure shows a schematic diagram of the structure of a computer device provided in Example 7 of the present invention. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a voice interaction method is implemented.

[0082] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the voice interaction method in the above embodiment is implemented, for example Figures 2 to 5 Alternatively, when the processor executes the computer program, the functions of the voice interaction device or each module / unit in the embodiment of the voice interaction device are realized, for example Figure 6 The functions of the feature fusion module 61, classification module 62, data analysis module 63, and response module 64 are not described here in detail to avoid repetition.

[0083] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the voice interaction method in the above embodiment is implemented, such as Figures 2 to 5Alternatively, when the computer program is executed by the processor, the functions of each module / unit in the embodiment of the above-mentioned voice interaction device are realized, for example Figure 6 The feature fusion module 61, classification module 62, data analysis module 63, and response module 64 are not described here in detail to avoid repetition. The computer-readable storage medium may be non-volatile or volatile.

[0084] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0085] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0086] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A voice interaction method, characterized in that: The voice interaction method is applied to a television terminal, and the voice interaction method includes: Acquire the user's interactive speech, extract the spectrum features and speech time sequence features of the interactive speech, and fuse the spectrum features and the speech time sequence features to obtain an emotion feature vector; Inputting the emotional feature vector into a preset neural network model for classification to obtain the user's current emotional state; Acquiring historical behavior data of the user, determining the user's historical preferences based on the historical behavior data and the interactive voice, and determining recommended content for the television device based on the historical preferences and the current emotional state; A response strategy is determined according to the current emotional state, and the recommended content is output according to the response strategy.

2. The voice interaction method according to claim 1, characterized in that The spectral features include pitch features and energy features, and the speech time sequence features include speech rate features, speech pause frequency features, and intonation change features; The step of fusing the spectral features and the speech time sequence features to obtain an emotion feature vector includes: The pitch feature, the energy feature, the speech rate feature, the speech pause frequency feature, and the intonation change feature are fused to form a multidimensional feature vector, which is the emotion feature vector.

3. The voice interaction method according to claim 1, wherein: The determining the user's historical preferences based on the historical behavior data and the interactive voice includes: Extracting historical viewing records, rating data, and favorites data from the historical behavior data; The speech semantics of the interactive speech are analyzed, and the user's historical preferences are determined based on the speech semantics, the historical viewing records, the rating data, and the favorite data.

4. The voice interaction method according to claim 1, wherein: The response strategy includes response tone, response speed and response vocabulary; The step of determining a response strategy based on the current emotional state and outputting the recommended content based on the response strategy includes: determining a tone of voice for a response based on the current emotional state; Determining a response speed based on the current emotional state; determining a vocabulary type of a response vocabulary according to the current emotional state; A response voice is generated according to the tone style, the speed and the vocabulary type, and the recommended content is output according to the response voice.

5. The voice interaction method according to claim 4, characterized in that: After outputting the recommended content, the method further includes: Obtaining user feedback on the recommended content, and analyzing the feedback to obtain feedback results; An optimization strategy for the recommended content is determined based on the feedback result.

6. The voice interaction method according to claim 5, characterized in that: The feedback behavior includes voice feedback and behavioral feedback; Determining an optimization strategy for the recommended content based on the feedback result includes: Determine whether the voice feedback matches a preset voice keyword, and if so, determine a voice reply result; Determine whether the behavior feedback conforms to the preset operation behavior, and if so, determine the behavior response result; An optimization strategy for the response voice and the recommended content is determined based on the voice response result and the behavioral response result.

7. The voice interaction method according to claim 1, wherein: Before obtaining the user's interactive voice, the method further includes: Acquire the original voice of the user received by the microphone of the television device, analyze the original voice, and determine the voice starting point and the voice ending point; According to the speech starting point and the speech ending point, a valid speech segment is determined, and noise reduction is performed on the valid speech segment to obtain interactive speech.

8. A voice interaction device, characterized in that: The voice interaction device is applied to a television terminal, and the voice interaction device includes: A feature fusion module is used to obtain the user's interactive speech, extract the spectrum features and speech time sequence features of the interactive speech, and fuse the spectrum features and the speech time sequence features to obtain an emotion feature vector; A classification module is used to input the emotional feature vector into a preset neural network model for classification to obtain the user's current emotional state; a data analysis module, configured to obtain historical behavior data of the user, determine the user's historical preferences based on the historical behavior data and the interactive voice, and determine recommended content for the television device based on the historical preferences and the current emotional state; The response module is used to determine a response strategy according to the current emotional state, and output the recommended content according to the response strategy.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the voice interaction method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the voice interaction method according to any one of claims 1 to 7 is implemented.

Images