Interaction method and device for output terminal, output terminal, and computer-readable storage medium

Abstract

The application relates to the technical field of smart home, and discloses an interaction method for an output terminal, which comprises the following steps: acquiring physiological features, behavior features and language features of a user; determining a user emotion according to the physiological features, the behavior features and the language features; and controlling the output terminal to run according to the user emotion or according to the user emotion and a user instruction. Through such intelligent environment regulation, not only can the emotion analysis result be provided, but also measures can be actively taken to help the user to adjust the emotion, so that a complete closed loop from analysis to intervention is realized, and a more considerate and comprehensive emotion care experience is brought to the user. In this way, the interaction between the user and the output terminal is more intelligent. The application further discloses an interaction device for the output terminal, the output terminal and a computer readable storage medium.

Description

Technical Field

[0001] This application relates to the field of smart home technology, such as an interaction method and device for an output terminal, an output terminal, and a computer-readable storage medium. Background Technology

[0002] Currently, emotion recognition technology is increasingly demonstrating its potential in emotion state analysis and classification applications through biometric data. However, most existing systems rely on single-modal data, such as facial expressions or voice features, collected via sensors like high-resolution cameras and microphones. While these data are diverse, the independent processing and analysis methods limit their full potential.

[0003] The related technology discloses a control method for outputting audiovisual information from a household appliance, comprising: acquiring voice information and image information of a target object, wherein the voice information is used to control the output device to perform a preset action, and the image information includes at least facial expression information of the target object; identifying one of the voice information and the image information to obtain first emotion information; and correcting the first emotion information using a correction model based on the other of the voice information and the image information to obtain second emotion information, wherein the first emotion information includes at least one first emotion state of the target object, and the second emotion information includes at least one second emotion state of the target object; and outputting audiovisual information matching the second emotion information based on the second emotion information, wherein the audiovisual information includes at least one of the following: image information, light information, and audio information.

[0004] In the process of implementing the embodiments of this disclosure, at least the following problems were found in the related art:

[0005] The use of any single piece of information to determine or correct emotions by related technologies will produce biases, and these biases will accumulate, making it impossible to obtain accurate emotions and thus affecting the intelligence of user-device interaction.

[0006] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0008] This disclosure provides an interaction method and apparatus for an output terminal, an output terminal, and a computer-readable storage medium, to make the interaction between the user and the output terminal more intelligent.

[0009] In some embodiments, the interaction method for the output terminal includes: acquiring a user's physiological characteristics, behavioral characteristics, and linguistic characteristics; determining a user's emotion based on the physiological characteristics, behavioral characteristics, and linguistic characteristics; and controlling the operation of the output terminal based on the user's emotion or based on the user's emotion and user instructions.

[0010] In some embodiments, the interactive device for the output terminal includes a processor and a memory storing program instructions, the processor being configured to execute the aforementioned interactive method for the output terminal when the program instructions are executed.

[0011] In some embodiments, the output terminal includes: an output terminal body; and an interactive device for the output terminal as described above, which is installed on the output terminal body.

[0012] In some embodiments, the computer-readable storage medium stores program instructions that, when executed, perform the aforementioned interactive method for an output terminal.

[0013] The interactive method and apparatus for an output terminal, the output terminal, and the computer-readable storage medium provided in this disclosure can achieve the following technical effects:

[0014] Based on physiological, behavioral, and linguistic characteristics that can reveal a user's outward and potential emotions, user emotions are accurately determined. The output terminal's operation is controlled based on the user's emotions, enabling the terminal's operation to match the user's emotions and facilitating emotional interaction between the terminal and the user. Alternatively, the output terminal's operation can be controlled based on both user emotions and user commands, ensuring that the terminal's operation both aligns with user instructions and considers the user's emotions. Through this intelligent environmental control, not only are emotion analysis results provided, but proactive measures can also be taken to help users regulate their emotions, achieving a complete closed loop from analysis to intervention, bringing users a more considerate and comprehensive emotional care experience. This makes the interaction between the user and the output terminal more intelligent.

[0015] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0016] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein:

[0017] Figure 1 This is a schematic diagram of an output terminal provided in an embodiment of this disclosure;

[0018] Figure 2 This is a schematic diagram of an interaction method for an output terminal provided in an embodiment of this disclosure;

[0019] Figure 3 This is a schematic diagram of another interaction method for an output terminal provided in an embodiment of this disclosure;

[0020] Figure 4 This is a schematic diagram of another interaction method for an output terminal provided in an embodiment of this disclosure;

[0021] Figure 5 This is a schematic diagram illustrating how the target environmental atmosphere is determined based on the user's emotions, provided in an embodiment of this disclosure.

[0022] Figure 6 This is a schematic diagram of another interaction method for an output terminal provided in an embodiment of this disclosure;

[0023] Figure 7 This is a schematic diagram of another interaction method for an output terminal provided in an embodiment of this disclosure;

[0024] Figure 8 This is a schematic diagram of another interaction method for an output terminal provided in an embodiment of this disclosure;

[0025] Figure 9 This is a schematic diagram of an interactive device for an output terminal provided in an embodiment of this disclosure;

[0026] Figure 10 This is a schematic diagram of another interactive device for an output terminal provided in an embodiment of this disclosure;

[0027] Figure 11 This is a schematic diagram of another output terminal provided in an embodiment of this disclosure. Detailed Implementation

[0028] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0029] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0030] Unless otherwise stated, the term "multiple" means two or more.

[0031] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0032] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0033] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.

[0034] Combination Figure 1 As shown in the figure, this disclosure provides an output terminal, including a data acquisition unit and a data processing unit.

[0035] Data Acquisition Unit: The Cornerstone of Multi-Dimensional Data Acquisition

[0036] This is used to collect biometric data from users. This data includes human movements, facial images, and voice data, providing rich foundational information for emotion recognition. Optionally, the data acquisition unit includes: an image acquisition module, a sound acquisition module, and a sensing module.

[0037] Image acquisition module: Used to acquire user behavioral characteristics, including body movements, postures, and other features. Simultaneously, the image acquisition module is also used to acquire physiological characteristics, such as facial expressions.

[0038] Voice Acquisition Module: Used to collect the user's speech characteristics. Optionally, the voice acquisition module includes a microphone. The high-sensitivity microphone is designed for recording clear, noise-free audio, featuring noise suppression capabilities to effectively filter background noise and preserve the user's speech data. This lays a solid foundation for subsequent extraction of features reflecting emotional states, such as Mel-frequency cepstral coefficients (MFCCs).

[0039] Sensing module: Used to collect some physiological characteristics of the user, such as heart rate and blood pressure. Optionally, the sensing module is a wearable device worn by the user. Wearable devices include, for example, smartwatches, smart bracelets, pedometers, etc.

[0040] Data Processing Unit: The Heart of Intelligent Analysis and Fusion

[0041] A high-speed, secure communication connection is established between the data processing unit and the data acquisition unit to ensure the real-time performance, reliability, and security of data transmission. It also supports data encryption to strictly protect user privacy. The data processing unit is responsible for receiving, preprocessing, fusing, and classifying biometric data. It performs efficient and accurate preprocessing on the collected data to purify it and provide high-quality material for subsequent analysis. The preprocessed data features are then fed into a multimodal ensemble algorithm. This algorithm combines features collected from different modules and uses advanced data fusion technology to generate a unified feature vector that comprehensively reflects the user's emotional state.

[0042] The data processing unit is equipped with advanced neural networks such as Convolutional Neural Networks (CNN) and Long Short-Term Memory Networks (LSTM) to learn and predict user emotion categories from a unified feature vector. These networks are trained on a large amount of training data and are able to accurately identify different emotional states.

[0043] Optionally, the image acquisition module and the sound acquisition module are mounted on the output terminal. Meanwhile, with the user's authorization, the data processing unit integrates data from existing cameras or sensors in the home to ensure accurate recognition of the user's emotions.

[0044] Optionally, the image acquisition module and the sound acquisition module are integrated into the same output terminal. In this way, the output terminal not only collects the user's voice data but also captures subtle emotional changes such as facial expressions. Furthermore, with the user's authorization, the output terminal can connect with smart home appliances throughout the house to achieve comprehensive and multi-dimensional user data collection, providing rich and multi-dimensional data support for emotion analysis.

[0045] Optionally, the image acquisition module and the sound acquisition module are mounted on different output terminals.

[0046] Optionally, the data processing unit can be a standalone server or a processor within the output terminal itself. If the output terminal has audio and video output modules, the data processing unit communicates with these modules to control the output terminal to output audio and / or video. For example, controlling a smart TV to play video and output sound.

[0047] Output terminal: A window for user-friendly feedback and interaction

[0048] The output terminal is responsible for presenting the emotion classification results to the user in an intuitive and user-friendly way.

[0049] Optionally, the output terminal includes one or more smart output terminals such as smart TVs, smart curtains, smart lighting, smart speakers, smart air conditioners, and smart mobile devices (such as smartphones). In this embodiment of the disclosure, a smart output terminal refers to a home appliance product formed by introducing microprocessors, sensor technology, and network communication technology into the output device. It has the characteristics of intelligent control, intelligent sensing, and intelligent application. The operation of a smart output terminal often relies on the application and processing of modern technologies such as the Internet of Things, the Internet, and electronic chips. For example, a smart output terminal can be connected to electronic devices to enable users to remotely control and manage the smart output terminal.

[0050] Optionally, the output terminal is a smart TV. The smart TV can provide users with intuitive feedback through its screen: emotion classification results are primarily displayed on the smart TV screen, allowing users to intuitively understand their emotional state. The smart TV's processor can send the processing results to associated smart home appliances, achieving whole-house smart linkage and enabling functions such as proactively comforting users and soothing emotions. For example, it can recommend suitable video resources or adjust the external environment based on the user's current mood, providing a more humanized and intelligent service experience. As a large screen in the home, the TV facilitates simultaneous use by multiple people, promoting emotional interaction and communication among family members.

[0051] The output terminal sets up an independent account for each family member and intelligently builds a relationship network based on the family structure. It can not only analyze the possible reasons behind a user's emotions based on historical data, but also proactively and accurately send relevant information to relevant family members when family conflicts are detected. Furthermore, it will promptly notify relatives when a user experiences severe negative emotions, demonstrating humane care.

[0052] Combination Figure 2 As shown, this disclosure provides an interaction method for an output terminal, including:

[0053] S101, the data processing unit acquires the user's physiological characteristics, behavioral characteristics and language characteristics.

[0054] S102, the data processing unit determines the user's emotions based on physiological characteristics, behavioral characteristics, and language characteristics.

[0055] S103, the data processing unit controls the operation of the output terminal based on the user's emotions or based on the user's emotions and instructions.

[0056] First, the data acquisition unit obtains the user's physiological, behavioral, and linguistic characteristics. Physiological characteristics can reflect the user's emotions; for example, when a user experiences significant emotional fluctuations such as anger or excitement, physiological parameters such as heart rate and blood pressure will change noticeably. Behavioral characteristics can also reflect a user's emotions; for example, a user rubbing their forehead may indicate annoyance, while clapping may indicate happiness. Linguistic characteristics also reflect a user's emotions and are one of the most direct ways people express their emotions. Then, the user's emotions are determined by combining physiological, behavioral, and linguistic characteristics. The output terminal is then controlled based on the user's emotions, or based on the user's emotions and the commands issued by the user.

[0057] The interaction method for an output terminal provided in this disclosure accurately determines user emotions based on physiological, behavioral, and linguistic characteristics that can express a user's outward and potential emotions. Controlling the output terminal's operation based on user emotions enables the output terminal's operating plan to match the user's emotions and facilitates sensory interaction between the output terminal and the user. Alternatively, controlling the output terminal's operation based on both user emotions and user commands ensures that the output terminal's operation both conforms to user instructions and accommodates user emotions. Through this intelligent environmental control, not only can emotion analysis results be provided, but proactive measures can also be taken to help users regulate their emotions, achieving a complete closed loop from analysis to intervention, and bringing users a more considerate and comprehensive emotional care experience. This makes the interaction between the user and the output terminal more intelligent.

[0058] Combination Figure 3 As shown in the embodiments of this disclosure, another interaction method for an output terminal is provided, including:

[0059] S101, the data processing unit acquires the user's physiological characteristics, behavioral characteristics and language characteristics.

[0060] S112, the data processing unit determines the user's stress based on physiological and behavioral characteristics.

[0061] S122, the data processing unit determines the user's tone based on language characteristics.

[0062] S132, the data processing unit determines the user's emotions based on the user's stress and tone of voice.

[0063] S103, the data processing unit controls the operation of the output terminal based on the user's emotions or based on the user's emotions and instructions.

[0064] When determining a user's emotions by comprehensively considering physiological, behavioral, and linguistic characteristics, user stress is determined based on physiological and behavioral characteristics. Physiological characteristics such as heart rate, heart rate variability, skin conductance, respiratory rate, blood pressure, body temperature, and electroencephalogram (EEG) deviate from normal values ​​when a user is under significant stress. Behavioral characteristics such as limb twitching, trembling, and repetitive pacing are somatic manifestations of high stress. Therefore, combining physiological and behavioral characteristics allows for a more accurate assessment of whether a user's stress level is normal. Optionally, a database is stored in the memory, containing mapping relationships between various physiological and behavioral characteristics and stress. By accessing the database, the data processing unit can determine not only whether the user is normal but also their stress level.

[0065] Determining user tone based on linguistic features. Linguistic features include: sentence structure, intonation, speech rate, volume, and timbre. Sentences refer to the content of the user's conversation. Intonation, speech rate, volume, and timbre can reflect a user's emotions. For example, when a user is angry, their tone is usually sharp, their speech is rapid and fragmented, their volume is significantly increased with plosive sounds, and their timbre trembles. When a user is happy, their tone is usually light and cheerful, their speech is slightly faster but coherent, their volume is moderately loud with resonance, and their timbre is clear. It can be seen that these features are somewhat similar in expressing user emotions; therefore, combining these features with sentence structure can more accurately determine a user's tone.

[0066] In this way, by combining user stress and tone of voice, and based on the comprehensive expression of both internal and external emotions, user emotions can be determined more accurately.

[0067] Combination Figure 4 As shown in the embodiments of this disclosure, another interaction method for an output terminal is provided, including:

[0068] S101, the data processing unit acquires the user's physiological characteristics, behavioral characteristics and language characteristics.

[0069] S102, the data processing unit determines the user's emotions based on physiological characteristics, behavioral characteristics, and language characteristics.

[0070] S113, the data processing unit determines the target environmental atmosphere based on the user's emotions.

[0071] S123, the data processing unit determines the target output terminal and the target operation plan of the target output terminal based on the target environment atmosphere.

[0072] S133, the data processing unit controls the target output terminal to operate according to the corresponding target operation plan.

[0073] When controlling the output terminal, the target environment atmosphere is first determined based on the user's emotions. A suitable environment atmosphere can soothe and heal users with negative emotions, and can also resonate with, promote, and encourage users with positive emotions. Then, based on the target environment atmosphere, the target output terminal to be operated and the target operation plan for the target output terminal are determined.

[0074] Optionally, when a user is feeling down, the smart air conditioner automatically raises the indoor temperature, the smart speaker automatically plays warm and soothing music, the smart lighting automatically emits warm light, and the smart curtains automatically open to let sunlight into the room, creating a warm and comfortable atmosphere to help the user alleviate their low mood.

[0075] Optionally, when a user is agitated, the smart air conditioner automatically lowers the indoor temperature, the smart speaker automatically plays soft, calming music, the smart lighting automatically emits cool light, and the smart curtains automatically close to block out external disturbances, creating a quiet and cool environment that helps the user calm down.

[0076] In this way, the target output terminals are controlled to operate according to the corresponding target operation plan, enabling one or more target output terminals to cooperate and form a seamless connection and linkage of smart output terminals throughout the house. This allows for the creation of an environment conducive to emotional guidance based on the user's current emotional state. It is evident that accurately determining the target environmental atmosphere is a key step in controlling the operation of the output terminals, facilitating precise interaction between the user and the output terminals.

[0077] Optionally, if the smart TV is used as the main body for collecting data, processing data, and determining emotions, then when the target output terminal includes not only the smart TV itself but also home appliances such as smart air conditioners and smart curtains, the data processing unit of the smart TV will send the target operation plan to the corresponding target output terminal so that the target output terminal can run.

[0078] Combination Figure 5 As shown in S113, the data processing unit determines the target environmental atmosphere based on the user's emotions, including:

[0079] S1113, the data processing unit analyzes the user's emotions to obtain the emotion type and acquires the user's historical emotional data.

[0080] S1213, the data processing unit determines the cause of the user's emotions based on historical sentiment data.

[0081] S1313, the data processing unit determines the target environmental atmosphere based on the type of user emotion and the reason for the user's emotion.

[0082] When determining the target environment atmosphere, the user's emotions are first analyzed to determine the emotion type, whether the user's emotion is negative or positive, and what kind of negative / positive emotion it is. Optionally, negative emotions include: anger, sadness, fear, disgust, anxiety, guilt, shame, jealousy, etc. Positive emotions include: happiness, joy, satisfaction, love, hope, pride, gratitude, tranquility, etc. Simultaneously, the user's historical emotional data stored in the memory is retrieved. Optionally, historical emotional data is obtained based on the analysis of events actively uploaded by the user in historical time or automatically collected by the mobile terminal. For example, analyzing events such as "the notepad shows a lot of work today," "the enterprise work software shows overtime today," and "the map shows a lot of trips" can yield the corresponding emotional data as "fatigue." Similarly, analyzing events such as "the electronic calendar shows an outing today" and "the electronic calendar shows a dinner party today" can yield the corresponding emotional data as "happy." Optionally, historical emotional data is obtained based on the analysis of acquired user physiological characteristics, behavioral characteristics, and language characteristics.

[0083] Then, the causes of user emotions are determined based on historical sentiment data. Optionally, when historical sentiment data is obtained based on event analysis, the corresponding causes can be directly inferred from the historical sentiment data. For example, "working overtime" can be used to infer that the user's fatigue is caused by excessive workload, and "having an argument with someone" can be used to infer that the user's fatigue is caused by tense interpersonal relationships. Optionally, when historical sentiment data is obtained based on the analysis of user biometrics, the user's historical life experiences within the corresponding historical time period can be further obtained, and the causes of user emotions can be determined by combining historical life experiences with historical sentiment data.

[0084] Finally, based on the user's emotional type and the reason for that emotion, the target environmental atmosphere is determined. Optionally, if the user's emotional type is fatigue and the reason is work, the target environmental atmosphere is determined to be a sunny and tranquil atmosphere. If the user's emotional type is bored and the reason is unsuitable temperature and humidity, the target environmental atmosphere is determined to be a dehumidifying / humidifying / warming / cool atmosphere. Optionally, if the user's emotional type is happiness and the reason is passing an exam, the target environmental atmosphere is determined to be a vibrant and celebratory atmosphere. It should be noted that the memory stores many more combinations of emotional types and their reasons for being mapped to environmental atmospheres, and is not limited to the examples above.

[0085] In this way, by combining the type of user emotion and the reason for that emotion, the target environment atmosphere can be obtained more accurately, thereby improving the user's interactive experience with the output terminal.

[0086] Combination Figure 6 As shown in the embodiments of this disclosure, another interaction method for an output terminal is provided, including:

[0087] S101, the data processing unit acquires the user's physiological characteristics, behavioral characteristics and language characteristics.

[0088] S102, the data processing unit determines the user's emotions based on physiological characteristics, behavioral characteristics, and language characteristics.

[0089] S143, the data processing unit determines the target output terminal and the intended use of the target device based on the user's instructions.

[0090] S153, the data processing unit determines the target operation plan of the target output terminal based on the user's emotions and usage intentions.

[0091] When the control output terminal is running, if the user issues a control command, such as "I want to watch a movie," the system first determines the target output terminal and the user's intention to use the target terminal device based on the user's command. Then, it controls the target output terminal's operation plan based on the user's mood and usage intention. For example, if the user command is "I want to watch a movie," the target output terminal can be determined to be a smart TV. Simultaneously, if it's analyzed that the user is in a bad mood today, the system controls the smart TV to prioritize recommending movies that can bring a pleasant mood, or to insert humorous jokes into the smart dialogue with the smart TV to alleviate the user's negative emotions. As another example, if the user command is "I want to listen to music," the target output terminal can be determined to be a smart speaker. Simultaneously, if it's analyzed that the user is in a good mood today, the system controls the smart speaker to play upbeat or rhythmic music, or to express the user's desire to share happiness in the smart dialogue with the smart speaker.

[0092] The content output by this terminal can both satisfy the user's instructions and effectively maintain the user's mood, creating a suitable environment for the user in many ways.

[0093] Optionally, when a user actively searches for programs, the output terminal will take into account the user's interests and current emotional state, and intelligently recommend programs that match the user's mood.

[0094] Combination Figure 7 As shown in the embodiments of this disclosure, another interaction method for an output terminal is provided, including:

[0095] S101, the data processing unit acquires the user's physiological characteristics, behavioral characteristics and language characteristics.

[0096] S102, the data processing unit determines the user's emotions based on physiological characteristics, behavioral characteristics, and language characteristics.

[0097] S103, the data processing unit controls the operation of the output terminal based on the user's emotions or based on the user's emotions and instructions.

[0098] S104, the data processing unit obtains user feedback information.

[0099] S105, the data processing unit evaluates the user's satisfaction with and effectiveness of the output terminal operation based on the feedback information.

[0100] S106, the data processing unit optimizes the sentiment analysis strategy and operation strategy of the output terminal based on the satisfaction level and effectiveness.

[0101] After the output terminal is running, real-time user feedback is acquired. Optionally, the feedback is voice information, such as the user saying "Change the program." Optionally, the feedback is micro-expression information, such as frowning or pursing lips, which indicate user dissatisfaction with the output content. Conversely, smiling or nodding indicates satisfaction. Based on user feedback, the system evaluates metrics such as user satisfaction and effectiveness with the output content, thereby gaining a deeper understanding of user preferences for the current program. This allows for optimization of the output terminal's emotion analysis and operational strategies, providing accurate data support for subsequent content recommendations. Thus, user feedback can improve the accuracy of user emotion recognition and the quality of the output terminal's response.

[0102] Optionally, advanced natural language processing technology and sentiment analysis algorithms can be used to accurately identify the emotional state of the user during the communication process with the output terminal.

[0103] Combination Figure 8 As shown in the embodiments of this disclosure, another interaction method for an output terminal is provided, including:

[0104] S101, the data processing unit acquires the user's physiological characteristics, behavioral characteristics and language characteristics.

[0105] S102, the data processing unit determines the user's emotions based on physiological characteristics, behavioral characteristics, and language characteristics.

[0106] S103, the data processing unit controls the operation of the output terminal based on the user's emotions or based on the user's emotions and instructions.

[0107] S107, after executing S102, the data processing unit determines the user's identity among family members.

[0108] S108, the data processing unit sends emotional cues and communication strategies to other family members based on the user's emotions and identity.

[0109] While ensuring user privacy and security, the storage device stores family member information provided by the user, including relationships and contact information. This stored information is then used to intelligently build a family network, clearly displaying the relationship structure among family members.

[0110] After determining the user's emotions, the system identifies the user's role within the family, such as elders, parents, children, seniors, or children. When abnormal emotions or potential conflicts arise among family members, the data processing unit immediately activates an early warning mechanism. By analyzing communication between family members, the system assesses the nature and severity of the conflict, generates emotional alerts and communication strategies for the user, such as communication skills and psychological support. These alerts and strategies are then promptly sent to other family members. This encourages family members to pay attention and provide care for the user. This timely interaction not only helps cultivate a more harmonious and warm family relationship but also effectively alleviates negative emotions through mutual care and support, promoting emotional exchange and harmony within the family.

[0111] Optionally, after determining the user's emotion, the data processing unit controls the display module of the output terminal to display the user's emotion, so as to show the recognition result to the user and enable the user to understand and grasp their own emotional state in a timely manner.

[0112] The following will briefly illustrate the application of various scenarios, using the example of data acquisition and data processing units mounted on an output terminal:

[0113] Scene 1:

[0114] User: Any good shows to watch lately?

[0115] Output terminal: (Collecting user emotions through devices such as cameras and microphones) Here we recommend XXXX for you.

[0116] User: (Watching)

[0117] Output terminal: (Detects user's facial expression indicating dislike for this type of content, collects relevant information, and stores the user's preferences)

[0118] User: Change it.

[0119] Output terminal: Okay, here is XXX (provide the corresponding content).

[0120] Scene 2:

[0121] Output terminal: (Detecting that the user is in a bad mood, and based on historical data analysis, it may be due to family conflicts) Proactively provides appropriate content.

[0122] User: (My mood has improved)

[0123] Output terminal: (Record data)

[0124] Scene 3:

[0125] Output terminal: (Detecting that the user is in a bad mood, based on historical data analysis, it may be due to recent weather or poor ambient temperature and humidity) Actively adjusts the smart home appliances in the home.

[0126] User: (My mood has improved)

[0127] Output terminal: (Record data)

[0128] Scene 4:

[0129] Output terminal: (Detecting that the user is in a bad mood) Provide relevant measures.

[0130] User: (Mood has not improved)

[0131] Output terminal: (Records data, detects that the current user's negative emotions are relatively severe, sends relevant information to the user's family members, and informs them of the cause of the incident)

[0132] Combination Figure 9 As shown, this embodiment of the disclosure provides an interactive device 90 for an output terminal, including: an acquisition module 91, a determination module 92, and a control module 93. The acquisition module 91 is configured to acquire the user's physiological characteristics, behavioral characteristics, and linguistic characteristics. The determination module 92 is configured to determine the user's emotion based on the physiological characteristics, behavioral characteristics, and linguistic characteristics. The control module 93 is configured to control the operation of the output terminal based on the user's emotion or based on the user's emotion and user instructions.

[0133] The interactive device 90 for the output terminal provided in this disclosure accurately determines the user's emotions based on physiological, behavioral, and linguistic characteristics that can express the user's external and potential emotions. Controlling the output terminal's operation based on the user's emotions enables the output terminal's operating scheme to match the user's emotions and facilitates sensory interaction between the output terminal and the user. Alternatively, controlling the output terminal's operation based on both user emotions and user commands ensures that the output terminal's operation both conforms to user instructions and accommodates the user's emotions. Through this intelligent environmental control, not only can emotion analysis results be provided, but proactive measures can also be taken to help users regulate their emotions, achieving a complete closed loop from analysis to intervention, bringing users a more considerate and comprehensive emotional care experience. This makes the interaction between the user and the output terminal more intelligent.

[0134] Combination Figure 10As shown, this disclosure provides an interactive device 100 for an output terminal, including a processor 101 and a memory 102. Optionally, the device 100 may further include a communication interface 103 and a bus 104. The processor 101, communication interface 103, and memory 102 can communicate with each other via the bus 104. The communication interface 103 can be used for information transmission. The processor 101 can call logical instructions in the memory 102 to execute the interactive method for the output terminal described in the above embodiment.

[0135] Furthermore, the logical instructions in the aforementioned memory 102 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0136] The memory 102, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 101 executes functional applications and data processing by running the program instructions / modules stored in the memory 102, thereby implementing the interactive method for outputting the terminal in the above embodiments.

[0137] The memory 102 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 102 may include high-speed random access memory and may also include non-volatile memory.

[0138] Combination Figure 11 As shown, this embodiment of the disclosure provides an output terminal 110, including: a product body, and the aforementioned interactive device 90 (100) for the output terminal. The interactive device 90 (100) for the output terminal is installed on the output terminal body. The installation relationship described herein is not limited to placement inside the output terminal body, but also includes installation and connection with other components of the output terminal 110, including but not limited to physical connection, electrical connection, or signal transmission connection. Those skilled in the art will understand that the interactive device 90 (100) for the output terminal can be adapted to feasible product bodies to achieve other feasible embodiments.

[0139] This disclosure provides a computer-readable storage medium storing computer-executable instructions configured to perform the above-described interactive method for an output terminal.

[0140] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the interactive method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., and other media capable of storing program code.

[0141] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used in this application means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0142] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0143] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. Furthermore, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0144] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. An interactive method for an output terminal, characterized in that, The interaction method includes: Obtain users' physiological, behavioral, and linguistic characteristics; Determine the user's emotions based on the physiological characteristics, behavioral characteristics, and linguistic characteristics; The output terminal is controlled to operate based on the user's emotions or based on the user's emotions and user instructions.

2. The interaction method according to claim 1, characterized in that, Determining user emotions based on the physiological characteristics, behavioral characteristics, and linguistic characteristics includes: Determine user stress based on the physiological and behavioral characteristics; Determine the user's tone based on the aforementioned language features; The user's emotion is determined based on the user's stress and tone of voice.

3. The interaction method according to claim 1, characterized in that, The step of controlling the operation of the output terminal based on the user's emotions includes: Determine the target environment atmosphere based on the user's emotions; Based on the target environment, determine the target output terminal and the target operation plan for the target output terminal; Control the target output terminal to operate according to the corresponding target operation scheme.

4. The interaction method according to claim 3, characterized in that, Determining the target environmental atmosphere based on the user's emotions includes: The user's emotions are analyzed to determine the emotion type, and the user's historical emotional data is obtained. The cause of the user's emotion is determined based on the historical sentiment data; The target environmental atmosphere is determined based on the type of user emotion and the reason for the occurrence of the user emotion.

5. The interaction method according to any one of claims 1 to 4, characterized in that, The step of controlling the operation of the output terminal based on the user's emotions and instructions includes: Based on the user instructions, the target output terminal and the intention to use the target device are determined; Based on the user's emotions and usage intentions, the target operation plan for the target output terminal is determined.

6. The interaction method according to any one of claims 1 to 4, characterized in that, After the control output terminal is activated, the interaction method further includes: Obtain user feedback information; Based on the feedback information, assess the user's satisfaction with and the effectiveness of the output terminal's operation; The sentiment analysis strategy and operation strategy of the output terminal are optimized based on the satisfaction level and the effectiveness.

7. The interaction method according to any one of claims 1 to 4, characterized in that, After determining the user's emotion, the interaction method further includes: Determine the user's identity within the family members; Based on the user's emotions and identity, send emotional cues and communication strategies to other family members.

8. An interactive device for an output terminal, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to, when executing the program instructions, perform the interactive method for an output terminal as described in any one of claims 1 to 7.

9. An output terminal, characterized in that, include: Output terminal body; The interactive device for an output terminal as described in claim 8 is installed on the output terminal body.

10. A computer-readable storage medium storing program instructions, characterized in that, When the program instructions are executed, they cause the computer to perform the interactive method for the output terminal as described in any one of claims 1 to 7.

Images